Skip to contentSkip to conversation

security: add software synchronous crypto process

RFC PATCH In review Fan Zhang 5 participants 87 messages Sep 3, 2019 – Nov 1, 2019 Page 1 of 2

Subsystems (2)
lore
Share & tools
lore

Fan Zhang

author
#0 lore raw

cover security: add software synchronous crypto process

This RFC patch adds a way to rte_security to process symmetric crypto workload in bulk synchronously for SW crypto devices.

Originally both SW and HW crypto PMDs works under rte_cryptodev to process the crypto workload asynchronously. This way provides uniformity to both PMD types but also introduce unnecessary performance penalty to SW PMDs such as extra SW ring enqueue/dequeue steps to "simulate" asynchronous working manner and unnecessary HW addresses computation.

We introduce a new way for SW crypto devices that perform crypto operation synchronously with only fields required for the computation as input. The proof-of-concept AESNI-GCM and AESNI-MB SW PMDs are updated with the support of this new method. To demonstrate the performance gain with this method 2 simple performance evaluation apps under unit-test are added "app/test: security_aesni_gcm_perftest/security_aesni_mb_perftest". The users can freely compare their results against crypto perf application results.

Fan Zhang (9):
security: introduce CPU Crypto action type and API crypto/aesni_gcm: add rte_security handler app/test: add security cpu crypto autotest app/test: add security cpu crypto perftest crypto/aesni_mb: add rte_security handler app/test: add aesni_mb security cpu crypto autotest app/test: add aesni_mb security cpu crypto perftest
ipsec: add rte_security cpu_crypto action support examples/ipsec-secgw: add security cpu_crypto action support

 app/test/Makefile                                  |    1 +
 app/test/meson.build                               |    1 +
 app/test/test_security_cpu_crypto.c                | 1326 ++++++++++++++++++++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c           |   91 +-
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c       |   95 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h   |   23 +
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  291 ++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c     |   91 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |   21 +-
 examples/ipsec-secgw/ipsec.c                       |   22 +
 examples/ipsec-secgw/ipsec_process.c               |    4 +-
 examples/ipsec-secgw/sa.c                          |   13 +-
 examples/ipsec-secgw/test/run_test.sh              |   10 +
 .../test/trs_3descbc_sha1_cpu_crypto_defs.sh       |    5 +
 .../test/trs_aescbc_sha1_cpu_crypto_defs.sh        |    5 +
 .../test/trs_aesctr_sha1_cpu_crypto_defs.sh        |    5 +
 .../ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh |    5 +
 .../test/trs_aesgcm_mb_cpu_crypto_defs.sh          |    7 +
 .../test/tun_3descbc_sha1_cpu_crypto_defs.sh       |    5 +
 .../test/tun_aescbc_sha1_cpu_crypto_defs.sh        |    5 +
 .../test/tun_aesctr_sha1_cpu_crypto_defs.sh        |    5 +
 .../ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh |    5 +
 .../test/tun_aesgcm_mb_cpu_crypto_defs.sh          |    7 +
 lib/librte_ipsec/esp_inb.c                         |  174 ++-
 lib/librte_ipsec/esp_outb.c                        |  290 ++++-
 lib/librte_ipsec/sa.c                              |   53 +-
 lib/librte_ipsec/sa.h                              |   29 +
 lib/librte_ipsec/ses.c                             |    4 +-
 lib/librte_security/rte_security.c                 |   16 +
 lib/librte_security/rte_security.h                 |   51 +-
 lib/librte_security/rte_security_driver.h          |   19 +
 lib/librte_security/rte_security_version.map       |    1 +
 32 files changed, 2658 insertions(+), 22 deletions(-)
 create mode 100644 app/test/test_security_cpu_crypto.c
 create mode 100644 examples/ipsec-secgw/test/trs_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_mb_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_mb_cpu_crypto_defs.sh
Signature · 1 line
2.14.5
author
#1 lore raw

RFC PATCH 1/9 security: introduce CPU Crypto action type and API

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

---
 lib/librte_security/rte_security.c           | 16 +++++++++
 lib/librte_security/rte_security.h           | 51 +++++++++++++++++++++++++++-
 lib/librte_security/rte_security_driver.h    | 19 +++++++++++
 lib/librte_security/rte_security_version.map |  1 +
 4 files changed, 86 insertions(+), 1 deletion(-)
lib/librte_security/rte_security.c +16 -0
index bc81ce15d..0f85c1b59 100644
@@ -141,3 +141,19 @@ rte_security_capability_get(struct rte_security_ctx *instance,
 
 	return NULL;
 }
+
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	uint32_t i;
+
+	for (i = 0; i < num; i++)
+		status[i] = -1;
+
+	RTE_FUNC_PTR_OR_RET(*instance->ops->process_cpu_crypto_bulk);
+	instance->ops->process_cpu_crypto_bulk(sess, buf, iv,
+			aad, digest, status, num);
+}
lib/librte_security/rte_security.h +50 -1

84 lines not on this page · raw message

lib/librte_security/rte_security_driver.h +19 -0
index 1b561f852..70fcb0c26 100644
@@ -132,6 +132,23 @@ typedef int (*security_get_userdata_t)(void *device,
 typedef const struct rte_security_capability *(*security_capabilities_get_t)(
 		void *device);
 
+/**
+ * Process security operations in bulk using CPU accelerated method.
+ *
+ * @param	sess		Security session structure.
+ * @param	buf		Buffer to the vectors to be processed.
+ * @param	iv		IV pointers.
+ * @param	aad		AAD pointers.
+ * @param	digest		Digest pointers.
+ * @param	status		Array of status value.
+ * @param	num		Number of elements in each array.
+ */
+
+typedef void (*security_process_cpu_crypto_bulk_t)(
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 /** Security operations function pointer table */
 struct rte_security_ops {
 	security_session_create_t session_create;
@@ -150,6 +167,8 @@ struct rte_security_ops {
 	/**< Get userdata associated with session which processed the packet. */
 	security_capabilities_get_t capabilities_get;
 	/**< Get security capabilities. */
+	security_process_cpu_crypto_bulk_t process_cpu_crypto_bulk;
+	/**< Process data in bulk. */
 };
 
 #ifdef __cplusplus
lib/librte_security/rte_security_version.map +1 -0
index 53267bf3c..2132e7a00 100644
@@ -18,4 +18,5 @@ EXPERIMENTAL {
 	rte_security_get_userdata;
 	rte_security_session_stats_get;
 	rte_security_session_update;
+	rte_security_process_cpu_crypto_bulk;
 };
Signature · 1 line
2.14.5
#2 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 7 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action type to
security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could you please elaborate the flow in the documentation. If not in documentation right now, then please elaborate the flow in cover letter.
Also I see that there are new APIs for processing crypto operations in bulk. What does that mean. How are they different from the existing APIs which are also handling bulk crypto ops depending on the budget.

-Akhil

#3 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

This action type allows the burst of symmetric crypto workload using the same algorithm, key, and direction being processed by CPU cycles synchronously. This flexible action type does not require external hardware involvement, having the crypto workload processed synchronously, and is more performant than Cryptodev SW PMD due to the saved cycles on removed "async mode simulation" as well as 3 cacheline access of the crypto ops.

AESNI-GCM and AESNI-MB PMDs are updated with this support. There is a small performance test app under app/test/security_aesni_gcm(mb)_perftest to prove.

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the session data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error number assigned to the appropriate index of the status array.

Will update the program's guide in the v1 patch.

Regards,
Fan

Quoted 28 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty, Declan <[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 2 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

action

Quoted 4 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could you please elaborate the flow in the documentation. If not in documentation right now, then please elaborate the flow in cover letter.
Also I see that there are new APIs for processing crypto operations in bulk. What does that mean. How are they different from the existing APIs which are also handling bulk crypto ops depending on the budget.

-Akhil

#4 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 9 lines

Hi Akhil,

This action type allows the burst of symmetric crypto workload using the same algorithm, key, and direction being processed by CPU cycles synchronously. This flexible action type does not require external hardware involvement, having the crypto workload processed synchronously, and is more performant than Cryptodev SW PMD due to the saved cycles on removed "async mode simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst and corresponding dequeue burst.
It would be a new API something like process_packets and it will have the crypto processed packets while returning from the API?

I still do not understand why we cannot do with the conventional crypto lib only. As far as I can understand, you are not doing any protocol processing or any value add To the crypto processing. IMO, you just need a synchronous crypto processing API which Can be defined in cryptodev, you don't need to re-create a crypto session in the name of Security session in the driver just to do a synchronous processing.

Quoted 46 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support. There is a small performance test app under app/test/security_aesni_gcm(mb)_perftest to prove.

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the session data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error number assigned to the appropriate index of the status array.

Will update the program's guide in the v1 patch.

Regards,
Fan

Quoted 28 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty, Declan <[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 2 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

action

Quoted 4 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could you please elaborate the flow in the documentation. If not in documentation right now, then please elaborate the flow in cover letter.
Also I see that there are new APIs for processing crypto operations in bulk. What does that mean. How are they different from the existing APIs which are also handling bulk crypto ops depending on the budget.

-Akhil

#5 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

You are right, the new API will process the crypto workload, no heavy enqueue Dequeue operations required.

Cryptodev tends to support multiple crypto devices, including HW and SW. The 3-cache line access, iova address computation and assignment, simulation of async enqueue/dequeue operations, allocate and free crypto ops, even the mbuf linked-list for scatter-gather buffers are too heavy for SW crypto PMDs.

To create this new synchronous API in cryptodev cannot avoid the problem listed above: first the API shall not serve only to part of the crypto (SW) PMDs - as you know, it is Cryptodev. The users can expect some PMD only support part of the overall algorithms, but not the workload processing API.

Another reason is, there is assumption made, first when creating a crypto op we have to allocate the memory to hold crypto op + sym op + iv, - we cannot simply declare an array of crypto ops in the run-time and discard it when processing is done. Also we need to fill aad and digest HW address, which is not required for SW at all.

Bottom line: using crypto op will still have 3 cache-line access performance problem.

So if we to create the new API in Cryptodev instead of rte_security, we need to create new crypto op structure only for the SW PMDs, carefully document them to not confuse with existing cryptodev APIs, make new device feature flags to indicate the API is not supported by some PMDs, and again carefully document them of these device feature flags.

So, to push these changes to rte_security instead the above problem can be resolved, and the performance improvement because of this change is big for smaller packets
- I attached a performance test app in the patchset.

For rte_security, we already have inline-crypto type that works quite close to the this new API, the only difference is that it is processed by the CPU cycles. As you may have already seen the ipsec-library has wrapped these changes, and ipsec-secgw has only minimum updates to adopt this change too. So to the end user, if they use IPSec this patchset can seamlessly enabled with just commandline update when creating an SA.

Regards,
Fan

Quoted 85 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Friday, September 6, 2019 10:01 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty, Declan <[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 5 lines

Hi Akhil,

This action type allows the burst of symmetric crypto workload using the same algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 4 lines

This flexible action type does not require external hardware involvement, having the crypto workload processed synchronously, and is more performant than Cryptodev SW PMD due to the saved cycles on removed "async mode simulation" as well as 3 cacheline access of the

crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst and corresponding dequeue burst.
It would be a new API something like process_packets and it will have the crypto processed packets while returning from the API?

I still do not understand why we cannot do with the conventional crypto lib only.
As far as I can understand, you are not doing any protocol processing or any value add To the crypto processing. IMO, you just need a synchronous crypto processing API which Can be defined in cryptodev, you don't need to re- create a crypto session in the name of Security session in the driver just to do a synchronous processing.

Quoted 5 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support. There is a small performance test app under app/test/security_aesni_gcm(mb)_perftest to prove.

Quoted 7 lines

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the session data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error number assigned to the appropriate index of the status

array.

Quoted 29 lines

Will update the program's guide in the v1 patch.

Regards,
Fan

Quoted 23 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty, Declan <[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 2 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

action

Quoted 4 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could you please elaborate the flow in the documentation. If not in documentation right now, then please elaborate the flow in cover letter. Also I see that there are new APIs for processing crypto operations in

bulk.

Quoted 5 lines
Quoted 5 lines

What does that mean. How are they different from the existing APIs which are also handling bulk crypto ops depending on the budget.

-Akhil

#6 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 10 lines

Hi Akhil,

You are right, the new API will process the crypto workload, no heavy enqueue Dequeue operations required.

Cryptodev tends to support multiple crypto devices, including HW and SW. The 3-cache line access, iova address computation and assignment, simulation of async enqueue/dequeue operations, allocate and free crypto ops, even the mbuf linked-list for scatter-gather buffers are too heavy for SW crypto PMDs.

Why cant we have a cryptodev synchronous API which work on plain bufs as your suggested API and use the same crypto sym_session creation logic as it was before? It will perform same as it is doing in this series.

Quoted 5 lines

To create this new synchronous API in cryptodev cannot avoid the problem listed above: first the API shall not serve only to part of the crypto (SW) PMDs - as you know, it is Cryptodev. The users can expect some PMD only support part of the overall algorithms, but not the workload processing API.

Why cant we have an optional data path in cryptodev for synchronous behavior if the underlying PMD support it. It depends on the PMD to decide whether it can have it supported or not. Only a feature flag will be needed to decide that.
One more option could be a PMD API which the application can directly call if the mode is only supported in very few PMDs. This could be a backup if there is a requirement of deprecation notice etc.

Quoted 7 lines

Another reason is, there is assumption made, first when creating a crypto op we have to allocate the memory to hold crypto op + sym op + iv, - we cannot simply declare an array of crypto ops in the run-time and discard it when processing
is done. Also we need to fill aad and digest HW address, which is not required for SW at all.

We are defining a new API which may have its own parameters and requirements which Need to be fulfilled. In case it was a rte_security API, then also you are defining a new way Of packet execution and API params. So it would be same.
You can reduce the cache line accesses as you need in the new API.
The session logic need not be changed from crypto session to security session. Only the data patch need to be altered as per the new API.

Quoted 9 lines

Bottom line: using crypto op will still have 3 cache-line access performance problem.

So if we to create the new API in Cryptodev instead of rte_security, we need to create new crypto op structure only for the SW PMDs, carefully document them to not confuse with existing cryptodev APIs, make new device feature flags to indicate the API is not supported by some PMDs, and again carefully document them of these device feature flags.

The explanation of the new API will also happen in case it is a security API. Instead you need to add more explanation for session also which is already there in cryptodev.

Quoted 6 lines

So, to push these changes to rte_security instead the above problem can be resolved,
and the performance improvement because of this change is big for smaller packets
- I attached a performance test app in the patchset.

I believe there wont be any perf gap in case the optimized new cryptodev API is used.

Quoted 9 lines

For rte_security, we already have inline-crypto type that works quite close to the this
new API, the only difference is that it is processed by the CPU cycles. As you may have already seen the ipsec-library has wrapped these changes, and ipsec-secgw has only minimum updates to adopt this change too. So to the end user, if they use IPSec this patchset can seamlessly enabled with just commandline update when
creating an SA.

In the IPSec application I do not see the changes wrt the new execution API. So the data path is not getting handled there. It looks incomplete. The user experience to use the new API will definitely be changed.

So I believe this patchset is not required in rte_security, we can have it in cryptodev unless I have missed something.

Quoted 90 lines

Regards,
Fan

Quoted 85 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Friday, September 6, 2019 10:01 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty, Declan <[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 5 lines

Hi Akhil,

This action type allows the burst of symmetric crypto workload using the same algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 4 lines

This flexible action type does not require external hardware involvement, having the crypto workload processed synchronously, and is more performant than Cryptodev SW PMD due to the saved cycles on removed "async mode simulation" as well as 3 cacheline access of the

crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst and corresponding dequeue burst.
It would be a new API something like process_packets and it will have the crypto processed packets while returning from the API?

I still do not understand why we cannot do with the conventional crypto lib only.
As far as I can understand, you are not doing any protocol processing or any value add To the crypto processing. IMO, you just need a synchronous crypto processing API which Can be defined in cryptodev, you don't need to re- create a crypto session in the name of Security session in the driver just to do a synchronous processing.

Quoted 5 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support. There is a small performance test app under app/test/security_aesni_gcm(mb)_perftest to prove.

Quoted 7 lines

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the session data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error number assigned to the appropriate index of the status

array.

Quoted 29 lines

Will update the program's guide in the v1 patch.

Regards,
Fan

Quoted 23 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty, Declan <[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 2 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

action

Quoted 4 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could you please elaborate the flow in the documentation. If not in documentation right now, then please elaborate the flow in cover letter. Also I see that there are new APIs for processing crypto operations in

bulk.

Quoted 5 lines
Quoted 5 lines

What does that mean. How are they different from the existing APIs which are also handling bulk crypto ops depending on the budget.

-Akhil

#7 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi lads,

Quoted 4 lines
Quoted 4 lines

You are right, the new API will process the crypto workload, no heavy enqueue Dequeue operations required.

Quoted 8 lines
Quoted 4 lines

Cryptodev tends to support multiple crypto devices, including HW and SW. The 3-cache line access, iova address computation and assignment, simulation of async enqueue/dequeue operations, allocate and free crypto ops, even the mbuf linked-list for scatter-gather buffers are too heavy for SW crypto PMDs.

Why cant we have a cryptodev synchronous API which work on plain bufs as your suggested API and use the same crypto sym_session creation logic as it was before? It will perform same as it is doing in this series.

I tried to summarize our reasons in another mail in that thread.

Quoted 61 lines
Quoted 6 lines

To create this new synchronous API in cryptodev cannot avoid the problem listed above: first the API shall not serve only to part of the crypto (SW) PMDs - as you know, it is Cryptodev. The users can expect some PMD only support part of the overall algorithms, but not the workload processing API.

Why cant we have an optional data path in cryptodev for synchronous behavior if the underlying PMD support it. It depends on the PMD to decide whether it can have it supported or not. Only a feature flag will be needed to decide that.
One more option could be a PMD API which the application can directly call if the mode is only supported in very few PMDs. This could be a backup if there is a requirement of deprecation notice etc.

Quoted 7 lines

Another reason is, there is assumption made, first when creating a crypto op we have to allocate the memory to hold crypto op + sym op + iv, - we cannot simply declare an array of crypto ops in the run-time and discard it when processing
is done. Also we need to fill aad and digest HW address, which is not required for SW at all.

We are defining a new API which may have its own parameters and requirements which Need to be fulfilled. In case it was a rte_security API, then also you are defining a new way Of packet execution and API params. So it would be same.
You can reduce the cache line accesses as you need in the new API.
The session logic need not be changed from crypto session to security session. Only the data patch need to be altered as per the new API.

Quoted 9 lines

Bottom line: using crypto op will still have 3 cache-line access performance problem.

So if we to create the new API in Cryptodev instead of rte_security, we need to create new crypto op structure only for the SW PMDs, carefully document them to not confuse with existing cryptodev APIs, make new device feature flags to indicate the API is not supported by some PMDs, and again carefully document them of these device feature flags.

The explanation of the new API will also happen in case it is a security API. Instead you need to add more explanation for session also which is already there in cryptodev.

Quoted 6 lines

So, to push these changes to rte_security instead the above problem can be resolved,
and the performance improvement because of this change is big for smaller packets
- I attached a performance test app in the patchset.

I believe there wont be any perf gap in case the optimized new cryptodev API is used.

Quoted 9 lines

For rte_security, we already have inline-crypto type that works quite close to the this
new API, the only difference is that it is processed by the CPU cycles. As you may have already seen the ipsec-library has wrapped these changes, and ipsec-secgw has only minimum updates to adopt this change too. So to the end user, if they use IPSec this patchset can seamlessly enabled with just commandline update when
creating an SA.

Quoted 3 lines

In the IPSec application I do not see the changes wrt the new execution API. So the data path is not getting handled there. It looks incomplete. The user experience to use the new API will definitely be changed.

I believe we do support it for libtre_ipsec mode. librte_ipsec hides all processing complexity inside and does call rte_security_process_cpu_crypto_bulk() internally. That's why for librte_ipsec it is literally 2 lines change:

examples/ipsec-secgw/ipsec_process.c +4 -3
@@ -101,7 +101,8 @@  fill_ipsec_session(struct rte_ipsec_session *ss, struct ipsec_ctx *ctx,
 		}
 		ss->crypto.ses = sa->crypto_session;
 	/* setup session action type */
-	} else if (sa->type == RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL) {
+	} else if (sa->type == RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL ||
+			sa->type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
 		if (sa->sec_session == NULL) {
 			rc = create_lookaside_session(ctx, sa);
 			if (rc != 0)
@@ -227,8 +228,8 @@  ipsec_process(struct ipsec_ctx *ctx, struct ipsec_traffic *trf)
 
 		/* process packets inline */
 		else if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO ||
-				sa->type ==
-				RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL) {
+			sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL ||
+			sa->type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
 
 			satp = rte_ipsec_sa_type(ips->sa);
 
#8 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

In reply to Akhil Goyal

Hi Akhil,

Quoted 8 lines
Quoted 6 lines

This action type allows the burst of symmetric crypto workload using the same algorithm, key, and direction being processed by CPU cycles synchronously. This flexible action type does not require external hardware involvement, having the crypto workload processed synchronously, and is more performant than Cryptodev SW PMD due to the saved cycles on removed "async mode simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst and corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will have the crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers, not mbufs.

Quoted 6 lines

I still do not understand why we cannot do with the conventional crypto lib only. As far as I can understand, you are not doing any protocol processing or any value add To the crypto processing. IMO, you just need a synchronous crypto processing API which Can be defined in cryptodev, you don't need to re-create a crypto session in the name of Security session in the driver just to do a synchronous processing.

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead? The main reason is that would require disruptive changes in existing cryptodev API (would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need some extra information that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra in future). Also right now there is no way to add new type of crypto_sym_session without either breaking existing crypto-dev ABI/API or introducing new structure (rte_crypto_sym_cpu_session or so) for that.
While rte_security is designed in a way that we can add new session types and related parameters without causing API/ABI breakage.

BTW, what is your concern with proposed approach (via rte_security)? From my perspective it is a lightweight change and it is totally optional for the crypto PMDs to support it or not.
Konstantin

Quoted 46 lines
Quoted 46 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support. There is a small performance test app under app/test/security_aesni_gcm(mb)_perftest to prove.

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the session data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error number assigned to the appropriate index of the status array.

Will update the program's guide in the v1 patch.

Regards,
Fan

Quoted 28 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty, Declan <[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 2 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

action

Quoted 4 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could you please elaborate the flow in the documentation. If not in documentation right now, then please elaborate the flow in cover letter.
Also I see that there are new APIs for processing crypto operations in bulk. What does that mean. How are they different from the existing APIs which are also handling bulk crypto ops depending on the budget.

-Akhil

#9 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 33 lines

Hi Akhil,

Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto workload using the

same

Quoted 3 lines
Quoted 3 lines

algorithm, key, and direction being processed by CPU cycles synchronously. This flexible action type does not require external hardware involvement, having the crypto workload processed synchronously, and is more

performant

Quoted 4 lines
Quoted 2 lines

than Cryptodev SW PMD due to the saved cycles on removed "async mode simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst and

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will have the

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers, not mbufs.

Quoted 2 lines

I still do not understand why we cannot do with the conventional crypto lib

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol processing or any

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous crypto processing

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a crypto session in

the name of

Quoted 1 line

Security session in the driver just to do a synchronous processing.

Quoted 9 lines

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in existing cryptodev API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need some extra information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra in future).

Cipher offset will be part of rte_crypto_op. If you intend not to use rte_crypto_op You can pass this as an argument in the new cryptodev API.
Something extra will also cause ABI breakage in security as well.
So it will be same.

Quoted 3 lines

Also right now there is no way to add new type of crypto_sym_session without either breaking existing crypto-dev ABI/API or introducing new structure (rte_crypto_sym_cpu_session or so) for that.

What extra info is required in rte_cryptodev_sym_session to get the rte_crypto_sym_cpu_session. I don't think there is any.
I believe the same crypto session will be able to work synchronously as well. We would only need a new API to perform synchronous actions. That will reduce the duplication code significantly in the driver to support 2 different kind of APIs with similar code inside. Please correct me in case I am missing something.

Quoted 2 lines

While rte_security is designed in a way that we can add new session types and related parameters without causing API/ABI breakage.

Yes the intent is to add new sessions based on various protocols that can be supported by the driver. It is not that we should find it as an alternative to cryptodev and using it just because it will not cause ABI/API breakage. IMO the code should be placed where its intent is.

Quoted 57 lines

BTW, what is your concern with proposed approach (via rte_security)? From my perspective it is a lightweight change and it is totally optional for the crypto PMDs to support it or not.
Konstantin

Quoted 2 lines
Quoted 2 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support. There is a

small

Quoted 6 lines
Quoted 6 lines

performance test app under app/test/security_aesni_gcm(mb)_perftest to prove.

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the

session

Quoted 3 lines
Quoted 3 lines

data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error

number

Quoted 4 lines
Quoted 4 lines

assigned to the appropriate index of the status array.

Will update the program's guide in the v1 patch.

Quoted 8 lines
Quoted 8 lines

Regards,
Fan

Quoted 5 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty,

Declan

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

<[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type

and

Quoted 20 lines
Quoted 20 lines
Quoted 20 lines

API

Hi Fan,

Quoted 2 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

action

Quoted 4 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could you please elaborate the flow in the documentation. If not in documentation right now, then please elaborate the flow in cover letter.
Also I see that there are new APIs for processing crypto operations in bulk. What does that mean. How are they different from the existing APIs which are also handling bulk crypto ops depending on the budget.

-Akhil

#10 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Quoted 41 lines
Quoted 31 lines
Quoted 2 lines
Quoted 2 lines

This action type allows the burst of symmetric crypto workload using the

same

Quoted 3 lines
Quoted 3 lines

algorithm, key, and direction being processed by CPU cycles synchronously. This flexible action type does not require external hardware involvement, having the crypto workload processed synchronously, and is more

performant

Quoted 4 lines
Quoted 2 lines

than Cryptodev SW PMD due to the saved cycles on removed "async mode simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst and

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will have the

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers, not mbufs.

Quoted 2 lines

I still do not understand why we cannot do with the conventional crypto lib

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol processing or any

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous crypto processing

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a crypto session in

the name of

Quoted 1 line

Security session in the driver just to do a synchronous processing.

Quoted 9 lines

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in existing cryptodev API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need some extra information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra in future).

Quoted 1 line

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown current crypto-op approach. That's why the general idea - have all data that wouldn't change from packet to packet included into the session and setup it once at session_init().

Quoted 2 lines

If you intend not to use rte_crypto_op
You can pass this as an argument in the new cryptodev API.

You mean extra parameter in rte_security_process_cpu_crypto_bulk()?
It can be in theory, but that solution looks a bit ugly:
why to pass for each call something that would be constant per session? Again having that value constant per session might allow some extra optimisations That would be hard to achieve for dynamic case.
and not extendable:
Suppose tomorrow will need to add something extra (some new algorithm support or so). With what you proposing will need to new parameter to the function,
which means API breakage.

Quoted 2 lines

Something extra will also cause ABI breakage in security as well. So it will be same.

I don't think it would.
AFAIK, right now this patch doesn't introduce any API/ABI breakage. Iinside struct rte_security_session_conf we have a union of xforms depending on session type.
So as long as cpu_crypto_xform wouldn't exceed sizes of other xform - I believe no ABI breakage will appear.

Quoted 6 lines
Quoted 4 lines

Also right now there is no way to add new type of crypto_sym_session without either breaking existing crypto-dev ABI/API or introducing new structure (rte_crypto_sym_cpu_session or so) for that.

What extra info is required in rte_cryptodev_sym_session to get the rte_crypto_sym_cpu_session.

Right now - just cipher_offset (see above).
What else in future (if any) - don't know.

Quoted 2 lines

I don't think there is any.
I believe the same crypto session will be able to work synchronously as well.

Exactly the same - problematically, see above.

Quoted 4 lines

We would only need a new API to perform synchronous actions. That will reduce the duplication code significantly
in the driver to support 2 different kind of APIs with similar code inside. Please correct me in case I am missing something.

To add new API into crypto-dev would also require changes in the PMD,
it wouldn't come totally free and I believe would require roughly the same amount of changes.

Quoted 6 lines
Quoted 4 lines

While rte_security is designed in a way that we can add new session types and related parameters without causing API/ABI breakage.

Yes the intent is to add new sessions based on various protocols that can be supported by the driver.

Various protocols and different types of sessions (and devices they belong to). Let say right now we have INLINE_CRYPTO, INLINE_PROTO, LOOKASIDE_PROTO, etc. Here we introduce new type of session.

Quoted 2 lines

It is not that we should find it as an alternative to cryptodev and using it just because it will not cause ABI/API breakage.

I am considering this new API as an alternative to existing ones, but as an extension.
Existing crypto-op API has its own advantages (generic), and I think we should keep it supported by all crypto-devs. From other side rte_security is an extendable framework that suits the purpose:
allows easily (and yes without ABI breakage) introduce new API for special type of crypto-dev (SW based).

Quoted 59 lines

IMO the code should be placed where its intent is.

Quoted 57 lines

BTW, what is your concern with proposed approach (via rte_security)? From my perspective it is a lightweight change and it is totally optional for the crypto PMDs to support it or not.
Konstantin

Quoted 2 lines
Quoted 2 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support. There is a

small

Quoted 6 lines
Quoted 6 lines

performance test app under app/test/security_aesni_gcm(mb)_perftest to prove.

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the

session

Quoted 3 lines
Quoted 3 lines

data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error

number

Quoted 4 lines
Quoted 4 lines

assigned to the appropriate index of the status array.

Will update the program's guide in the v1 patch.

Quoted 8 lines
Quoted 8 lines

Regards,
Fan

Quoted 5 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty,

Declan

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

<[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type

and

Quoted 20 lines
Quoted 20 lines
Quoted 20 lines

API

Hi Fan,

Quoted 2 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

action

Quoted 4 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could you please elaborate the flow in the documentation. If not in documentation right now, then please elaborate the flow in cover letter.
Also I see that there are new APIs for processing crypto operations in bulk. What does that mean. How are they different from the existing APIs which are also handling bulk crypto ops depending on the budget.

-Akhil

#11 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 49 lines

Hi Akhil,

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

This action type allows the burst of symmetric crypto workload using the

same

Quoted 1 line
Quoted 1 line

algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

This flexible action type does not require external hardware involvement, having the crypto workload processed synchronously, and is more

performant

Quoted 1 line
Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on removed "async

mode

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst and

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will have the

Quoted 8 lines
Quoted 8 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers, not mbufs.

Quoted 2 lines

I still do not understand why we cannot do with the conventional crypto lib

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol processing or

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous crypto

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a crypto session

in

Quoted 6 lines
Quoted 6 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous processing.

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in existing

cryptodev

Quoted 6 lines
Quoted 6 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need some extra information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra in

future).

Quoted 1 line
Quoted 7 lines
Quoted 1 line

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown current crypto-op approach.
That's why the general idea - have all data that wouldn't change from packet to packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you can have it as a parameter until You get it approved in the crypto xform. I believe it will be beneficial in case of other crypto cases as well. We can have cipher offset at both places(crypto-op and cipher_xform). It will give flexibility to the user to override it.

Quoted 19 lines
Quoted 3 lines

If you intend not to use rte_crypto_op
You can pass this as an argument in the new cryptodev API.

You mean extra parameter in rte_security_process_cpu_crypto_bulk()? It can be in theory, but that solution looks a bit ugly:
why to pass for each call something that would be constant per session? Again having that value constant per session might allow some extra
optimisations
That would be hard to achieve for dynamic case.
and not extendable:
Suppose tomorrow will need to add something extra (some new algorithm support or so).
With what you proposing will need to new parameter to the function, which means API breakage.

Quoted 2 lines

Something extra will also cause ABI breakage in security as well. So it will be same.

Quoted 6 lines

I don't think it would.
AFAIK, right now this patch doesn't introduce any API/ABI breakage. Iinside struct rte_security_session_conf we have a union of xforms depending on session type.
So as long as cpu_crypto_xform wouldn't exceed sizes of other xform - I believe no ABI breakage will appear.

Agreed, it will not break ABI in case of security till we do not exceed current size.

Saving an ABI/API breakage is more important or placing the code at the correct place. We need to find a tradeoff. Others can comment on this.
@Thomas Monjalon, @De Lara Guarch, Pablo Any comments?

Quoted 19 lines
Quoted 4 lines
Quoted 4 lines

Also right now there is no way to add new type of crypto_sym_session

without

Quoted 4 lines
Quoted 2 lines

either breaking existing crypto-dev ABI/API or introducing new structure (rte_crypto_sym_cpu_session or so) for that.

What extra info is required in rte_cryptodev_sym_session to get the

rte_crypto_sym_cpu_session.

Right now - just cipher_offset (see above).
What else in future (if any) - don't know.

Quoted 2 lines

I don't think there is any.
I believe the same crypto session will be able to work synchronously as well.

Exactly the same - problematically, see above.

Quoted 8 lines
Quoted 4 lines

We would only need a new API to perform synchronous actions. That will reduce the duplication code significantly
in the driver to support 2 different kind of APIs with similar code inside. Please correct me in case I am missing something.

To add new API into crypto-dev would also require changes in the PMD, it wouldn't come totally free and I believe would require roughly the same amount of changes.

It will be required only in the PMDs which support it and would be minimal. You would need a feature flag, support for that synchronous API. Session information will already be there in the session. The changes wrt cipher_offset need to be added but with some default value to identify override will be done or not.

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

While rte_security is designed in a way that we can add new session types

and

Quoted 2 lines
Quoted 1 line

related parameters without causing API/ABI breakage.

Quoted 7 lines
Quoted 1 line

Yes the intent is to add new sessions based on various protocols that can be

supported by the driver.

Various protocols and different types of sessions (and devices they belong to). Let say right now we have INLINE_CRYPTO, INLINE_PROTO, LOOKASIDE_PROTO, etc.
Here we introduce new type of session.

What is the new value add to the existing sessions. The changes that we are doing here is just to avoid an API/ABI breakage. The synchronous processing can happen on both crypto and security session. This would mean, only the processing API should be defined, rest all should be already there in the sessions.
In All other cases, INLINE - eth device was not having any format to perform crypto op LOOKASIDE - PROTO - add protocol specific sessions which is not available in crypto.

Quoted 5 lines
Quoted 2 lines

It is not that we should find it as an alternative to cryptodev and using it just

because it will not cause

Quoted 1 line

ABI/API breakage.

Quoted 9 lines

I am considering this new API as an alternative to existing ones, but as an extension.
Existing crypto-op API has its own advantages (generic), and I think we should keep it supported by all crypto-devs.
From other side rte_security is an extendable framework that suits the purpose:
allows easily (and yes without ABI breakage) introduce new API for special type of crypto-dev (SW based).

Adding a synchronous processing API is understandable and can be added in both Crypto as well as Security, but a new action type for it is not required. Now whether to support that, we have ABI/API breakage, that is a different issue. And we may have to deal with it if no other option is there.

Quoted 70 lines
Quoted 13 lines

IMO the code should be placed where its intent is.

Quoted 8 lines

BTW, what is your concern with proposed approach (via rte_security)? From my perspective it is a lightweight change and it is totally optional for the crypto PMDs to support it or not.
Konstantin

Quoted 2 lines
Quoted 2 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support. There is

a

Quoted 2 lines
Quoted 2 lines

small

Quoted 1 line
Quoted 1 line

performance test app under app/test/security_aesni_gcm(mb)_perftest

to

Quoted 22 lines
Quoted 22 lines
Quoted 5 lines
Quoted 5 lines

prove.

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the

session

Quoted 3 lines
Quoted 3 lines

data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error

number

Quoted 12 lines
Quoted 12 lines

assigned to the appropriate index of the status array.

Will update the program's guide in the v1 patch.

Regards,
Fan

Quoted 5 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty,

Quoted 4 lines
Quoted 4 lines

Declan

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

<[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action

type

Quoted 7 lines
Quoted 7 lines

and

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

API

Hi Fan,

Quoted 2 lines

This patch introduce new

RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

action

Quoted 4 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could

you

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

please elaborate the flow in the documentation. If not in

documentation

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

right now, then please elaborate the flow in cover letter. Also I see that there are new APIs for processing crypto operations in

bulk.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

What does that mean. How are they different from the existing APIs

which

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

are also handling bulk crypto ops depending on the budget.

-Akhil

#12 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Quoted 55 lines
Quoted 47 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto workload using the

same

Quoted 1 line
Quoted 1 line

algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

This flexible action type does not require external hardware involvement, having the crypto workload processed synchronously, and is more

performant

Quoted 1 line
Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on removed "async

mode

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst and

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will have the

Quoted 8 lines
Quoted 8 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers, not mbufs.

Quoted 2 lines

I still do not understand why we cannot do with the conventional crypto lib

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol processing or

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous crypto

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a crypto session

in

Quoted 6 lines
Quoted 6 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous processing.

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in existing

cryptodev

Quoted 6 lines
Quoted 6 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need some extra information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra in

future).

Quoted 1 line
Quoted 7 lines
Quoted 1 line

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown current crypto-op approach.
That's why the general idea - have all data that wouldn't change from packet to packet
included into the session and setup it once at session_init().

Quoted 6 lines

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you can have it as a parameter until You get it approved in the crypto xform. I believe it will be beneficial in case of other crypto cases as well. We can have cipher offset at both places(crypto-op and cipher_xform). It will give flexibility to the user to override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for CPU related stuff here? Let say we can have :
num rte_crypto_sym_xform_type {
RTE_CRYPTO_SYM_XFORM_NOT_SPECIFIED = 0, /**< No xform specified */
RTE_CRYPTO_SYM_XFORM_AUTH, /**< Authentication xform */
RTE_CRYPTO_SYM_XFORM_CIPHER, /**< Cipher xform */
RTE_CRYPTO_SYM_XFORM_AEAD /**< AEAD xform */
+ RTE_CRYPTO_SYM_XFORM_CPU = INT32_MIN,
+ RTE_CRYPTO_SYM_XFORM_CPU_AEAD = (RTE_CRYPTO_SYM_XFORM_CPU | RTE_CRYPTO_SYM_XFORM_CPU),

      /* same for auth and crypto xforms */
};

Then we either can re-define some values in struct rte_crypto_aead_xform (via unions), or even have new struct rte_crypto_cpu_aead_xform (same for crypto and auth xforms). Then if PMD wants to support new sync API it would need to recognize new xform types and internally it might end up with different session structure (one for sync, another for async mode). That I think should allow us to introduce cpu_crypto as part of crypto-dev API without ABI breakage. What do you think?
Konstantin

Quoted 177 lines
Quoted 20 lines
Quoted 4 lines

If you intend not to use rte_crypto_op
You can pass this as an argument in the new cryptodev API.

You mean extra parameter in rte_security_process_cpu_crypto_bulk()? It can be in theory, but that solution looks a bit ugly:
why to pass for each call something that would be constant per session? Again having that value constant per session might allow some extra
optimisations
That would be hard to achieve for dynamic case.
and not extendable:
Suppose tomorrow will need to add something extra (some new algorithm support or so).
With what you proposing will need to new parameter to the function, which means API breakage.

Quoted 2 lines

Something extra will also cause ABI breakage in security as well. So it will be same.

Quoted 6 lines

I don't think it would.
AFAIK, right now this patch doesn't introduce any API/ABI breakage. Iinside struct rte_security_session_conf we have a union of xforms depending on session type.
So as long as cpu_crypto_xform wouldn't exceed sizes of other xform - I believe no ABI breakage will appear.

Agreed, it will not break ABI in case of security till we do not exceed current size.

Saving an ABI/API breakage is more important or placing the code at the correct place. We need to find a tradeoff. Others can comment on this.
@Thomas Monjalon, @De Lara Guarch, Pablo Any comments?

Quoted 19 lines
Quoted 4 lines
Quoted 4 lines

Also right now there is no way to add new type of crypto_sym_session

without

Quoted 4 lines
Quoted 2 lines

either breaking existing crypto-dev ABI/API or introducing new structure (rte_crypto_sym_cpu_session or so) for that.

What extra info is required in rte_cryptodev_sym_session to get the

rte_crypto_sym_cpu_session.

Right now - just cipher_offset (see above).
What else in future (if any) - don't know.

Quoted 2 lines

I don't think there is any.
I believe the same crypto session will be able to work synchronously as well.

Exactly the same - problematically, see above.

Quoted 8 lines
Quoted 4 lines

We would only need a new API to perform synchronous actions. That will reduce the duplication code significantly
in the driver to support 2 different kind of APIs with similar code inside. Please correct me in case I am missing something.

To add new API into crypto-dev would also require changes in the PMD, it wouldn't come totally free and I believe would require roughly the same amount of changes.

It will be required only in the PMDs which support it and would be minimal. You would need a feature flag, support for that synchronous API. Session information will already be there in the session. The changes wrt cipher_offset need to be added but with some default value to identify override will be done or not.

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

While rte_security is designed in a way that we can add new session types

and

Quoted 2 lines
Quoted 1 line

related parameters without causing API/ABI breakage.

Quoted 7 lines
Quoted 1 line

Yes the intent is to add new sessions based on various protocols that can be

supported by the driver.

Various protocols and different types of sessions (and devices they belong to). Let say right now we have INLINE_CRYPTO, INLINE_PROTO, LOOKASIDE_PROTO, etc.
Here we introduce new type of session.

What is the new value add to the existing sessions. The changes that we are doing here is just to avoid an API/ABI breakage. The synchronous processing can happen on both crypto and security session. This would mean, only the processing API should be defined, rest all should be already there in the sessions.
In All other cases, INLINE - eth device was not having any format to perform crypto op LOOKASIDE - PROTO - add protocol specific sessions which is not available in crypto.

Quoted 5 lines
Quoted 2 lines

It is not that we should find it as an alternative to cryptodev and using it just

because it will not cause

Quoted 1 line

ABI/API breakage.

Quoted 9 lines

I am considering this new API as an alternative to existing ones, but as an extension.
Existing crypto-op API has its own advantages (generic), and I think we should keep it supported by all crypto-devs.
From other side rte_security is an extendable framework that suits the purpose:
allows easily (and yes without ABI breakage) introduce new API for special type of crypto-dev (SW based).

Adding a synchronous processing API is understandable and can be added in both Crypto as well as Security, but a new action type for it is not required. Now whether to support that, we have ABI/API breakage, that is a different issue. And we may have to deal with it if no other option is there.

Quoted 70 lines
Quoted 13 lines

IMO the code should be placed where its intent is.

Quoted 8 lines

BTW, what is your concern with proposed approach (via rte_security)? From my perspective it is a lightweight change and it is totally optional for the crypto PMDs to support it or not.
Konstantin

Quoted 2 lines
Quoted 2 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support. There is

a

Quoted 2 lines
Quoted 2 lines

small

Quoted 1 line
Quoted 1 line

performance test app under app/test/security_aesni_gcm(mb)_perftest

to

Quoted 22 lines
Quoted 22 lines
Quoted 5 lines
Quoted 5 lines

prove.

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the

session

Quoted 3 lines
Quoted 3 lines

data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error

number

Quoted 12 lines
Quoted 12 lines

assigned to the appropriate index of the status array.

Will update the program's guide in the v1 patch.

Regards,
Fan

Quoted 5 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty,

Quoted 4 lines
Quoted 4 lines

Declan

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

<[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action

type

Quoted 7 lines
Quoted 7 lines

and

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

API

Hi Fan,

Quoted 2 lines

This patch introduce new

RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

action

Quoted 4 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could

you

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

please elaborate the flow in the documentation. If not in

documentation

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

right now, then please elaborate the flow in cover letter. Also I see that there are new APIs for processing crypto operations in

bulk.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

What does that mean. How are they different from the existing APIs

which

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

are also handling bulk crypto ops depending on the budget.

-Akhil

#13 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Quoted 73 lines

Hi Akhil,

Quoted 55 lines
Quoted 47 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto workload using the

same

Quoted 1 line
Quoted 1 line

algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

This flexible action type does not require external hardware involvement, having the crypto workload processed synchronously, and is more

performant

Quoted 1 line
Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on removed "async

mode

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst and

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will have the

Quoted 8 lines
Quoted 8 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers, not mbufs.

Quoted 2 lines

I still do not understand why we cannot do with the conventional crypto lib

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol processing or

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous crypto

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a crypto session

in

Quoted 6 lines
Quoted 6 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous processing.

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in existing

cryptodev

Quoted 6 lines
Quoted 6 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need some extra information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra in

future).

Quoted 1 line
Quoted 7 lines
Quoted 1 line

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown current crypto-op approach.
That's why the general idea - have all data that wouldn't change from packet to packet
included into the session and setup it once at session_init().

Quoted 6 lines

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you can have it as a parameter until You get it approved in the crypto xform. I believe it will be beneficial in case of other crypto cases as well. We can have cipher offset at both places(crypto-op and cipher_xform). It will give flexibility to the user to override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for CPU related stuff here? Let say we can have :
num rte_crypto_sym_xform_type {
RTE_CRYPTO_SYM_XFORM_NOT_SPECIFIED = 0, /**< No xform specified */
RTE_CRYPTO_SYM_XFORM_AUTH, /**< Authentication xform */
RTE_CRYPTO_SYM_XFORM_CIPHER, /**< Cipher xform */
RTE_CRYPTO_SYM_XFORM_AEAD /**< AEAD xform */

Review

Quoted 2 lines
(quoted diff) +2 -0
+     RTE_CRYPTO_SYM_XFORM_CPU = INT32_MIN,
+    RTE_CRYPTO_SYM_XFORM_CPU_AEAD = (RTE_CRYPTO_SYM_XFORM_CPU | RTE_CRYPTO_SYM_XFORM_CPU),

Meant
RTE_CRYPTO_SYM_XFORM_CPU_AEAD = (RTE_CRYPTO_SYM_XFORM_CPU | RTE_CRYPTO_SYM_XFORM_AEAD), of course.

Quoted 188 lines
      /* same for auth and crypto xforms */
};

Then we either can re-define some values in struct rte_crypto_aead_xform (via unions), or even have new struct rte_crypto_cpu_aead_xform (same for crypto and auth xforms). Then if PMD wants to support new sync API it would need to recognize new xform types and internally it might end up with different session structure (one for sync, another for async mode). That I think should allow us to introduce cpu_crypto as part of crypto-dev API without ABI breakage. What do you think?
Konstantin

Quoted 177 lines
Quoted 20 lines
Quoted 4 lines

If you intend not to use rte_crypto_op
You can pass this as an argument in the new cryptodev API.

You mean extra parameter in rte_security_process_cpu_crypto_bulk()? It can be in theory, but that solution looks a bit ugly:
why to pass for each call something that would be constant per session? Again having that value constant per session might allow some extra
optimisations
That would be hard to achieve for dynamic case.
and not extendable:
Suppose tomorrow will need to add something extra (some new algorithm support or so).
With what you proposing will need to new parameter to the function, which means API breakage.

Quoted 2 lines

Something extra will also cause ABI breakage in security as well. So it will be same.

Quoted 6 lines

I don't think it would.
AFAIK, right now this patch doesn't introduce any API/ABI breakage. Iinside struct rte_security_session_conf we have a union of xforms depending on session type.
So as long as cpu_crypto_xform wouldn't exceed sizes of other xform - I believe no ABI breakage will appear.

Agreed, it will not break ABI in case of security till we do not exceed current size.

Saving an ABI/API breakage is more important or placing the code at the correct place. We need to find a tradeoff. Others can comment on this.
@Thomas Monjalon, @De Lara Guarch, Pablo Any comments?

Quoted 19 lines
Quoted 4 lines
Quoted 4 lines

Also right now there is no way to add new type of crypto_sym_session

without

Quoted 4 lines
Quoted 2 lines

either breaking existing crypto-dev ABI/API or introducing new structure (rte_crypto_sym_cpu_session or so) for that.

What extra info is required in rte_cryptodev_sym_session to get the

rte_crypto_sym_cpu_session.

Right now - just cipher_offset (see above).
What else in future (if any) - don't know.

Quoted 2 lines

I don't think there is any.
I believe the same crypto session will be able to work synchronously as well.

Exactly the same - problematically, see above.

Quoted 8 lines
Quoted 4 lines

We would only need a new API to perform synchronous actions. That will reduce the duplication code significantly
in the driver to support 2 different kind of APIs with similar code inside. Please correct me in case I am missing something.

To add new API into crypto-dev would also require changes in the PMD, it wouldn't come totally free and I believe would require roughly the same amount of changes.

It will be required only in the PMDs which support it and would be minimal. You would need a feature flag, support for that synchronous API. Session information will already be there in the session. The changes wrt cipher_offset need to be added but with some default value to identify override will be done or not.

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

While rte_security is designed in a way that we can add new session types

and

Quoted 2 lines
Quoted 1 line

related parameters without causing API/ABI breakage.

Quoted 7 lines
Quoted 1 line

Yes the intent is to add new sessions based on various protocols that can be

supported by the driver.

Various protocols and different types of sessions (and devices they belong to). Let say right now we have INLINE_CRYPTO, INLINE_PROTO, LOOKASIDE_PROTO, etc.
Here we introduce new type of session.

What is the new value add to the existing sessions. The changes that we are doing here is just to avoid an API/ABI breakage. The synchronous processing can happen on both crypto and security session. This would mean, only the processing API should be defined, rest all should be already there in the sessions.
In All other cases, INLINE - eth device was not having any format to perform crypto op LOOKASIDE - PROTO - add protocol specific sessions which is not available in crypto.

Quoted 5 lines
Quoted 2 lines

It is not that we should find it as an alternative to cryptodev and using it just

because it will not cause

Quoted 1 line

ABI/API breakage.

Quoted 9 lines

I am considering this new API as an alternative to existing ones, but as an extension.
Existing crypto-op API has its own advantages (generic), and I think we should keep it supported by all crypto-devs.
From other side rte_security is an extendable framework that suits the purpose:
allows easily (and yes without ABI breakage) introduce new API for special type of crypto-dev (SW based).

Adding a synchronous processing API is understandable and can be added in both Crypto as well as Security, but a new action type for it is not required. Now whether to support that, we have ABI/API breakage, that is a different issue. And we may have to deal with it if no other option is there.

Quoted 70 lines
Quoted 13 lines

IMO the code should be placed where its intent is.

Quoted 8 lines

BTW, what is your concern with proposed approach (via rte_security)? From my perspective it is a lightweight change and it is totally optional for the crypto PMDs to support it or not.
Konstantin

Quoted 2 lines
Quoted 2 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support. There is

a

Quoted 2 lines
Quoted 2 lines

small

Quoted 1 line
Quoted 1 line

performance test app under app/test/security_aesni_gcm(mb)_perftest

to

Quoted 22 lines
Quoted 22 lines
Quoted 5 lines
Quoted 5 lines

prove.

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the

session

Quoted 3 lines
Quoted 3 lines

data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error

number

Quoted 12 lines
Quoted 12 lines

assigned to the appropriate index of the status array.

Will update the program's guide in the v1 patch.

Regards,
Fan

Quoted 5 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty,

Quoted 4 lines
Quoted 4 lines

Declan

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

<[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action

type

Quoted 7 lines
Quoted 7 lines

and

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

API

Hi Fan,

Quoted 2 lines

This patch introduce new

RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

action

Quoted 4 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

I am not able to get the flow of execution for this action type. Could

you

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

please elaborate the flow in the documentation. If not in

documentation

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

right now, then please elaborate the flow in cover letter. Also I see that there are new APIs for processing crypto operations in

bulk.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

What does that mean. How are they different from the existing APIs

which

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

are also handling bulk crypto ops depending on the budget.

-Akhil

#14 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

In reply to Ananyev, Konstantin

Hi Konstantin,

Quoted 74 lines

Hi Akhil,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto workload using

the

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

same

Quoted 1 line
Quoted 1 line

algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This flexible action type does not require external hardware

involvement,

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously, and is more

performant

Quoted 1 line
Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on removed "async

mode

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst

and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will have

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers, not

mbufs.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the conventional

crypto lib

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol processing

or

Quoted 6 lines
Quoted 6 lines

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous crypto

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a crypto

session

Quoted 11 lines
Quoted 11 lines

in

Quoted 6 lines
Quoted 6 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous processing.

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in existing

cryptodev

Quoted 3 lines
Quoted 3 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need some

extra

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra in

future).

Quoted 2 lines

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown current

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't change from packet

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you can have it as

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be beneficial in case of

Quoted 8 lines

other crypto cases as well.

Quoted 1 line

We can have cipher offset at both places(crypto-op and cipher_xform). It will

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for CPU related stuff here?

I also thought of adding new xforms, but that wont serve the purpose for may be all the cases. You would be needing all information currently available in the current xforms.
So if you are adding new fields in the new xform, the size will be more than that of the union of xforms. ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done, then that can be done for each of the Xforms and we can have a solution to this issue.

Quoted 10 lines

Let say we can have :
num rte_crypto_sym_xform_type {
RTE_CRYPTO_SYM_XFORM_NOT_SPECIFIED = 0, /**< No xform specified
*/

        RTE_CRYPTO_SYM_XFORM_AUTH,              /**< Authentication xform */
        RTE_CRYPTO_SYM_XFORM_CIPHER,            /**< Cipher xform  */
        RTE_CRYPTO_SYM_XFORM_AEAD               /**< AEAD xform  */
(quoted diff) +2 -0
+     RTE_CRYPTO_SYM_XFORM_CPU = INT32_MIN,
+    RTE_CRYPTO_SYM_XFORM_CPU_AEAD = (RTE_CRYPTO_SYM_XFORM_CPU |

RTE_CRYPTO_SYM_XFORM_CPU),

Instead of CPU I believe SYNC would be better.

Quoted 219 lines
      /* same for auth and crypto xforms */
};

Then we either can re-define some values in struct rte_crypto_aead_xform (via unions),
or even have new struct rte_crypto_cpu_aead_xform (same for crypto and auth xforms).
Then if PMD wants to support new sync API it would need to recognize new xform types
and internally it might end up with different session structure (one for sync, another for async mode).
That I think should allow us to introduce cpu_crypto as part of crypto-dev API without ABI breakage.
What do you think?
Konstantin

Quoted 20 lines
Quoted 20 lines
Quoted 4 lines

If you intend not to use rte_crypto_op
You can pass this as an argument in the new cryptodev API.

You mean extra parameter in rte_security_process_cpu_crypto_bulk()? It can be in theory, but that solution looks a bit ugly:
why to pass for each call something that would be constant per session? Again having that value constant per session might allow some extra
optimisations
That would be hard to achieve for dynamic case.
and not extendable:
Suppose tomorrow will need to add something extra (some new algorithm support or so).
With what you proposing will need to new parameter to the function, which means API breakage.

Quoted 2 lines

Something extra will also cause ABI breakage in security as well. So it will be same.

Quoted 7 lines
Quoted 6 lines

I don't think it would.
AFAIK, right now this patch doesn't introduce any API/ABI breakage. Iinside struct rte_security_session_conf we have a union of xforms depending on session type.
So as long as cpu_crypto_xform wouldn't exceed sizes of other xform - I believe no ABI breakage will appear.

Agreed, it will not break ABI in case of security till we do not exceed current

size.

Quoted 2 lines

Saving an ABI/API breakage is more important or placing the code at the

correct place.

Quoted 27 lines

We need to find a tradeoff. Others can comment on this. @Thomas Monjalon, @De Lara Guarch, Pablo Any comments?

Quoted 24 lines
Quoted 4 lines
Quoted 4 lines

Also right now there is no way to add new type of crypto_sym_session

without

Quoted 4 lines
Quoted 2 lines

either breaking existing crypto-dev ABI/API or introducing new structure (rte_crypto_sym_cpu_session or so) for that.

What extra info is required in rte_cryptodev_sym_session to get the

rte_crypto_sym_cpu_session.

Right now - just cipher_offset (see above).
What else in future (if any) - don't know.

Quoted 2 lines

I don't think there is any.
I believe the same crypto session will be able to work synchronously as well.

Exactly the same - problematically, see above.

Quoted 4 lines

We would only need a new API to perform synchronous actions. That will reduce the duplication code significantly
in the driver to support 2 different kind of APIs with similar code inside. Please correct me in case I am missing something.

Quoted 6 lines
Quoted 3 lines

To add new API into crypto-dev would also require changes in the PMD, it wouldn't come totally free and I believe would require roughly the same amount of changes.

It will be required only in the PMDs which support it and would be minimal. You would need a feature flag, support for that synchronous API. Session

information will

Quoted 1 line

already be there in the session. The changes wrt cipher_offset need to be

added

Quoted 6 lines

but with some default value to identify override will be done or not.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

While rte_security is designed in a way that we can add new session

types

Quoted 4 lines
Quoted 4 lines

and

Quoted 3 lines
Quoted 1 line

related parameters without causing API/ABI breakage.

Yes the intent is to add new sessions based on various protocols that can

be

Quoted 3 lines
Quoted 3 lines

supported by the driver.

Various protocols and different types of sessions (and devices they belong

to).

Quoted 1 line
Quoted 1 line

Let say right now we have INLINE_CRYPTO, INLINE_PROTO,

LOOKASIDE_PROTO,

Quoted 4 lines
Quoted 2 lines

etc.
Here we introduce new type of session.

What is the new value add to the existing sessions. The changes that we are

doing

Quoted 1 line

here is just to avoid an API/ABI breakage. The synchronous processing can

happen on both

Quoted 1 line

crypto and security session. This would mean, only the processing API should

be defined,

Quoted 2 lines

rest all should be already there in the sessions.
In All other cases, INLINE - eth device was not having any format to perform

crypto op

Quoted 1 line

LOOKASIDE - PROTO - add protocol specific sessions which is not available in

crypto.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

It is not that we should find it as an alternative to cryptodev and using it

just

Quoted 6 lines
Quoted 6 lines

because it will not cause

Quoted 1 line

ABI/API breakage.

I am considering this new API as an alternative to existing ones, but as an extension.
Existing crypto-op API has its own advantages (generic), and I think we

should

Quoted 2 lines
Quoted 2 lines

keep it supported by all crypto-devs.
From other side rte_security is an extendable framework that suits the

purpose:

Quoted 1 line
Quoted 1 line

allows easily (and yes without ABI breakage) introduce new API for special

type

Quoted 5 lines
Quoted 3 lines

of crypto-dev (SW based).

Adding a synchronous processing API is understandable and can be added in

both

Quoted 2 lines

Crypto as well as Security, but a new action type for it is not required. Now whether to support that, we have ABI/API breakage, that is a different

issue.

Quoted 15 lines

And we may have to deal with it if no other option is there.

Quoted 13 lines
Quoted 13 lines

IMO the code should be placed where its intent is.

Quoted 8 lines

BTW, what is your concern with proposed approach (via rte_security)? From my perspective it is a lightweight change and it is totally optional for the crypto PMDs to support it or not.
Konstantin

Quoted 2 lines
Quoted 2 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support.

There is

Quoted 3 lines
Quoted 3 lines

a

Quoted 2 lines
Quoted 2 lines

small

Quoted 1 line
Quoted 1 line

performance test app under

app/test/security_aesni_gcm(mb)_perftest

Quoted 9 lines
Quoted 9 lines

to

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

prove.

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the

session

Quoted 2 lines
Quoted 2 lines

data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user

will

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

expect the buffers are either processed successfully, or having the

error

Quoted 24 lines
Quoted 24 lines
Quoted 13 lines
Quoted 13 lines

number

Quoted 12 lines
Quoted 12 lines

assigned to the appropriate index of the status array.

Will update the program's guide in the v1 patch.

Regards,
Fan

Quoted 5 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty,

Quoted 4 lines
Quoted 4 lines

Declan

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

<[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action

type

Quoted 6 lines
Quoted 6 lines

and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

API

Hi Fan,

Quoted 1 line
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This patch introduce new

RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

action

Quoted 3 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for

PMDs.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

I am not able to get the flow of execution for this action type.

Could

Quoted 5 lines
Quoted 5 lines

you

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

please elaborate the flow in the documentation. If not in

documentation

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

right now, then please elaborate the flow in cover letter. Also I see that there are new APIs for processing crypto operations

in

Quoted 7 lines
Quoted 7 lines

bulk.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

What does that mean. How are they different from the existing APIs

which

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

are also handling bulk crypto ops depending on the budget.

-Akhil

#15 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Quoted 80 lines
Quoted 71 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto workload using

the

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

same

Quoted 1 line
Quoted 1 line

algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This flexible action type does not require external hardware

involvement,

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously, and is more

performant

Quoted 1 line
Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on removed "async

mode

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst

and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will have

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers, not

mbufs.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the conventional

crypto lib

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol processing

or

Quoted 6 lines
Quoted 6 lines

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous crypto

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a crypto

session

Quoted 11 lines
Quoted 11 lines

in

Quoted 6 lines
Quoted 6 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous processing.

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in existing

cryptodev

Quoted 3 lines
Quoted 3 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need some

extra

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra in

future).

Quoted 2 lines

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown current

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't change from packet

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you can have it as

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be beneficial in case of

Quoted 8 lines

other crypto cases as well.

Quoted 1 line

We can have cipher offset at both places(crypto-op and cipher_xform). It will

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for CPU related stuff here?

Quoted 7 lines

I also thought of adding new xforms, but that wont serve the purpose for may be all the cases. You would be needing all information currently available in the current xforms.
So if you are adding new fields in the new xform, the size will be more than that of the union of xforms. ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done, then that can be done for each of the Xforms and we can have a solution to this issue.

I think that we can re-use iv.offset for our purposes (for crypto offset). So for now we can make that path work without any ABI breakage. Fan, please feel free to correct me here, if I missed something.
If in future we would need to add some extra information it might require ABI breakage, though by now I don't envision anything particular to add. Anyway, if there is no objection to go that way, we can try to make these changes for v2.

Quoted 12 lines
Quoted 11 lines

Let say we can have :
num rte_crypto_sym_xform_type {
RTE_CRYPTO_SYM_XFORM_NOT_SPECIFIED = 0, /**< No xform specified
*/

        RTE_CRYPTO_SYM_XFORM_AUTH,              /**< Authentication xform */
        RTE_CRYPTO_SYM_XFORM_CIPHER,            /**< Cipher xform  */
        RTE_CRYPTO_SYM_XFORM_AEAD               /**< AEAD xform  */
(quoted diff) +2 -0
+     RTE_CRYPTO_SYM_XFORM_CPU = INT32_MIN,
+    RTE_CRYPTO_SYM_XFORM_CPU_AEAD = (RTE_CRYPTO_SYM_XFORM_CPU |

RTE_CRYPTO_SYM_XFORM_CPU),

Quoted 1 line

Instead of CPU I believe SYNC would be better.

I don't mind to name it to SYNC, but I'd like to outline, that it's not really more CPU then generic SYNC API (it doesn't pass IOVA for data buffers, etc., only VA).

Quoted 220 lines
Quoted 220 lines
      /* same for auth and crypto xforms */
};

Then we either can re-define some values in struct rte_crypto_aead_xform (via unions),
or even have new struct rte_crypto_cpu_aead_xform (same for crypto and auth xforms).
Then if PMD wants to support new sync API it would need to recognize new xform types
and internally it might end up with different session structure (one for sync, another for async mode).
That I think should allow us to introduce cpu_crypto as part of crypto-dev API without ABI breakage.
What do you think?
Konstantin

Quoted 20 lines
Quoted 20 lines
Quoted 4 lines

If you intend not to use rte_crypto_op
You can pass this as an argument in the new cryptodev API.

You mean extra parameter in rte_security_process_cpu_crypto_bulk()? It can be in theory, but that solution looks a bit ugly:
why to pass for each call something that would be constant per session? Again having that value constant per session might allow some extra
optimisations
That would be hard to achieve for dynamic case.
and not extendable:
Suppose tomorrow will need to add something extra (some new algorithm support or so).
With what you proposing will need to new parameter to the function, which means API breakage.

Quoted 2 lines

Something extra will also cause ABI breakage in security as well. So it will be same.

Quoted 7 lines
Quoted 6 lines

I don't think it would.
AFAIK, right now this patch doesn't introduce any API/ABI breakage. Iinside struct rte_security_session_conf we have a union of xforms depending on session type.
So as long as cpu_crypto_xform wouldn't exceed sizes of other xform - I believe no ABI breakage will appear.

Agreed, it will not break ABI in case of security till we do not exceed current

size.

Quoted 2 lines

Saving an ABI/API breakage is more important or placing the code at the

correct place.

Quoted 27 lines

We need to find a tradeoff. Others can comment on this. @Thomas Monjalon, @De Lara Guarch, Pablo Any comments?

Quoted 24 lines
Quoted 4 lines
Quoted 4 lines

Also right now there is no way to add new type of crypto_sym_session

without

Quoted 4 lines
Quoted 2 lines

either breaking existing crypto-dev ABI/API or introducing new structure (rte_crypto_sym_cpu_session or so) for that.

What extra info is required in rte_cryptodev_sym_session to get the

rte_crypto_sym_cpu_session.

Right now - just cipher_offset (see above).
What else in future (if any) - don't know.

Quoted 2 lines

I don't think there is any.
I believe the same crypto session will be able to work synchronously as well.

Exactly the same - problematically, see above.

Quoted 4 lines

We would only need a new API to perform synchronous actions. That will reduce the duplication code significantly
in the driver to support 2 different kind of APIs with similar code inside. Please correct me in case I am missing something.

Quoted 6 lines
Quoted 3 lines

To add new API into crypto-dev would also require changes in the PMD, it wouldn't come totally free and I believe would require roughly the same amount of changes.

It will be required only in the PMDs which support it and would be minimal. You would need a feature flag, support for that synchronous API. Session

information will

Quoted 1 line

already be there in the session. The changes wrt cipher_offset need to be

added

Quoted 6 lines

but with some default value to identify override will be done or not.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

While rte_security is designed in a way that we can add new session

types

Quoted 4 lines
Quoted 4 lines

and

Quoted 3 lines
Quoted 1 line

related parameters without causing API/ABI breakage.

Yes the intent is to add new sessions based on various protocols that can

be

Quoted 3 lines
Quoted 3 lines

supported by the driver.

Various protocols and different types of sessions (and devices they belong

to).

Quoted 1 line
Quoted 1 line

Let say right now we have INLINE_CRYPTO, INLINE_PROTO,

LOOKASIDE_PROTO,

Quoted 4 lines
Quoted 2 lines

etc.
Here we introduce new type of session.

What is the new value add to the existing sessions. The changes that we are

doing

Quoted 1 line

here is just to avoid an API/ABI breakage. The synchronous processing can

happen on both

Quoted 1 line

crypto and security session. This would mean, only the processing API should

be defined,

Quoted 2 lines

rest all should be already there in the sessions.
In All other cases, INLINE - eth device was not having any format to perform

crypto op

Quoted 1 line

LOOKASIDE - PROTO - add protocol specific sessions which is not available in

crypto.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

It is not that we should find it as an alternative to cryptodev and using it

just

Quoted 6 lines
Quoted 6 lines

because it will not cause

Quoted 1 line

ABI/API breakage.

I am considering this new API as an alternative to existing ones, but as an extension.
Existing crypto-op API has its own advantages (generic), and I think we

should

Quoted 2 lines
Quoted 2 lines

keep it supported by all crypto-devs.
From other side rte_security is an extendable framework that suits the

purpose:

Quoted 1 line
Quoted 1 line

allows easily (and yes without ABI breakage) introduce new API for special

type

Quoted 5 lines
Quoted 3 lines

of crypto-dev (SW based).

Adding a synchronous processing API is understandable and can be added in

both

Quoted 2 lines

Crypto as well as Security, but a new action type for it is not required. Now whether to support that, we have ABI/API breakage, that is a different

issue.

Quoted 15 lines

And we may have to deal with it if no other option is there.

Quoted 13 lines
Quoted 13 lines

IMO the code should be placed where its intent is.

Quoted 8 lines

BTW, what is your concern with proposed approach (via rte_security)? From my perspective it is a lightweight change and it is totally optional for the crypto PMDs to support it or not.
Konstantin

Quoted 2 lines
Quoted 2 lines

AESNI-GCM and AESNI-MB PMDs are updated with this support.

There is

Quoted 3 lines
Quoted 3 lines

a

Quoted 2 lines
Quoted 2 lines

small

Quoted 1 line
Quoted 1 line

performance test app under

app/test/security_aesni_gcm(mb)_perftest

Quoted 9 lines
Quoted 9 lines

to

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

prove.

For the new API
The packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the

session

Quoted 2 lines
Quoted 2 lines

data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user

will

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

expect the buffers are either processed successfully, or having the

error

Quoted 24 lines
Quoted 24 lines
Quoted 13 lines
Quoted 13 lines

number

Quoted 12 lines
Quoted 12 lines

assigned to the appropriate index of the status array.

Will update the program's guide in the v1 patch.

Regards,
Fan

Quoted 5 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Wednesday, September 4, 2019 11:33 AM
To: Zhang, Roy Fan <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; Doherty,

Quoted 4 lines
Quoted 4 lines

Declan

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

<[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action

type

Quoted 6 lines
Quoted 6 lines

and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

API

Hi Fan,

Quoted 1 line
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This patch introduce new

RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

action

Quoted 3 lines

type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for

PMDs.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

I am not able to get the flow of execution for this action type.

Could

Quoted 5 lines
Quoted 5 lines

you

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

please elaborate the flow in the documentation. If not in

documentation

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

right now, then please elaborate the flow in cover letter. Also I see that there are new APIs for processing crypto operations

in

Quoted 7 lines
Quoted 7 lines

bulk.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

What does that mean. How are they different from the existing APIs

which

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

are also handling bulk crypto ops depending on the budget.

-Akhil

#16 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Quoted 89 lines
Quoted 81 lines
Quoted 72 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

This action type allows the burst of symmetric crypto workload using

the

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

same

Quoted 1 line
Quoted 1 line

algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This flexible action type does not require external hardware

involvement,

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously, and is more

performant

Quoted 1 line
Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on removed "async

mode

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the cryptodev_enqueue_burst

and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will have

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers, not

mbufs.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the conventional

crypto lib

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol processing

or

Quoted 6 lines
Quoted 6 lines

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous crypto

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a crypto

session

Quoted 11 lines
Quoted 11 lines

in

Quoted 6 lines
Quoted 6 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous processing.

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in existing

cryptodev

Quoted 3 lines
Quoted 3 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need some

extra

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra in

future).

Quoted 2 lines

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown current

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't change from packet

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you can have it as

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be beneficial in case of

Quoted 8 lines

other crypto cases as well.

Quoted 1 line

We can have cipher offset at both places(crypto-op and cipher_xform). It will

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for CPU related stuff here?

Quoted 7 lines

I also thought of adding new xforms, but that wont serve the purpose for may be all the cases. You would be needing all information currently available in the current xforms.
So if you are adding new fields in the new xform, the size will be more than that of the union of xforms. ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done, then that can be done for each of the Xforms and we can have a solution to this issue.

Quoted 8 lines

I think that we can re-use iv.offset for our purposes (for crypto offset). So for now we can make that path work without any ABI breakage. Fan, please feel free to correct me here, if I missed something.
If in future we would need to add some extra information it might require ABI breakage, though by now I don't envision anything particular to add. Anyway, if there is no objection to go that way, we can try to make these changes for v2.

Actually, after looking at it more deeply it appears not that easy as I thought it would be :) Below is a very draft version of proposed API additions.
I think it avoids ABI breakages right now and provides enough flexibility for future extensions (if any). For now, it doesn't address your comments about naming conventions (_CPU_ vs _SYNC_) , etc. but I suppose is comprehensive enough to provide a main idea beyond it.
Akhil and other interested parties, please try to review and provide feedback ASAP,
as related changes would take some time and we still like to hit 19.11 deadline. Konstantin

diff --git a/lib/librte_cryptodev/rte_crypto_sym.h b/lib/librte_cryptodev/rte_crypto_sym.h
index bc8da2466..c03069e23 100644

lib/librte_cryptodev/rte_crypto_sym.h +31 -3
@@ -103,6 +103,9 @@ rte_crypto_cipher_operation_strings[];
  *
  * This structure contains data relating to Cipher (Encryption and Decryption)
  *  use to create a session.
+ * Actually I was wrong saying that we don't have free space inside xforms.
+ * Making key struct packed (see below) allow us to regain 6B that could be
+ * used for future extensions.
  */
 struct rte_crypto_cipher_xform {
        enum rte_crypto_cipher_operation op;
@@ -116,7 +119,25 @@ struct rte_crypto_cipher_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /**
+         * offset for cipher to start within user provided data buffer.
+        * Fan suggested another (and less space consuming way) -
+         * reuse iv.offset space below, by changing:
+        * struct {uint16_t offset, length;} iv;
+        * to uunamed union:
+        * union {
+        *      struct {uint16_t offset, length;} iv;
+        *      struct {uint16_t iv_len, crypto_offset} cpu_crypto_param;
+        * };
+        * Both approaches seems ok to me in general.
+        * Comments/suggestions are welcome.
+         */
+       uint16_t offset;
+
+       uint8_t reserved1[4];
+
        /**< Cipher key
         *
         * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation, key.data will
@@ -284,7 +305,7 @@ struct rte_crypto_auth_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
        /**< Authentication key data.
         * The authentication key length MUST be less than or equal to the
         * block size of the algorithm. It is the callers responsibility to
@@ -292,6 +313,8 @@ struct rte_crypto_auth_xform {
         * (for example RFC 2104, FIPS 198a).
         */
 
+       uint8_t reserved1[6];
+
        struct {
                uint16_t offset;
                /**< Starting point for Initialisation Vector or Counter,
@@ -376,7 +399,12 @@ struct rte_crypto_aead_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /** offset for cipher to start within data buffer */
+       uint16_t cipher_offset;
+
+       uint8_t reserved1[4];
 
        struct {
                uint16_t offset;
lib/librte_cryptodev/rte_cryptodev.h +95 -0

101 lines not on this page · raw message

lib/librte_cryptodev/rte_cryptodev_pmd.h +19 -0
index defe05ea0..ed7e63fab 100644
@@ -310,6 +310,20 @@ typedef void (*cryptodev_sym_free_session_t)(struct rte_cryptodev *dev,
 typedef void (*cryptodev_asym_free_session_t)(struct rte_cryptodev *dev,
                struct rte_cryptodev_asym_session *sess);
 
+typedef int (*cryptodev_cpu_sym_session_size_t) (struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef int (*cryptodev_cpu_sym_session_init_t) (struct rte_cryptodev *dev,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef void (*cryptodev_cpu_sym_session_fini_t) (struct rte_cryptodev *dev,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+typedef rte_crypto_cpu_sym_process_t (*cryptodev_cpu_sym_session_func_t) (
+                       struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
        cryptodev_configure_t dev_configure;    /**< Configure device. */
@@ -343,6 +357,11 @@ struct rte_cryptodev_ops {
        /**< Clear a Crypto sessions private data. */
        cryptodev_asym_free_session_t asym_session_clear;
        /**< Clear a Crypto sessions private data. */
+
+       cryptodev_cpu_sym_session_size_t sym_cpu_session_get_size;
+       cryptodev_cpu_sym_session_func_t sym_cpu_session_get_func;
+       cryptodev_cpu_sym_session_init_t sym_cpu_session_init;
+       cryptodev_cpu_sym_session_fini_t sym_cpu_session_fini;
 };
 
 
#17 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 164 lines

-----Original Message-----
From: Ananyev, Konstantin <[email protected]>
Sent: Wednesday, September 25, 2019 11:54 PM
To: Akhil Goyal <[email protected]>; '[email protected]' <[email protected]>; De Lara Guarch, Pablo <[email protected]>; 'Thomas Monjalon' <[email protected]>
Cc: Zhang, Roy Fan <[email protected]>; Doherty, Declan <[email protected]>; 'Anoob Joseph' <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto workload

using

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

the

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

same

Quoted 1 line
Quoted 1 line

algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This flexible action type does not require external hardware

involvement,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously, and is

more

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

performant

Quoted 1 line
Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on removed

"async

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

mode

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the

cryptodev_enqueue_burst

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will

have

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers,

not

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

mbufs.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the conventional

crypto lib

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol

processing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

or

Quoted 3 lines
Quoted 3 lines

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous

crypto

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a crypto

session

Quoted 4 lines
Quoted 4 lines

in

Quoted 3 lines
Quoted 3 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous processing.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in existing

cryptodev

Quoted 3 lines
Quoted 3 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need

some

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

extra

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra

in

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

future).

Quoted 2 lines

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown current

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't change from

packet

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you can have

it as

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be beneficial in

case of

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

other crypto cases as well.

Quoted 1 line

We can have cipher offset at both places(crypto-op and cipher_xform). It

will

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for CPU

related

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

stuff here?

I also thought of adding new xforms, but that wont serve the purpose for

may be all the cases.

Quoted 1 line
Quoted 1 line

You would be needing all information currently available in the current

xforms.

Quoted 1 line
Quoted 1 line

So if you are adding new fields in the new xform, the size will be more than

that of the union of xforms.

Quoted 3 lines
Quoted 3 lines

ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done, then that

can be done for each of the

Quoted 7 lines
Quoted 1 line

Xforms and we can have a solution to this issue.

I think that we can re-use iv.offset for our purposes (for crypto offset). So for now we can make that path work without any ABI breakage. Fan, please feel free to correct me here, if I missed something. If in future we would need to add some extra information it might require ABI breakage, though by now I don't envision anything particular to

add.

Quoted 3 lines

Anyway, if there is no objection to go that way, we can try to make these changes for v2.

Actually, after looking at it more deeply it appears not that easy as I thought it would be :)
Below is a very draft version of proposed API additions.
I think it avoids ABI breakages right now and provides enough flexibility for future extensions (if any).
For now, it doesn't address your comments about naming conventions (_CPU_ vs _SYNC_) , etc.
but I suppose is comprehensive enough to provide a main idea beyond it. Akhil and other interested parties, please try to review and provide feedback ASAP,
as related changes would take some time and we still like to hit 19.11 deadline. Konstantin

diff --git a/lib/librte_cryptodev/rte_crypto_sym.h
b/lib/librte_cryptodev/rte_crypto_sym.h index bc8da2466..c03069e23 100644

lib/librte_cryptodev/rte_crypto_sym.h +6 -1
@@ -103,6 +103,9 @@ rte_crypto_cipher_operation_strings[];
  *
  * This structure contains data relating to Cipher (Encryption and Decryption)
  *  use to create a session.
+ * Actually I was wrong saying that we don't have free space inside xforms.
+ * Making key struct packed (see below) allow us to regain 6B that could be
+ * used for future extensions.
  */
 struct rte_crypto_cipher_xform {
        enum rte_crypto_cipher_operation op;
@@ -116,7 +119,25 @@ struct rte_crypto_cipher_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /**

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 134

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +10 -0
+         * offset for cipher to start within user provided data buffer.
+        * Fan suggested another (and less space consuming way) -
+         * reuse iv.offset space below, by changing:
+        * struct {uint16_t offset, length;} iv;
+        * to uunamed union:
+        * union {
+        *      struct {uint16_t offset, length;} iv;
+        *      struct {uint16_t iv_len, crypto_offset} cpu_crypto_param;
+        * };
+        * Both approaches seems ok to me in general.

No strong opinions here. OK with this one.

Quoted 47 lines
lib/librte_cryptodev/rte_crypto_sym.h +15 -2
+        * Comments/suggestions are welcome.
+         */
+       uint16_t offset;
+
+       uint8_t reserved1[4];
+
        /**< Cipher key
         *
         * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation, key.data will
@@ -284,7 +305,7 @@ struct rte_crypto_auth_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
        /**< Authentication key data.
         * The authentication key length MUST be less than or equal to the
         * block size of the algorithm. It is the callers responsibility to
@@ -292,6 +313,8 @@ struct rte_crypto_auth_xform {
         * (for example RFC 2104, FIPS 198a).
         */
 
+       uint8_t reserved1[6];
+
        struct {
                uint16_t offset;
                /**< Starting point for Initialisation Vector or Counter,
@@ -376,7 +399,12 @@ struct rte_crypto_aead_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /** offset for cipher to start within data buffer */
+       uint16_t cipher_offset;
+
+       uint8_t reserved1[4];
 
        struct {
                uint16_t offset;
lib/librte_cryptodev/rte_cryptodev.h +0 -0
index e175b838c..c0c7bfed7 100644

lib/librte_cryptodev/rte_cryptodev.h void * · line 1279

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
@@ -1272,6 +1272,101 @@ void *
 rte_cryptodev_sym_session_get_user_data(
                                        struct rte_cryptodev_sym_session *sess);
 
+/*
+ * After several thoughts decided not to try to squeeze CPU_CRYPTO
+ * into existing rte_crypto_sym_session structure/API, but instead
+ * introduce an extentsion to it via new fully opaque
+ * struct rte_crypto_cpu_sym_session and additional related API.

What all things do we need to squeeze?
In this proposal I do not see the new struct cpu_sym_session defined here.
I believe you will have same lib API/struct for cpu_sym_session and sym_session.
I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is supported(from feature flag) and Have relevant fields in xform(the newly added ones which are packed as per your suggestions) set, It will create that type of session.

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ * Main points:
+ * - Current crypto-dev API is reasonably mature and it is desirable
+ *   to keep it unchanged (API/ABI stability). From other side, this
+ *   new sync API is new one and probably would require extra changes.

lib/librte_cryptodev/rte_cryptodev.h void * · line 1293

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ *   Having it as a new one allows to mark it as experimental, without
+ *   affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more flexibility
+ *   to the PMD writers and again allows to avoid ABI breakages in future.
+ * - process() function per set of xforms
+ *   allows to expose different process() functions for different
+ *   xform combinations. PMD writer can decide, does he wants to
+ *   push all supported algorithms into one process() function,
+ *   or spread it across several ones.
+ *   I.E. More flexibility for PMD writer.

Which process function should be chosen is internal to PMD, how would that info be visible to the application or the library. These will get stored in the session private data. It would be upto the PMD writer, to store the per session process function in the session private data.

Process function would be a dev ops just like enc/deq operations and it should call The respective process API stored in the session private data.

I am not sure if you would need a new session init API for this as nothing would be visible to the app or lib.

lib/librte_cryptodev/rte_cryptodev.h void * · line 1298

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+ * - Not storing process() pointer inside the session -
+ *   Allows user to choose does he want to store a process() pointer
+ *   per session, or per group of sessions for that device that share
+ *   the same input xforms. I.E. extra flexibility for the user,
+ *   plus allows us to keep cpu_sym_session totally opaque, see above.

If multiple sessions need to be processed via the same process function, PMD would save the same process in all the sessions, I don't think there would be any perf overhead with that.

Quoted 115 lines
lib/librte_cryptodev/rte_cryptodev.h +71 -0
+ * Sketched usage model:
+ * ....
+ * /* control path, alloc/init session */
+ * int32_t sz = rte_crypto_cpu_sym_session_size(dev_id, &xform);
+ * struct rte_crypto_cpu_sym_session *ses = user_alloc(..., sz);
+ * rte_crypto_cpu_sym_process_t process =
+ *     rte_crypto_cpu_sym_session_func(dev_id, &xform);
+ * rte_crypto_cpu_sym_session_init(dev_id, ses, &xform);
+ * ...
+ * /* data-path*/
+ * process(ses, ....);
+ * ....
+ * /* control path, termiante/free session */
+ * rte_crypto_cpu_sym_session_fini(dev_id, ses);
+ */
+
+/**
+ * vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_crypto_vec {
+       struct iovec *vec;
+       uint32_t num;
+};
+
+/*
+ * Data-path bulk process crypto function.
+ */
+typedef void (*rte_crypto_cpu_sym_process_t)(
+               struct rte_crypto_cpu_sym_session *sess,
+               struct rte_crypto_vec buf[], void *iv[], void *aad[],
+               void *digest[], int status[], uint32_t num);
+/*
+ * for given device return process function specific to input xforms
+ * on error - return NULL and set rte_errno value.
+ * Note that for same input xfroms for the same device should return
+ * the same process function.
+ */
+__rte_experimental
+rte_crypto_cpu_sym_process_t
+rte_crypto_cpu_sym_session_func(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Return required session size in bytes for given set of xforms.
+ * if xforms == NULL, then return the max possible session size,
+ * that would fit session for any supported by the device algorithm.
+ * if CPU mode is not supported at all, or requeted in xform
+ * algorithm is not supported, then return -ENOTSUP.
+ */
+__rte_experimental
+int
+rte_crypto_cpu_sym_session_size(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Initialize session.
+ * It is caller responsibility to allocate enough space for it.
+ * See rte_crypto_cpu_sym_session_size above.
+ */
+__rte_experimental
+int rte_crypto_cpu_sym_session_init(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+__rte_experimental
+void
+rte_crypto_cpu_sym_session_fini(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +11 -0
index defe05ea0..ed7e63fab 100644
@@ -310,6 +310,20 @@ typedef void (*cryptodev_sym_free_session_t)(struct
 rte_cryptodev *dev,
 typedef void (*cryptodev_asym_free_session_t)(struct rte_cryptodev *dev,
                struct rte_cryptodev_asym_session *sess);
 
+typedef int (*cryptodev_cpu_sym_session_size_t) (struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef int (*cryptodev_cpu_sym_session_init_t) (struct rte_cryptodev *dev,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef void (*cryptodev_cpu_sym_session_fini_t) (struct rte_cryptodev *dev,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+typedef rte_crypto_cpu_sym_process_t (*cryptodev_cpu_sym_session_func_t)

(

lib/librte_cryptodev/rte_cryptodev_pmd.h +8 -0
+                       struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
        cryptodev_configure_t dev_configure;    /**< Configure device. */
@@ -343,6 +357,11 @@ struct rte_cryptodev_ops {
        /**< Clear a Crypto sessions private data. */
        cryptodev_asym_free_session_t asym_session_clear;
        /**< Clear a Crypto sessions private data. */
+
+       cryptodev_cpu_sym_session_size_t sym_cpu_session_get_size;
+       cryptodev_cpu_sym_session_func_t sym_cpu_session_get_func;
+       cryptodev_cpu_sym_session_init_t sym_cpu_session_init;
+       cryptodev_cpu_sym_session_fini_t sym_cpu_session_fini;
 };
 
 
#18 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Quoted 157 lines
Quoted 157 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto workload

using

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

the

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

same

Quoted 1 line
Quoted 1 line

algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This flexible action type does not require external hardware

involvement,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously, and is

more

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

performant

Quoted 1 line
Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on removed

"async

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

mode

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto ops.

Does that mean application will not call the

cryptodev_enqueue_burst

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it will

have

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers,

not

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

mbufs.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the conventional

crypto lib

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol

processing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

or

Quoted 3 lines
Quoted 3 lines

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous

crypto

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a crypto

session

Quoted 4 lines
Quoted 4 lines

in

Quoted 3 lines
Quoted 3 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous processing.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in existing

cryptodev

Quoted 3 lines
Quoted 3 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need

some

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

extra

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something extra

in

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

future).

Quoted 2 lines

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown current

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't change from

packet

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you can have

it as

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be beneficial in

case of

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

other crypto cases as well.

Quoted 1 line

We can have cipher offset at both places(crypto-op and cipher_xform). It

will

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for CPU

related

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

stuff here?

I also thought of adding new xforms, but that wont serve the purpose for

may be all the cases.

Quoted 1 line
Quoted 1 line

You would be needing all information currently available in the current

xforms.

Quoted 1 line
Quoted 1 line

So if you are adding new fields in the new xform, the size will be more than

that of the union of xforms.

Quoted 3 lines
Quoted 3 lines

ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done, then that

can be done for each of the

Quoted 7 lines
Quoted 1 line

Xforms and we can have a solution to this issue.

I think that we can re-use iv.offset for our purposes (for crypto offset). So for now we can make that path work without any ABI breakage. Fan, please feel free to correct me here, if I missed something. If in future we would need to add some extra information it might require ABI breakage, though by now I don't envision anything particular to

add.

Quoted 3 lines

Anyway, if there is no objection to go that way, we can try to make these changes for v2.

Actually, after looking at it more deeply it appears not that easy as I thought it would be :)
Below is a very draft version of proposed API additions.
I think it avoids ABI breakages right now and provides enough flexibility for future extensions (if any).
For now, it doesn't address your comments about naming conventions (_CPU_ vs _SYNC_) , etc.
but I suppose is comprehensive enough to provide a main idea beyond it. Akhil and other interested parties, please try to review and provide feedback ASAP,
as related changes would take some time and we still like to hit 19.11 deadline. Konstantin

diff --git a/lib/librte_cryptodev/rte_crypto_sym.h
b/lib/librte_cryptodev/rte_crypto_sym.h index bc8da2466..c03069e23 100644

lib/librte_cryptodev/rte_crypto_sym.h +10 -1
@@ -103,6 +103,9 @@ rte_crypto_cipher_operation_strings[];
  *
  * This structure contains data relating to Cipher (Encryption and Decryption)
  *  use to create a session.
+ * Actually I was wrong saying that we don't have free space inside xforms.
+ * Making key struct packed (see below) allow us to regain 6B that could be
+ * used for future extensions.
  */
 struct rte_crypto_cipher_xform {
        enum rte_crypto_cipher_operation op;
@@ -116,7 +119,25 @@ struct rte_crypto_cipher_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /**
+         * offset for cipher to start within user provided data buffer.
+        * Fan suggested another (and less space consuming way) -
+         * reuse iv.offset space below, by changing:
+        * struct {uint16_t offset, length;} iv;
Quoted 12 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 134

Quoted 6 lines
lib/librte_cryptodev/rte_crypto_sym.h +6 -0
+        * to uunamed union:
+        * union {
+        *      struct {uint16_t offset, length;} iv;
+        *      struct {uint16_t iv_len, crypto_offset} cpu_crypto_param;
+        * };
+        * Both approaches seems ok to me in general.

No strong opinions here. OK with this one.

Quoted 3 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+        * Comments/suggestions are welcome.
+         */
+       uint16_t offset;

After another thought - it is probably a bit better to have offset as a separate field. In that case we can use the same xforms to create both type of sessions.

Quoted 47 lines
Quoted 47 lines
lib/librte_cryptodev/rte_crypto_sym.h +12 -2
+
+       uint8_t reserved1[4];
+
        /**< Cipher key
         *
         * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation, key.data will
@@ -284,7 +305,7 @@ struct rte_crypto_auth_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
        /**< Authentication key data.
         * The authentication key length MUST be less than or equal to the
         * block size of the algorithm. It is the callers responsibility to
@@ -292,6 +313,8 @@ struct rte_crypto_auth_xform {
         * (for example RFC 2104, FIPS 198a).
         */
 
+       uint8_t reserved1[6];
+
        struct {
                uint16_t offset;
                /**< Starting point for Initialisation Vector or Counter,
@@ -376,7 +399,12 @@ struct rte_crypto_aead_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /** offset for cipher to start within data buffer */
+       uint16_t cipher_offset;
+
+       uint8_t reserved1[4];
 
        struct {
                uint16_t offset;
lib/librte_cryptodev/rte_cryptodev.h +0 -0
index e175b838c..c0c7bfed7 100644
@@ -1272,6 +1272,101 @@ void *
 rte_cryptodev_sym_session_get_user_data(
                                        struct rte_cryptodev_sym_session *sess);
 
Quoted 9 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1279

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+/*
+ * After several thoughts decided not to try to squeeze CPU_CRYPTO
+ * into existing rte_crypto_sym_session structure/API, but instead
+ * introduce an extentsion to it via new fully opaque
+ * struct rte_crypto_cpu_sym_session and additional related API.

What all things do we need to squeeze?
In this proposal I do not see the new struct cpu_sym_session defined here.

The plan is to have it totally opaque to the user, i.e. just:
struct rte_crypto_cpu_sym_session;
in public header files.

Quoted 1 line

I believe you will have same lib API/struct for cpu_sym_session and sym_session.

I thought about such way, but there are few things that looks clumsy to me:
1. Right now there is no 'type' (or so) field inside rte_cryptodev_sym_session,
so it is not possible to easy distinguish what session do you have: lksd_sym or cpu_sym.
In theory, there is a hole of 4B inside rte_cryptodev_sym_session, so we can add some extra field here, but in that case we wouldn't be able to use the same xform for both lksd_sym or cpu_sym (which seems really plausible thing for me).
2. Majority of rte_cryptodev_sym_session fields I think are unnecessary for rte_crypto_cpu_sym_session:
sess_data[], opaque_data, user_data, nb_drivers.
All that consumes space, that could be used somewhere else instead.
3. I am a bit reluctant to touch existing rte_cryptodev API - to avoid any breakages I can't foresee right now. From other side - if we'll add new functions/structs for cpu_sym_session we can mark it
and keep it for some time as experimental, so further changes (if needed) would still be possible.

Quoted 21 lines

I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is supported(from feature flag) and Have relevant fields in xform(the newly added ones which are packed as per your suggestions) set, It will create that type of session.

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ * Main points:
+ * - Current crypto-dev API is reasonably mature and it is desirable
+ *   to keep it unchanged (API/ABI stability). From other side, this
+ *   new sync API is new one and probably would require extra changes.

lib/librte_cryptodev/rte_cryptodev.h void * · line 1293

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ *   Having it as a new one allows to mark it as experimental, without
+ *   affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more flexibility
+ *   to the PMD writers and again allows to avoid ABI breakages in future.
+ * - process() function per set of xforms
+ *   allows to expose different process() functions for different
+ *   xform combinations. PMD writer can decide, does he wants to
+ *   push all supported algorithms into one process() function,
+ *   or spread it across several ones.
+ *   I.E. More flexibility for PMD writer.
Quoted 7 lines

Which process function should be chosen is internal to PMD, how would that info be visible to the application or the library. These will get stored in the session private data. It would be upto the PMD writer, to store the per session process function in the session private data.

Process function would be a dev ops just like enc/deq operations and it should call The respective process API stored in the session private data.

That model (via devops) is possible, but has several drawbacks from my perspective:

1. It means we'll need to pass dev_id as a parameter to process() function. Though in fact dev_id is not a relevant information for us here
(all we need is pointer to the session and pointer to the fuction to call) and I tried to avoid using it in data-path functions for that API.
2. As you pointed in that case it will be just one process() function per device. So if PMD would like to have several process() functions for different type of sessions (let say one per alg) first thing it has to do inside it's process() - read session data and based on that, do a jump/call to particular internal sub-routine.
Something like:
driver_id = get_pmd_driver_id();
priv_ses = ses->sess_data[driver_id];
Then either:
switch(priv_sess->alg) {case XXX: process_XXX(priv_sess, ...);break;...}
OR
priv_ses->process(priv_sess, ...);

to select and call the proper function.
Looks like totally unnecessary overhead to me.
Though if we'll have ability to query/extract some sort session_ops based on the xform - we can avoid this extra de-refererence+jump/call thing.

Quoted 10 lines

I am not sure if you would need a new session init API for this as nothing would be visible to the app or lib.

lib/librte_cryptodev/rte_cryptodev.h void * · line 1298

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+ * - Not storing process() pointer inside the session -
+ *   Allows user to choose does he want to store a process() pointer
+ *   per session, or per group of sessions for that device that share
+ *   the same input xforms. I.E. extra flexibility for the user,
+ *   plus allows us to keep cpu_sym_session totally opaque, see above.
Quoted 3 lines

If multiple sessions need to be processed via the same process function, PMD would save the same process in all the sessions, I don't think there would be any perf overhead with that.

I think it would, see above.

Quoted 116 lines
Quoted 116 lines
lib/librte_cryptodev/rte_cryptodev.h +71 -0
+ * Sketched usage model:
+ * ....
+ * /* control path, alloc/init session */
+ * int32_t sz = rte_crypto_cpu_sym_session_size(dev_id, &xform);
+ * struct rte_crypto_cpu_sym_session *ses = user_alloc(..., sz);
+ * rte_crypto_cpu_sym_process_t process =
+ *     rte_crypto_cpu_sym_session_func(dev_id, &xform);
+ * rte_crypto_cpu_sym_session_init(dev_id, ses, &xform);
+ * ...
+ * /* data-path*/
+ * process(ses, ....);
+ * ....
+ * /* control path, termiante/free session */
+ * rte_crypto_cpu_sym_session_fini(dev_id, ses);
+ */
+
+/**
+ * vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_crypto_vec {
+       struct iovec *vec;
+       uint32_t num;
+};
+
+/*
+ * Data-path bulk process crypto function.
+ */
+typedef void (*rte_crypto_cpu_sym_process_t)(
+               struct rte_crypto_cpu_sym_session *sess,
+               struct rte_crypto_vec buf[], void *iv[], void *aad[],
+               void *digest[], int status[], uint32_t num);
+/*
+ * for given device return process function specific to input xforms
+ * on error - return NULL and set rte_errno value.
+ * Note that for same input xfroms for the same device should return
+ * the same process function.
+ */
+__rte_experimental
+rte_crypto_cpu_sym_process_t
+rte_crypto_cpu_sym_session_func(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Return required session size in bytes for given set of xforms.
+ * if xforms == NULL, then return the max possible session size,
+ * that would fit session for any supported by the device algorithm.
+ * if CPU mode is not supported at all, or requeted in xform
+ * algorithm is not supported, then return -ENOTSUP.
+ */
+__rte_experimental
+int
+rte_crypto_cpu_sym_session_size(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Initialize session.
+ * It is caller responsibility to allocate enough space for it.
+ * See rte_crypto_cpu_sym_session_size above.
+ */
+__rte_experimental
+int rte_crypto_cpu_sym_session_init(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+__rte_experimental
+void
+rte_crypto_cpu_sym_session_fini(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +11 -0
index defe05ea0..ed7e63fab 100644
@@ -310,6 +310,20 @@ typedef void (*cryptodev_sym_free_session_t)(struct
 rte_cryptodev *dev,
 typedef void (*cryptodev_asym_free_session_t)(struct rte_cryptodev *dev,
                struct rte_cryptodev_asym_session *sess);
 
+typedef int (*cryptodev_cpu_sym_session_size_t) (struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef int (*cryptodev_cpu_sym_session_init_t) (struct rte_cryptodev *dev,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef void (*cryptodev_cpu_sym_session_fini_t) (struct rte_cryptodev *dev,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+typedef rte_crypto_cpu_sym_process_t (*cryptodev_cpu_sym_session_func_t)

(

lib/librte_cryptodev/rte_cryptodev_pmd.h +8 -0
+                       struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
        cryptodev_configure_t dev_configure;    /**< Configure device. */
@@ -343,6 +357,11 @@ struct rte_cryptodev_ops {
        /**< Clear a Crypto sessions private data. */
        cryptodev_asym_free_session_t asym_session_clear;
        /**< Clear a Crypto sessions private data. */
+
+       cryptodev_cpu_sym_session_size_t sym_cpu_session_get_size;
+       cryptodev_cpu_sym_session_func_t sym_cpu_session_get_func;
+       cryptodev_cpu_sym_session_init_t sym_cpu_session_init;
+       cryptodev_cpu_sym_session_fini_t sym_cpu_session_fini;
 };
 
 
#19 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 186 lines

Hi Akhil,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto

workload

Quoted 15 lines
Quoted 15 lines

using

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

the

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

same

Quoted 1 line
Quoted 1 line

algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This flexible action type does not require external hardware

involvement,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously, and is

more

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

performant

Quoted 1 line
Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on removed

"async

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

mode

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto ops.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Does that mean application will not call the

cryptodev_enqueue_burst

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it

will

Quoted 10 lines
Quoted 10 lines

have

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers,

not

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

mbufs.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the

conventional

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto lib

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol

processing

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

or

Quoted 3 lines
Quoted 3 lines

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous

crypto

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a

crypto

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

session

Quoted 3 lines
Quoted 3 lines

in

Quoted 2 lines
Quoted 2 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous

processing.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead? The main reason is that would require disruptive changes in

existing

Quoted 9 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

cryptodev

Quoted 3 lines
Quoted 3 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need

some

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

extra

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something

extra

Quoted 6 lines
Quoted 6 lines

in

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

future).

Quoted 2 lines

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown

current

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't change

from

Quoted 8 lines
Quoted 8 lines

packet

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you can

have

Quoted 3 lines
Quoted 3 lines

it as

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be beneficial

in

Quoted 3 lines
Quoted 3 lines

case of

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

other crypto cases as well.

Quoted 1 line

We can have cipher offset at both places(crypto-op and

cipher_xform). It

Quoted 6 lines
Quoted 6 lines

will

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for CPU

Quoted 8 lines
Quoted 8 lines

related

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

stuff here?

I also thought of adding new xforms, but that wont serve the purpose for

may be all the cases.

Quoted 1 line
Quoted 1 line

You would be needing all information currently available in the current

xforms.

Quoted 1 line
Quoted 1 line

So if you are adding new fields in the new xform, the size will be more

than

Quoted 4 lines
Quoted 4 lines

that of the union of xforms.

Quoted 3 lines
Quoted 3 lines

ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done, then

that

Quoted 8 lines
Quoted 8 lines

can be done for each of the

Quoted 7 lines
Quoted 1 line

Xforms and we can have a solution to this issue.

I think that we can re-use iv.offset for our purposes (for crypto offset). So for now we can make that path work without any ABI breakage. Fan, please feel free to correct me here, if I missed something. If in future we would need to add some extra information it might require ABI breakage, though by now I don't envision anything particular to

Quoted 6 lines
Quoted 6 lines

add.

Quoted 3 lines

Anyway, if there is no objection to go that way, we can try to make these changes for v2.

Actually, after looking at it more deeply it appears not that easy as I thought

it

Quoted 5 lines
Quoted 5 lines

would be :)
Below is a very draft version of proposed API additions.
I think it avoids ABI breakages right now and provides enough flexibility for future extensions (if any).
For now, it doesn't address your comments about naming conventions

(_CPU_

Quoted 5 lines
Quoted 5 lines

vs _SYNC_) , etc.
but I suppose is comprehensive enough to provide a main idea beyond it. Akhil and other interested parties, please try to review and provide feedback ASAP,
as related changes would take some time and we still like to hit 19.11

deadline.

Quoted 38 lines
Quoted 25 lines

Konstantin

diff --git a/lib/librte_cryptodev/rte_crypto_sym.h
b/lib/librte_cryptodev/rte_crypto_sym.h index bc8da2466..c03069e23 100644

lib/librte_cryptodev/rte_crypto_sym.h +6 -1
@@ -103,6 +103,9 @@ rte_crypto_cipher_operation_strings[];
  *
  * This structure contains data relating to Cipher (Encryption and Decryption)
  *  use to create a session.
+ * Actually I was wrong saying that we don't have free space inside xforms.
+ * Making key struct packed (see below) allow us to regain 6B that could be
+ * used for future extensions.
  */
 struct rte_crypto_cipher_xform {
        enum rte_crypto_cipher_operation op;
@@ -116,7 +119,25 @@ struct rte_crypto_cipher_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /**

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 134

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +10 -0
+         * offset for cipher to start within user provided data buffer.
+        * Fan suggested another (and less space consuming way) -
+         * reuse iv.offset space below, by changing:
+        * struct {uint16_t offset, length;} iv;
+        * to uunamed union:
+        * union {
+        *      struct {uint16_t offset, length;} iv;
+        *      struct {uint16_t iv_len, crypto_offset} cpu_crypto_param;
+        * };
+        * Both approaches seems ok to me in general.

No strong opinions here. OK with this one.

Quoted 7 lines
Quoted 3 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 137

Quoted 3 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+        * Comments/suggestions are welcome.
+         */
+       uint16_t offset;

After another thought - it is probably a bit better to have offset as a separate field.
In that case we can use the same xforms to create both type of sessions.

ok

Quoted 77 lines
Quoted 7 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform {

Quoted 7 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+
+       uint8_t reserved1[4];
+
        /**< Cipher key
         *
         * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation, key.data

will

Quoted 41 lines
Quoted 41 lines
lib/librte_cryptodev/rte_crypto_sym.h +9 -2
@@ -284,7 +305,7 @@ struct rte_crypto_auth_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
        /**< Authentication key data.
         * The authentication key length MUST be less than or equal to the
         * block size of the algorithm. It is the callers responsibility to
@@ -292,6 +313,8 @@ struct rte_crypto_auth_xform {
         * (for example RFC 2104, FIPS 198a).
         */
 
+       uint8_t reserved1[6];
+
        struct {
                uint16_t offset;
                /**< Starting point for Initialisation Vector or Counter,
@@ -376,7 +399,12 @@ struct rte_crypto_aead_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /** offset for cipher to start within data buffer */
+       uint16_t cipher_offset;
+
+       uint8_t reserved1[4];
 
        struct {
                uint16_t offset;
lib/librte_cryptodev/rte_cryptodev.h +0 -0
index e175b838c..c0c7bfed7 100644
@@ -1272,6 +1272,101 @@ void *
 rte_cryptodev_sym_session_get_user_data(
                                        struct rte_cryptodev_sym_session *sess);
 
Quoted 9 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1279

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+/*
+ * After several thoughts decided not to try to squeeze CPU_CRYPTO
+ * into existing rte_crypto_sym_session structure/API, but instead
+ * introduce an extentsion to it via new fully opaque
+ * struct rte_crypto_cpu_sym_session and additional related API.

What all things do we need to squeeze?
In this proposal I do not see the new struct cpu_sym_session defined here.

The plan is to have it totally opaque to the user, i.e. just:
struct rte_crypto_cpu_sym_session;
in public header files.

Quoted 1 line

I believe you will have same lib API/struct for cpu_sym_session and

sym_session.

I thought about such way, but there are few things that looks clumsy to me:
1. Right now there is no 'type' (or so) field inside rte_cryptodev_sym_session, so it is not possible to easy distinguish what session do you have: lksd_sym or cpu_sym.
In theory, there is a hole of 4B inside rte_cryptodev_sym_session, so we can add some extra field
here, but in that case we wouldn't be able to use the same xform for both lksd_sym or cpu_sym
(which seems really plausible thing for me).
2. Majority of rte_cryptodev_sym_session fields I think are unnecessary for rte_crypto_cpu_sym_session:

Quoted 9 lines

sess_data[], opaque_data, user_data, nb_drivers.
All that consumes space, that could be used somewhere else instead.
3. I am a bit reluctant to touch existing rte_cryptodev API - to avoid any breakages I can't foresee right now.
From other side - if we'll add new functions/structs for cpu_sym_session we can mark it
and keep it for some time as experimental, so further changes (if needed) would still be possible.

OK let us assume that you have a separate structure. But I have a few queries:
1. how can multiple drivers use a same session
2. Can somebody use the scheduler pmd for scheduling the different type of payloads for the same session?

With your proposal the APIs would be very specific to your use case only.
When you would add more functionality to this sync API/struct, it will end up being the same API/struct.

Let us see how close/ far we are from the existing APIs when the actual implementation is done.

Quoted 35 lines
Quoted 2 lines

I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is

supported(from feature flag) and

Quoted 1 line

Have relevant fields in xform(the newly added ones which are packed as per

your suggestions) set,

Quoted 18 lines

It will create that type of session.

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ * Main points:
+ * - Current crypto-dev API is reasonably mature and it is desirable
+ *   to keep it unchanged (API/ABI stability). From other side, this
+ *   new sync API is new one and probably would require extra changes.

lib/librte_cryptodev/rte_cryptodev.h void * · line 1293

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ *   Having it as a new one allows to mark it as experimental, without
+ *   affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more flexibility
+ *   to the PMD writers and again allows to avoid ABI breakages in future.
+ * - process() function per set of xforms
+ *   allows to expose different process() functions for different
+ *   xform combinations. PMD writer can decide, does he wants to
+ *   push all supported algorithms into one process() function,
+ *   or spread it across several ones.
+ *   I.E. More flexibility for PMD writer.
Quoted 1 line

Which process function should be chosen is internal to PMD, how would that

info

Quoted 1 line

be visible to the application or the library. These will get stored in the session

private

Quoted 1 line

data. It would be upto the PMD writer, to store the per session process

function in

Quoted 3 lines

the session private data.

Process function would be a dev ops just like enc/deq operations and it should

call

Quoted 1 line

The respective process API stored in the session private data.

Quoted 7 lines

That model (via devops) is possible, but has several drawbacks from my perspective:

1. It means we'll need to pass dev_id as a parameter to process() function. Though in fact dev_id is not a relevant information for us here (all we need is pointer to the session and pointer to the fuction to call) and I tried to avoid using it in data-path functions for that API.

You have a single vdev, but someone may have multiple vdevs for each thread, or may Have same dev with multiple queues for each core.

Quoted 14 lines

2. As you pointed in that case it will be just one process() function per device. So if PMD would like to have several process() functions for different type of sessions
(let say one per alg) first thing it has to do inside it's process() - read session data and
based on that, do a jump/call to particular internal sub-routine. Something like:
driver_id = get_pmd_driver_id();
priv_ses = ses->sess_data[driver_id];
Then either:
switch(priv_sess->alg) {case XXX: process_XXX(priv_sess, ...);break;...} OR
priv_ses->process(priv_sess, ...);

Quoted 5 lines

to select and call the proper function.
Looks like totally unnecessary overhead to me.
Though if we'll have ability to query/extract some sort session_ops based on the xform -
we can avoid this extra de-refererence+jump/call thing.

What is the issue in the priv_ses->process(); approach?
I don't understand what are you saving by not doing this.
In any case you would need to identify which session correspond to which process(). For that you would be doing it somewhere in your data path.

Quoted 139 lines
Quoted 3 lines

I am not sure if you would need a new session init API for this as nothing would

be visible to

Quoted 11 lines

the app or lib.

lib/librte_cryptodev/rte_cryptodev.h void * · line 1298

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+ * - Not storing process() pointer inside the session -
+ *   Allows user to choose does he want to store a process() pointer
+ *   per session, or per group of sessions for that device that share
+ *   the same input xforms. I.E. extra flexibility for the user,
+ *   plus allows us to keep cpu_sym_session totally opaque, see above.

If multiple sessions need to be processed via the same process function, PMD would save the same process in all the sessions, I don't think there would be any perf overhead with that.

I think it would, see above.

Quoted 73 lines
Quoted 73 lines
lib/librte_cryptodev/rte_cryptodev.h +71 -0
+ * Sketched usage model:
+ * ....
+ * /* control path, alloc/init session */
+ * int32_t sz = rte_crypto_cpu_sym_session_size(dev_id, &xform);
+ * struct rte_crypto_cpu_sym_session *ses = user_alloc(..., sz);
+ * rte_crypto_cpu_sym_process_t process =
+ *     rte_crypto_cpu_sym_session_func(dev_id, &xform);
+ * rte_crypto_cpu_sym_session_init(dev_id, ses, &xform);
+ * ...
+ * /* data-path*/
+ * process(ses, ....);
+ * ....
+ * /* control path, termiante/free session */
+ * rte_crypto_cpu_sym_session_fini(dev_id, ses);
+ */
+
+/**
+ * vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_crypto_vec {
+       struct iovec *vec;
+       uint32_t num;
+};
+
+/*
+ * Data-path bulk process crypto function.
+ */
+typedef void (*rte_crypto_cpu_sym_process_t)(
+               struct rte_crypto_cpu_sym_session *sess,
+               struct rte_crypto_vec buf[], void *iv[], void *aad[],
+               void *digest[], int status[], uint32_t num);
+/*
+ * for given device return process function specific to input xforms
+ * on error - return NULL and set rte_errno value.
+ * Note that for same input xfroms for the same device should return
+ * the same process function.
+ */
+__rte_experimental
+rte_crypto_cpu_sym_process_t
+rte_crypto_cpu_sym_session_func(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Return required session size in bytes for given set of xforms.
+ * if xforms == NULL, then return the max possible session size,
+ * that would fit session for any supported by the device algorithm.
+ * if CPU mode is not supported at all, or requeted in xform
+ * algorithm is not supported, then return -ENOTSUP.
+ */
+__rte_experimental
+int
+rte_crypto_cpu_sym_session_size(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Initialize session.
+ * It is caller responsibility to allocate enough space for it.
+ * See rte_crypto_cpu_sym_session_size above.
+ */
+__rte_experimental
+int rte_crypto_cpu_sym_session_init(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+__rte_experimental
+void
+rte_crypto_cpu_sym_session_fini(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+
 #ifdef __cplusplus
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +0 -0
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
index defe05ea0..ed7e63fab 100644
@@ -310,6 +310,20 @@ typedef void

(*cryptodev_sym_free_session_t)(struct

Quoted 5 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void (*cryptodev_sym_free_session_t)(struct rte_cryptodev *dev,

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +1 -0
 rte_cryptodev *dev,
 typedef void (*cryptodev_asym_free_session_t)(struct rte_cryptodev *dev,
                struct rte_cryptodev_asym_session *sess);
 
+typedef int (*cryptodev_cpu_sym_session_size_t) (struct rte_cryptodev

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 317

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef int (*cryptodev_cpu_sym_session_init_t) (struct rte_cryptodev

*dev,

Quoted 4 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 321

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +4 -0
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef void (*cryptodev_cpu_sym_session_fini_t) (struct rte_cryptodev

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 324

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       struct rte_crypto_cpu_sym_session *sess);
+
+typedef rte_crypto_cpu_sym_process_t

(*cryptodev_cpu_sym_session_func_t)

Quoted 20 lines
Quoted 20 lines

(

lib/librte_cryptodev/rte_cryptodev_pmd.h +8 -0
+                       struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
        cryptodev_configure_t dev_configure;    /**< Configure device. */
@@ -343,6 +357,11 @@ struct rte_cryptodev_ops {
        /**< Clear a Crypto sessions private data. */
        cryptodev_asym_free_session_t asym_session_clear;
        /**< Clear a Crypto sessions private data. */
+
+       cryptodev_cpu_sym_session_size_t sym_cpu_session_get_size;
+       cryptodev_cpu_sym_session_func_t sym_cpu_session_get_func;
+       cryptodev_cpu_sym_session_init_t sym_cpu_session_init;
+       cryptodev_cpu_sym_session_fini_t sym_cpu_session_fini;
 };
 
 
#20 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Quoted 278 lines
Quoted 183 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto

workload

Quoted 15 lines
Quoted 15 lines

using

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

the

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

same

Quoted 1 line
Quoted 1 line

algorithm, key, and direction being processed by CPU cycles

synchronously.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This flexible action type does not require external hardware

involvement,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously, and is

more

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

performant

Quoted 1 line
Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on removed

"async

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

mode

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto ops.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Does that mean application will not call the

cryptodev_enqueue_burst

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

corresponding dequeue burst.

Yes, instead it just call rte_security_process_cpu_crypto_bulk(...)

Quoted 1 line

It would be a new API something like process_packets and it

will

Quoted 10 lines
Quoted 10 lines

have

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data buffers,

not

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

mbufs.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the

conventional

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto lib

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol

processing

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

or

Quoted 3 lines
Quoted 3 lines

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a synchronous

crypto

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a

crypto

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

session

Quoted 3 lines
Quoted 3 lines

in

Quoted 2 lines
Quoted 2 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous

processing.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead? The main reason is that would require disruptive changes in

existing

Quoted 9 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

cryptodev

Quoted 3 lines
Quoted 3 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO need

some

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

extra

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be something

extra

Quoted 6 lines
Quoted 6 lines

in

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

future).

Quoted 2 lines

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown

current

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't change

from

Quoted 8 lines
Quoted 8 lines

packet

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you can

have

Quoted 3 lines
Quoted 3 lines

it as

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be beneficial

in

Quoted 3 lines
Quoted 3 lines

case of

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

other crypto cases as well.

Quoted 1 line

We can have cipher offset at both places(crypto-op and

cipher_xform). It

Quoted 6 lines
Quoted 6 lines

will

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for CPU

Quoted 8 lines
Quoted 8 lines

related

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

stuff here?

I also thought of adding new xforms, but that wont serve the purpose for

may be all the cases.

Quoted 1 line
Quoted 1 line

You would be needing all information currently available in the current

xforms.

Quoted 1 line
Quoted 1 line

So if you are adding new fields in the new xform, the size will be more

than

Quoted 4 lines
Quoted 4 lines

that of the union of xforms.

Quoted 3 lines
Quoted 3 lines

ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done, then

that

Quoted 8 lines
Quoted 8 lines

can be done for each of the

Quoted 7 lines
Quoted 1 line

Xforms and we can have a solution to this issue.

I think that we can re-use iv.offset for our purposes (for crypto offset). So for now we can make that path work without any ABI breakage. Fan, please feel free to correct me here, if I missed something. If in future we would need to add some extra information it might require ABI breakage, though by now I don't envision anything particular to

Quoted 6 lines
Quoted 6 lines

add.

Quoted 3 lines

Anyway, if there is no objection to go that way, we can try to make these changes for v2.

Actually, after looking at it more deeply it appears not that easy as I thought

it

Quoted 5 lines
Quoted 5 lines

would be :)
Below is a very draft version of proposed API additions.
I think it avoids ABI breakages right now and provides enough flexibility for future extensions (if any).
For now, it doesn't address your comments about naming conventions

(_CPU_

Quoted 5 lines
Quoted 5 lines

vs _SYNC_) , etc.
but I suppose is comprehensive enough to provide a main idea beyond it. Akhil and other interested parties, please try to review and provide feedback ASAP,
as related changes would take some time and we still like to hit 19.11

deadline.

Quoted 38 lines
Quoted 25 lines

Konstantin

diff --git a/lib/librte_cryptodev/rte_crypto_sym.h
b/lib/librte_cryptodev/rte_crypto_sym.h index bc8da2466..c03069e23 100644

lib/librte_cryptodev/rte_crypto_sym.h +6 -1
@@ -103,6 +103,9 @@ rte_crypto_cipher_operation_strings[];
  *
  * This structure contains data relating to Cipher (Encryption and Decryption)
  *  use to create a session.
+ * Actually I was wrong saying that we don't have free space inside xforms.
+ * Making key struct packed (see below) allow us to regain 6B that could be
+ * used for future extensions.
  */
 struct rte_crypto_cipher_xform {
        enum rte_crypto_cipher_operation op;
@@ -116,7 +119,25 @@ struct rte_crypto_cipher_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /**

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 134

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +10 -0
+         * offset for cipher to start within user provided data buffer.
+        * Fan suggested another (and less space consuming way) -
+         * reuse iv.offset space below, by changing:
+        * struct {uint16_t offset, length;} iv;
+        * to uunamed union:
+        * union {
+        *      struct {uint16_t offset, length;} iv;
+        *      struct {uint16_t iv_len, crypto_offset} cpu_crypto_param;
+        * };
+        * Both approaches seems ok to me in general.

No strong opinions here. OK with this one.

Quoted 7 lines
Quoted 3 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 137

Quoted 3 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+        * Comments/suggestions are welcome.
+         */
+       uint16_t offset;

After another thought - it is probably a bit better to have offset as a separate field.
In that case we can use the same xforms to create both type of sessions.

ok

Quoted 77 lines
Quoted 7 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform {

Quoted 7 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+
+       uint8_t reserved1[4];
+
        /**< Cipher key
         *
         * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation, key.data

will

Quoted 41 lines
Quoted 41 lines
lib/librte_cryptodev/rte_crypto_sym.h +9 -2
@@ -284,7 +305,7 @@ struct rte_crypto_auth_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
        /**< Authentication key data.
         * The authentication key length MUST be less than or equal to the
         * block size of the algorithm. It is the callers responsibility to
@@ -292,6 +313,8 @@ struct rte_crypto_auth_xform {
         * (for example RFC 2104, FIPS 198a).
         */
 
+       uint8_t reserved1[6];
+
        struct {
                uint16_t offset;
                /**< Starting point for Initialisation Vector or Counter,
@@ -376,7 +399,12 @@ struct rte_crypto_aead_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /** offset for cipher to start within data buffer */
+       uint16_t cipher_offset;
+
+       uint8_t reserved1[4];
 
        struct {
                uint16_t offset;
lib/librte_cryptodev/rte_cryptodev.h +0 -0
index e175b838c..c0c7bfed7 100644
@@ -1272,6 +1272,101 @@ void *
 rte_cryptodev_sym_session_get_user_data(
                                        struct rte_cryptodev_sym_session *sess);
 
Quoted 9 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1279

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+/*
+ * After several thoughts decided not to try to squeeze CPU_CRYPTO
+ * into existing rte_crypto_sym_session structure/API, but instead
+ * introduce an extentsion to it via new fully opaque
+ * struct rte_crypto_cpu_sym_session and additional related API.

What all things do we need to squeeze?
In this proposal I do not see the new struct cpu_sym_session defined here.

The plan is to have it totally opaque to the user, i.e. just:
struct rte_crypto_cpu_sym_session;
in public header files.

Quoted 1 line

I believe you will have same lib API/struct for cpu_sym_session and

sym_session.

I thought about such way, but there are few things that looks clumsy to me:
1. Right now there is no 'type' (or so) field inside rte_cryptodev_sym_session, so it is not possible to easy distinguish what session do you have: lksd_sym or cpu_sym.
In theory, there is a hole of 4B inside rte_cryptodev_sym_session, so we can add some extra field
here, but in that case we wouldn't be able to use the same xform for both lksd_sym or cpu_sym
(which seems really plausible thing for me).
2. Majority of rte_cryptodev_sym_session fields I think are unnecessary for rte_crypto_cpu_sym_session:

Quoted 9 lines

sess_data[], opaque_data, user_data, nb_drivers.
All that consumes space, that could be used somewhere else instead.
3. I am a bit reluctant to touch existing rte_cryptodev API - to avoid any breakages I can't foresee right now.
From other side - if we'll add new functions/structs for cpu_sym_session we can mark it
and keep it for some time as experimental, so further changes (if needed) would still be possible.

Quoted 2 lines

OK let us assume that you have a separate structure. But I have a few queries:
1. how can multiple drivers use a same session

As a short answer: they can't.
It is pretty much the same approach as with rte_security - each device needs to create/init its own session. So upper layer would need to maintain its own array (or so) for such case.
Though the question is why would you like to have same session over multiple SW backed devices? As it would be anyway just a synchronous function call that will be executed on the same cpu.

Quoted 1 line

2. Can somebody use the scheduler pmd for scheduling the different type of payloads for the same session?

In theory yes.
Though for that scheduler pmd should have inside it's rte_crypto_cpu_sym_session an array of pointers to the underlying devices sessions.

Quoted 2 lines

With your proposal the APIs would be very specific to your use case only.

Yes in some way.
I consider that API specific for SW backed crypto PMDs.
I can hardly see how any 'real HW' PMDs (lksd-none, lksd-proto) will benefit from it. Current crypto-op API is very much HW oriented.
Which is ok, that's for it was intended for, but I think we also need one that would be designed for SW backed implementation in mind.

Quoted 42 lines

When you would add more functionality to this sync API/struct, it will end up being the same API/struct.

Let us see how close/ far we are from the existing APIs when the actual implementation is done.

Quoted 38 lines
Quoted 2 lines

I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is

supported(from feature flag) and

Quoted 1 line

Have relevant fields in xform(the newly added ones which are packed as per

your suggestions) set,

Quoted 18 lines

It will create that type of session.

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ * Main points:
+ * - Current crypto-dev API is reasonably mature and it is desirable
+ *   to keep it unchanged (API/ABI stability). From other side, this
+ *   new sync API is new one and probably would require extra changes.

lib/librte_cryptodev/rte_cryptodev.h void * · line 1293

Quoted 10 lines
lib/librte_cryptodev/rte_cryptodev.h +10 -0
+ *   Having it as a new one allows to mark it as experimental, without
+ *   affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more flexibility
+ *   to the PMD writers and again allows to avoid ABI breakages in future.
+ * - process() function per set of xforms
+ *   allows to expose different process() functions for different
+ *   xform combinations. PMD writer can decide, does he wants to
+ *   push all supported algorithms into one process() function,
+ *   or spread it across several ones.
+ *   I.E. More flexibility for PMD writer.
Quoted 1 line

Which process function should be chosen is internal to PMD, how would that

info

Quoted 1 line

be visible to the application or the library. These will get stored in the session

private

Quoted 1 line

data. It would be upto the PMD writer, to store the per session process

function in

Quoted 3 lines

the session private data.

Process function would be a dev ops just like enc/deq operations and it should

call

Quoted 1 line

The respective process API stored in the session private data.

That model (via devops) is possible, but has several drawbacks from my perspective:

Quoted 7 lines
Quoted 4 lines

1. It means we'll need to pass dev_id as a parameter to process() function. Though in fact dev_id is not a relevant information for us here (all we need is pointer to the session and pointer to the fuction to call) and I tried to avoid using it in data-path functions for that API.

You have a single vdev, but someone may have multiple vdevs for each thread, or may Have same dev with multiple queues for each core.

That's fine. As I said above it is a SW backed implementation.
Each session has to be a separate entity that contains all necessary information (keys, alg/mode info, etc.) to process input buffers.
Plus we need the actual function pointer to call.
I just don't see what for we need a dev_id in that situation. Again, here we don't need care about queues and their pinning to cores. If let say someone would like to process buffers from the same IPsec SA on 2 different cores in parallel, he can just create 2 sessions for the same xform, give one to thread #1 and second to thread #2.
After that both threads are free to call process(this_thread_ses, ...) at will.

Quoted 15 lines
Quoted 15 lines

2. As you pointed in that case it will be just one process() function per device. So if PMD would like to have several process() functions for different type of sessions
(let say one per alg) first thing it has to do inside it's process() - read session data and
based on that, do a jump/call to particular internal sub-routine. Something like:
driver_id = get_pmd_driver_id();
priv_ses = ses->sess_data[driver_id];
Then either:
switch(priv_sess->alg) {case XXX: process_XXX(priv_sess, ...);break;...} OR
priv_ses->process(priv_sess, ...);

Quoted 7 lines
Quoted 5 lines

to select and call the proper function.
Looks like totally unnecessary overhead to me.
Though if we'll have ability to query/extract some sort session_ops based on the xform -
we can avoid this extra de-refererence+jump/call thing.

What is the issue in the priv_ses->process(); approach?

Nothing at all.
What I am saying that schema with dev_ops dev[dev_id]->dev_ops.process(ses->priv_ses[driver_id], ...)
|
|-> priv_ses->process(...)

Has bigger overhead then just:
process(ses,...);

So what for to introduce extra-level of indirection here?

Quoted 2 lines

I don't understand what are you saving by not doing this.
In any case you would need to identify which session correspond to which process().

Yes, sure, but I think we can make user to store information that relationship, in a way he likes: store process() pointer for each session, or group sessions that share the same process() somehow, or...

Quoted 1 line

For that you would be doing it somewhere in your data path.

Why at data-path?
Only once at session creation/initialization time. Or might be even once per group of sessions.

Quoted 140 lines
Quoted 140 lines
Quoted 4 lines

I am not sure if you would need a new session init API for this as nothing would

be visible to

Quoted 11 lines

the app or lib.

lib/librte_cryptodev/rte_cryptodev.h void * · line 1298

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+ * - Not storing process() pointer inside the session -
+ *   Allows user to choose does he want to store a process() pointer
+ *   per session, or per group of sessions for that device that share
+ *   the same input xforms. I.E. extra flexibility for the user,
+ *   plus allows us to keep cpu_sym_session totally opaque, see above.

If multiple sessions need to be processed via the same process function, PMD would save the same process in all the sessions, I don't think there would be any perf overhead with that.

I think it would, see above.

Quoted 73 lines
Quoted 73 lines
lib/librte_cryptodev/rte_cryptodev.h +71 -0
+ * Sketched usage model:
+ * ....
+ * /* control path, alloc/init session */
+ * int32_t sz = rte_crypto_cpu_sym_session_size(dev_id, &xform);
+ * struct rte_crypto_cpu_sym_session *ses = user_alloc(..., sz);
+ * rte_crypto_cpu_sym_process_t process =
+ *     rte_crypto_cpu_sym_session_func(dev_id, &xform);
+ * rte_crypto_cpu_sym_session_init(dev_id, ses, &xform);
+ * ...
+ * /* data-path*/
+ * process(ses, ....);
+ * ....
+ * /* control path, termiante/free session */
+ * rte_crypto_cpu_sym_session_fini(dev_id, ses);
+ */
+
+/**
+ * vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_crypto_vec {
+       struct iovec *vec;
+       uint32_t num;
+};
+
+/*
+ * Data-path bulk process crypto function.
+ */
+typedef void (*rte_crypto_cpu_sym_process_t)(
+               struct rte_crypto_cpu_sym_session *sess,
+               struct rte_crypto_vec buf[], void *iv[], void *aad[],
+               void *digest[], int status[], uint32_t num);
+/*
+ * for given device return process function specific to input xforms
+ * on error - return NULL and set rte_errno value.
+ * Note that for same input xfroms for the same device should return
+ * the same process function.
+ */
+__rte_experimental
+rte_crypto_cpu_sym_process_t
+rte_crypto_cpu_sym_session_func(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Return required session size in bytes for given set of xforms.
+ * if xforms == NULL, then return the max possible session size,
+ * that would fit session for any supported by the device algorithm.
+ * if CPU mode is not supported at all, or requeted in xform
+ * algorithm is not supported, then return -ENOTSUP.
+ */
+__rte_experimental
+int
+rte_crypto_cpu_sym_session_size(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Initialize session.
+ * It is caller responsibility to allocate enough space for it.
+ * See rte_crypto_cpu_sym_session_size above.
+ */
+__rte_experimental
+int rte_crypto_cpu_sym_session_init(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+__rte_experimental
+void
+rte_crypto_cpu_sym_session_fini(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+
 #ifdef __cplusplus
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +0 -0
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
index defe05ea0..ed7e63fab 100644
@@ -310,6 +310,20 @@ typedef void

(*cryptodev_sym_free_session_t)(struct

Quoted 5 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void (*cryptodev_sym_free_session_t)(struct rte_cryptodev *dev,

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +1 -0
 rte_cryptodev *dev,
 typedef void (*cryptodev_asym_free_session_t)(struct rte_cryptodev *dev,
                struct rte_cryptodev_asym_session *sess);
 
+typedef int (*cryptodev_cpu_sym_session_size_t) (struct rte_cryptodev

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 317

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef int (*cryptodev_cpu_sym_session_init_t) (struct rte_cryptodev

*dev,

Quoted 4 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 321

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +4 -0
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef void (*cryptodev_cpu_sym_session_fini_t) (struct rte_cryptodev

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 324

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       struct rte_crypto_cpu_sym_session *sess);
+
+typedef rte_crypto_cpu_sym_process_t

(*cryptodev_cpu_sym_session_func_t)

Quoted 20 lines
Quoted 20 lines

(

lib/librte_cryptodev/rte_cryptodev_pmd.h +8 -0
+                       struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
        cryptodev_configure_t dev_configure;    /**< Configure device. */
@@ -343,6 +357,11 @@ struct rte_cryptodev_ops {
        /**< Clear a Crypto sessions private data. */
        cryptodev_asym_free_session_t asym_session_clear;
        /**< Clear a Crypto sessions private data. */
+
+       cryptodev_cpu_sym_session_size_t sym_cpu_session_get_size;
+       cryptodev_cpu_sym_session_func_t sym_cpu_session_get_func;
+       cryptodev_cpu_sym_session_init_t sym_cpu_session_init;
+       cryptodev_cpu_sym_session_fini_t sym_cpu_session_fini;
 };
 
 
#21 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 311 lines

Hi Akhil,

Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto

workload

Quoted 4 lines
Quoted 4 lines

using

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

same

Quoted 1 line

algorithm, key, and direction being processed by CPU

cycles

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

synchronously.

Quoted 1 line
Quoted 1 line
Quoted 1 line

This flexible action type does not require external

hardware

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

involvement,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously,

and is

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

more

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

performant

Quoted 1 line

than Cryptodev SW PMD due to the saved cycles on

removed

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

"async

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

mode

Quoted 1 line
Quoted 1 line
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto

ops.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Does that mean application will not call the

cryptodev_enqueue_burst

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

and

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

corresponding dequeue burst.

Yes, instead it just call

rte_security_process_cpu_crypto_bulk(...)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It would be a new API something like process_packets and

it

Quoted 6 lines
Quoted 6 lines

will

Quoted 5 lines
Quoted 5 lines

have

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data

buffers,

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines

not

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

mbufs.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the

conventional

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto lib

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

only.

Quoted 1 line

As far as I can understand, you are not doing any protocol

processing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

or

Quoted 3 lines
Quoted 3 lines

any

Quoted 2 lines
Quoted 2 lines

value add

Quoted 1 line

To the crypto processing. IMO, you just need a

synchronous

Quoted 11 lines
Quoted 11 lines
Quoted 4 lines
Quoted 4 lines

crypto

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

processing

Quoted 2 lines
Quoted 2 lines

API which

Quoted 1 line

Can be defined in cryptodev, you don't need to re-create a

crypto

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

session

Quoted 3 lines
Quoted 3 lines

in

Quoted 2 lines
Quoted 2 lines

the name of

Quoted 1 line

Security session in the driver just to do a synchronous

processing.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I suppose your question is why not to have rte_crypot_process_cpu_crypto_bulk(...) instead? The main reason is that would require disruptive changes in

existing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

cryptodev

Quoted 3 lines
Quoted 3 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

need

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

some

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

extra

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain (cipher offset from the start of the buffer, might be

something

Quoted 12 lines
Quoted 12 lines

extra

Quoted 6 lines
Quoted 6 lines

in

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

future).

Quoted 2 lines

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown

current

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't change

from

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

packet

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you

can

Quoted 4 lines
Quoted 4 lines

have

Quoted 3 lines
Quoted 3 lines

it as

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be

beneficial

Quoted 11 lines
Quoted 11 lines

in

Quoted 3 lines
Quoted 3 lines

case of

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

other crypto cases as well.

Quoted 1 line

We can have cipher offset at both places(crypto-op and

cipher_xform). It

Quoted 6 lines
Quoted 6 lines

will

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for

CPU

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

related

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

stuff here?

I also thought of adding new xforms, but that wont serve the purpose

for

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

may be all the cases.

Quoted 1 line
Quoted 1 line

You would be needing all information currently available in the

current

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

xforms.

Quoted 1 line
Quoted 1 line

So if you are adding new fields in the new xform, the size will be more

than

Quoted 4 lines
Quoted 4 lines

that of the union of xforms.

Quoted 3 lines
Quoted 3 lines

ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done, then

that

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

can be done for each of the

Quoted 7 lines
Quoted 1 line

Xforms and we can have a solution to this issue.

I think that we can re-use iv.offset for our purposes (for crypto offset). So for now we can make that path work without any ABI breakage. Fan, please feel free to correct me here, if I missed something. If in future we would need to add some extra information it might require ABI breakage, though by now I don't envision anything

particular to

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

add.

Quoted 3 lines

Anyway, if there is no objection to go that way, we can try to make these changes for v2.

Actually, after looking at it more deeply it appears not that easy as I

thought

Quoted 4 lines
Quoted 4 lines

it

Quoted 3 lines
Quoted 3 lines

would be :)
Below is a very draft version of proposed API additions.
I think it avoids ABI breakages right now and provides enough flexibility

for

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines

future extensions (if any).
For now, it doesn't address your comments about naming conventions

(_CPU_

Quoted 3 lines
Quoted 3 lines

vs _SYNC_) , etc.
but I suppose is comprehensive enough to provide a main idea beyond it. Akhil and other interested parties, please try to review and provide

feedback

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

ASAP,
as related changes would take some time and we still like to hit 19.11

deadline.

Quoted 2 lines
Quoted 2 lines

Konstantin

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_crypto_sym.h rte_crypto_cipher_operation_strings[]; · line 104

Quoted 8 lines

diff --git a/lib/librte_cryptodev/rte_crypto_sym.h
b/lib/librte_cryptodev/rte_crypto_sym.h index bc8da2466..c03069e23 100644

lib/librte_cryptodev/rte_crypto_sym.h +0 -0
@@ -103,6 +103,9 @@ rte_crypto_cipher_operation_strings[];
  *
  * This structure contains data relating to Cipher (Encryption and

Decryption)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

lib/librte_cryptodev/rte_crypto_sym.h rte_crypto_cipher_operation_strings[]; · line 106

Quoted 2 lines
lib/librte_cryptodev/rte_crypto_sym.h +1 -0
  *  use to create a session.
+ * Actually I was wrong saying that we don't have free space inside

xforms.

Quoted 1 line
Quoted 1 line
Quoted 1 line

lib/librte_cryptodev/rte_crypto_sym.h rte_crypto_cipher_operation_strings[]; · line 107

Quoted 1 line
lib/librte_cryptodev/rte_crypto_sym.h +1 -0
+ * Making key struct packed (see below) allow us to regain 6B that could

be

Quoted 32 lines
Quoted 25 lines
Quoted 25 lines
Quoted 12 lines
lib/librte_cryptodev/rte_crypto_sym.h +4 -1
+ * used for future extensions.
  */
 struct rte_crypto_cipher_xform {
        enum rte_crypto_cipher_operation op;
@@ -116,7 +119,25 @@ struct rte_crypto_cipher_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /**

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 134

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +10 -0
+         * offset for cipher to start within user provided data buffer.
+        * Fan suggested another (and less space consuming way) -
+         * reuse iv.offset space below, by changing:
+        * struct {uint16_t offset, length;} iv;
+        * to uunamed union:
+        * union {
+        *      struct {uint16_t offset, length;} iv;
+        *      struct {uint16_t iv_len, crypto_offset} cpu_crypto_param;
+        * };
+        * Both approaches seems ok to me in general.

No strong opinions here. OK with this one.

Quoted 7 lines
Quoted 3 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 137

Quoted 3 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+        * Comments/suggestions are welcome.
+         */
+       uint16_t offset;

After another thought - it is probably a bit better to have offset as a separate field.
In that case we can use the same xforms to create both type of sessions.

Quoted 8 lines

ok

Quoted 7 lines
Quoted 7 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform {

Quoted 7 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+
+       uint8_t reserved1[4];
+
        /**< Cipher key
         *
         * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation,

key.data

Quoted 56 lines
Quoted 56 lines

will

Quoted 41 lines
Quoted 41 lines
lib/librte_cryptodev/rte_crypto_sym.h +9 -2
@@ -284,7 +305,7 @@ struct rte_crypto_auth_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
        /**< Authentication key data.
         * The authentication key length MUST be less than or equal to the
         * block size of the algorithm. It is the callers responsibility to
@@ -292,6 +313,8 @@ struct rte_crypto_auth_xform {
         * (for example RFC 2104, FIPS 198a).
         */
 
+       uint8_t reserved1[6];
+
        struct {
                uint16_t offset;
                /**< Starting point for Initialisation Vector or Counter,
@@ -376,7 +399,12 @@ struct rte_crypto_aead_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /** offset for cipher to start within data buffer */
+       uint16_t cipher_offset;
+
+       uint8_t reserved1[4];
 
        struct {
                uint16_t offset;
lib/librte_cryptodev/rte_cryptodev.h +0 -0
index e175b838c..c0c7bfed7 100644
@@ -1272,6 +1272,101 @@ void *
 rte_cryptodev_sym_session_get_user_data(
                                        struct rte_cryptodev_sym_session *sess);
 
Quoted 9 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1279

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+/*
+ * After several thoughts decided not to try to squeeze CPU_CRYPTO
+ * into existing rte_crypto_sym_session structure/API, but instead
+ * introduce an extentsion to it via new fully opaque
+ * struct rte_crypto_cpu_sym_session and additional related API.

What all things do we need to squeeze?
In this proposal I do not see the new struct cpu_sym_session defined here.

The plan is to have it totally opaque to the user, i.e. just:
struct rte_crypto_cpu_sym_session;
in public header files.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

I believe you will have same lib API/struct for cpu_sym_session and

sym_session.

I thought about such way, but there are few things that looks clumsy to me:
1. Right now there is no 'type' (or so) field inside rte_cryptodev_sym_session, so it is not possible to easy distinguish what session do you have: lksd_sym or cpu_sym.
In theory, there is a hole of 4B inside rte_cryptodev_sym_session, so we can

add

Quoted 11 lines
Quoted 11 lines

some extra field
here, but in that case we wouldn't be able to use the same xform for both lksd_sym or cpu_sym
(which seems really plausible thing for me).
2. Majority of rte_cryptodev_sym_session fields I think are unnecessary for rte_crypto_cpu_sym_session:
sess_data[], opaque_data, user_data, nb_drivers.
All that consumes space, that could be used somewhere else instead.
3. I am a bit reluctant to touch existing rte_cryptodev API - to avoid any breakages I can't foresee right now.
From other side - if we'll add new functions/structs for cpu_sym_session we

can

Quoted 2 lines
Quoted 2 lines

mark it
and keep it for some time as experimental, so further changes (if needed)

would

Quoted 5 lines
Quoted 2 lines

still be possible.

OK let us assume that you have a separate structure. But I have a few queries:
1. how can multiple drivers use a same session

Quoted 8 lines

As a short answer: they can't.
It is pretty much the same approach as with rte_security - each device needs to create/init its own session.
So upper layer would need to maintain its own array (or so) for such case. Though the question is why would you like to have same session over multiple SW backed devices?
As it would be anyway just a synchronous function call that will be executed on the same cpu.

I may have single FAT tunnel which may be distributed over multiple Cores, and each core is affined to a different SW device. So a single session may be accessed by multiple devices.

One more example would be depending on packet sizes, I may switch between HW/SW PMDs with the same session.

Quoted 12 lines
Quoted 2 lines

2. Can somebody use the scheduler pmd for scheduling the different type of

payloads for the same session?

In theory yes.
Though for that scheduler pmd should have inside it's rte_crypto_cpu_sym_session an array of pointers to the underlying devices sessions.

Quoted 2 lines

With your proposal the APIs would be very specific to your use case only.

Quoted 8 lines

Yes in some way.
I consider that API specific for SW backed crypto PMDs.
I can hardly see how any 'real HW' PMDs (lksd-none, lksd-proto) will benefit from it.
Current crypto-op API is very much HW oriented.
Which is ok, that's for it was intended for, but I think we also need one that would be designed
for SW backed implementation in mind.

We may re-use your API for HW PMDs as well which do not have requirement of Crypto-op/mbuf etc.
The return type of your new process API may have a status which say 'processed' Or can be say 'enqueued'. So if it is 'enqueued', we may have a new API for raw Bufs dequeue as well.

This requirement can be for any hardware PMDs like QAT as well. That is why a dev-ops would be a better option.

Quoted 55 lines
Quoted 2 lines

When you would add more functionality to this sync API/struct, it will end up

being the same API/struct.

Quoted 2 lines

Let us see how close/ far we are from the existing APIs when the actual

implementation is done.

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is

supported(from feature flag) and

Quoted 1 line

Have relevant fields in xform(the newly added ones which are packed as

per

Quoted 19 lines
Quoted 19 lines

your suggestions) set,

Quoted 18 lines

It will create that type of session.

Quoted 14 lines
lib/librte_cryptodev/rte_cryptodev.h +14 -0
+ * Main points:
+ * - Current crypto-dev API is reasonably mature and it is desirable
+ *   to keep it unchanged (API/ABI stability). From other side, this
+ *   new sync API is new one and probably would require extra changes.
+ *   Having it as a new one allows to mark it as experimental, without
+ *   affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more flexibility
+ *   to the PMD writers and again allows to avoid ABI breakages in future.
+ * - process() function per set of xforms
+ *   allows to expose different process() functions for different
+ *   xform combinations. PMD writer can decide, does he wants to
+ *   push all supported algorithms into one process() function,
+ *   or spread it across several ones.
+ *   I.E. More flexibility for PMD writer.
Quoted 1 line
Quoted 1 line
Quoted 1 line

Which process function should be chosen is internal to PMD, how would

that

Quoted 2 lines
Quoted 2 lines

info

Quoted 1 line

be visible to the application or the library. These will get stored in the

session

Quoted 6 lines
Quoted 6 lines

private

Quoted 1 line

data. It would be upto the PMD writer, to store the per session process

function in

Quoted 3 lines

the session private data.

Process function would be a dev ops just like enc/deq operations and it

should

Quoted 6 lines
Quoted 6 lines

call

Quoted 1 line

The respective process API stored in the session private data.

That model (via devops) is possible, but has several drawbacks from my perspective:

Quoted 6 lines
Quoted 4 lines

1. It means we'll need to pass dev_id as a parameter to process() function. Though in fact dev_id is not a relevant information for us here (all we need is pointer to the session and pointer to the fuction to call) and I tried to avoid using it in data-path functions for that API.

You have a single vdev, but someone may have multiple vdevs for each thread,

Quoted 8 lines

or may

Quoted 1 line

Have same dev with multiple queues for each core.

That's fine. As I said above it is a SW backed implementation.
Each session has to be a separate entity that contains all necessary information (keys, alg/mode info, etc.) to process input buffers.
Plus we need the actual function pointer to call.
I just don't see what for we need a dev_id in that situation.

To iterate the session private data in the session.

Quoted 5 lines

Again, here we don't need care about queues and their pinning to cores. If let say someone would like to process buffers from the same IPsec SA on 2 different cores in parallel, he can just create 2 sessions for the same xform, give one to thread #1 and second to thread #2.
After that both threads are free to call process(this_thread_ses, ...) at will.

Say you have a 16core device to handle 100G of traffic on a single tunnel. Will we make 16 sessions with same parameters?

Quoted 32 lines
Quoted 6 lines
Quoted 6 lines

2. As you pointed in that case it will be just one process() function per device. So if PMD would like to have several process() functions for different type of sessions
(let say one per alg) first thing it has to do inside it's process() - read session

data

Quoted 10 lines
Quoted 10 lines

and
based on that, do a jump/call to particular internal sub-routine. Something like:
driver_id = get_pmd_driver_id();
priv_ses = ses->sess_data[driver_id];
Then either:
switch(priv_sess->alg) {case XXX: process_XXX(priv_sess, ...);break;...} OR
priv_ses->process(priv_sess, ...);

Quoted 3 lines
Quoted 3 lines

to select and call the proper function.
Looks like totally unnecessary overhead to me.
Though if we'll have ability to query/extract some sort session_ops based on

the

Quoted 4 lines
Quoted 2 lines

xform -
we can avoid this extra de-refererence+jump/call thing.

What is the issue in the priv_ses->process(); approach?

Nothing at all.
What I am saying that schema with dev_ops dev[dev_id]->dev_ops.process(ses->priv_ses[driver_id], ...)
|
|-> priv_ses->process(...)

Quoted 4 lines

Has bigger overhead then just:
process(ses,...);

So what for to introduce extra-level of indirection here?

Explained above.

Quoted 8 lines
Quoted 3 lines

I don't understand what are you saving by not doing this.
In any case you would need to identify which session correspond to which

process().

Yes, sure, but I think we can make user to store information that relationship, in a way he likes: store process() pointer for each session, or group sessions that share the same process() somehow, or...

So whatever relationship that user will make and store will make its life complicated. If we can hide that information in the driver, then what is the issue in that and user Will not need to worry. He would just call the process() and driver will choose which Process need to be called.

I think we should have a POC around this and see the difference in the cycle count. IMO it would be negligible and we would end up making a generic API set which can be used by others as well.

Quoted 150 lines
Quoted 2 lines

For that you would be doing it somewhere in your data path.

Why at data-path?
Only once at session creation/initialization time. Or might be even once per group of sessions.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I am not sure if you would need a new session init API for this as nothing

would

Quoted 11 lines
Quoted 11 lines

be visible to

Quoted 10 lines

the app or lib.

lib/librte_cryptodev/rte_cryptodev.h void * · line 1298

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+ * - Not storing process() pointer inside the session -
+ *   Allows user to choose does he want to store a process() pointer
+ *   per session, or per group of sessions for that device that share
+ *   the same input xforms. I.E. extra flexibility for the user,
+ *   plus allows us to keep cpu_sym_session totally opaque, see above.

If multiple sessions need to be processed via the same process function, PMD would save the same process in all the sessions, I don't think there

would

Quoted 86 lines
Quoted 86 lines
Quoted 1 line

be any perf overhead with that.

I think it would, see above.

Quoted 73 lines
Quoted 73 lines
lib/librte_cryptodev/rte_cryptodev.h +71 -0
+ * Sketched usage model:
+ * ....
+ * /* control path, alloc/init session */
+ * int32_t sz = rte_crypto_cpu_sym_session_size(dev_id, &xform);
+ * struct rte_crypto_cpu_sym_session *ses = user_alloc(..., sz);
+ * rte_crypto_cpu_sym_process_t process =
+ *     rte_crypto_cpu_sym_session_func(dev_id, &xform);
+ * rte_crypto_cpu_sym_session_init(dev_id, ses, &xform);
+ * ...
+ * /* data-path*/
+ * process(ses, ....);
+ * ....
+ * /* control path, termiante/free session */
+ * rte_crypto_cpu_sym_session_fini(dev_id, ses);
+ */
+
+/**
+ * vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_crypto_vec {
+       struct iovec *vec;
+       uint32_t num;
+};
+
+/*
+ * Data-path bulk process crypto function.
+ */
+typedef void (*rte_crypto_cpu_sym_process_t)(
+               struct rte_crypto_cpu_sym_session *sess,
+               struct rte_crypto_vec buf[], void *iv[], void *aad[],
+               void *digest[], int status[], uint32_t num);
+/*
+ * for given device return process function specific to input xforms
+ * on error - return NULL and set rte_errno value.
+ * Note that for same input xfroms for the same device should return
+ * the same process function.
+ */
+__rte_experimental
+rte_crypto_cpu_sym_process_t
+rte_crypto_cpu_sym_session_func(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Return required session size in bytes for given set of xforms.
+ * if xforms == NULL, then return the max possible session size,
+ * that would fit session for any supported by the device algorithm.
+ * if CPU mode is not supported at all, or requeted in xform
+ * algorithm is not supported, then return -ENOTSUP.
+ */
+__rte_experimental
+int
+rte_crypto_cpu_sym_session_size(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Initialize session.
+ * It is caller responsibility to allocate enough space for it.
+ * See rte_crypto_cpu_sym_session_size above.
+ */
+__rte_experimental
+int rte_crypto_cpu_sym_session_init(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+__rte_experimental
+void
+rte_crypto_cpu_sym_session_fini(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+
 #ifdef __cplusplus
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +0 -0
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
index defe05ea0..ed7e63fab 100644
@@ -310,6 +310,20 @@ typedef void

(*cryptodev_sym_free_session_t)(struct

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 311

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
 rte_cryptodev *dev,
 typedef void (*cryptodev_asym_free_session_t)(struct rte_cryptodev

*dev,

Quoted 37 lines
Quoted 37 lines
Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void (*cryptodev_sym_free_session_t)(struct rte_cryptodev *dev,

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +1 -0
                struct rte_cryptodev_asym_session *sess);
 
+typedef int (*cryptodev_cpu_sym_session_size_t) (struct rte_cryptodev

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 317

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef int (*cryptodev_cpu_sym_session_init_t) (struct rte_cryptodev

*dev,

Quoted 4 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 321

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +4 -0
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef void (*cryptodev_cpu_sym_session_fini_t) (struct rte_cryptodev

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 324

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       struct rte_crypto_cpu_sym_session *sess);
+
+typedef rte_crypto_cpu_sym_process_t

(*cryptodev_cpu_sym_session_func_t)

Quoted 20 lines
Quoted 20 lines

(

lib/librte_cryptodev/rte_cryptodev_pmd.h +8 -0
+                       struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
        cryptodev_configure_t dev_configure;    /**< Configure device. */
@@ -343,6 +357,11 @@ struct rte_cryptodev_ops {
        /**< Clear a Crypto sessions private data. */
        cryptodev_asym_free_session_t asym_session_clear;
        /**< Clear a Crypto sessions private data. */
+
+       cryptodev_cpu_sym_session_size_t sym_cpu_session_get_size;
+       cryptodev_cpu_sym_session_func_t sym_cpu_session_get_func;
+       cryptodev_cpu_sym_session_init_t sym_cpu_session_init;
+       cryptodev_cpu_sym_session_fini_t sym_cpu_session_fini;
 };
 
 
#22 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Quoted 317 lines
Quoted 308 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto

workload

Quoted 4 lines
Quoted 4 lines

using

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

same
algorithm, key, and direction being processed by CPU

cycles

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

synchronously.

Quoted 1 line
Quoted 1 line

This flexible action type does not require external

hardware

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

involvement,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously,

and is

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

more

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

performant
than Cryptodev SW PMD due to the saved cycles on

removed

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

"async

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

mode

Quoted 1 line
Quoted 1 line

simulation" as well as 3 cacheline access of the crypto

ops.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Does that mean application will not call the

cryptodev_enqueue_burst

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

and

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

corresponding dequeue burst.

Yes, instead it just call

rte_security_process_cpu_crypto_bulk(...)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It would be a new API something like process_packets and

it

Quoted 6 lines
Quoted 6 lines

will

Quoted 5 lines
Quoted 5 lines

have

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data

buffers,

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines

not

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

mbufs.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the

conventional

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto lib

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

only.
As far as I can understand, you are not doing any protocol

processing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

or

Quoted 3 lines
Quoted 3 lines

any

Quoted 2 lines
Quoted 2 lines

value add
To the crypto processing. IMO, you just need a

synchronous

Quoted 11 lines
Quoted 11 lines
Quoted 4 lines
Quoted 4 lines

crypto

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

processing

Quoted 2 lines
Quoted 2 lines

API which
Can be defined in cryptodev, you don't need to re-create a

crypto

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

session

Quoted 3 lines
Quoted 3 lines

in

Quoted 2 lines
Quoted 2 lines

the name of
Security session in the driver just to do a synchronous

processing.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I suppose your question is why not to have
rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes in

existing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

cryptodev

Quoted 3 lines
Quoted 3 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

need

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

some

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

extra

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be

something

Quoted 12 lines
Quoted 12 lines

extra

Quoted 6 lines
Quoted 6 lines

in

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

future).

Quoted 2 lines

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that slowdown

current

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't change

from

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

packet

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which you

can

Quoted 4 lines
Quoted 4 lines

have

Quoted 3 lines
Quoted 3 lines

it as

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be

beneficial

Quoted 11 lines
Quoted 11 lines

in

Quoted 3 lines
Quoted 3 lines

case of

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

other crypto cases as well.

Quoted 1 line

We can have cipher offset at both places(crypto-op and

cipher_xform). It

Quoted 6 lines
Quoted 6 lines

will

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types for

CPU

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

related

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

stuff here?

I also thought of adding new xforms, but that wont serve the purpose

for

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

may be all the cases.

Quoted 1 line
Quoted 1 line

You would be needing all information currently available in the

current

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

xforms.

Quoted 1 line
Quoted 1 line

So if you are adding new fields in the new xform, the size will be more

than

Quoted 4 lines
Quoted 4 lines

that of the union of xforms.

Quoted 3 lines
Quoted 3 lines

ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done, then

that

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

can be done for each of the

Quoted 7 lines
Quoted 1 line

Xforms and we can have a solution to this issue.

I think that we can re-use iv.offset for our purposes (for crypto offset). So for now we can make that path work without any ABI breakage. Fan, please feel free to correct me here, if I missed something. If in future we would need to add some extra information it might require ABI breakage, though by now I don't envision anything

particular to

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

add.

Quoted 3 lines

Anyway, if there is no objection to go that way, we can try to make these changes for v2.

Actually, after looking at it more deeply it appears not that easy as I

thought

Quoted 4 lines
Quoted 4 lines

it

Quoted 3 lines
Quoted 3 lines

would be :)
Below is a very draft version of proposed API additions.
I think it avoids ABI breakages right now and provides enough flexibility

for

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines

future extensions (if any).
For now, it doesn't address your comments about naming conventions

(_CPU_

Quoted 3 lines
Quoted 3 lines

vs _SYNC_) , etc.
but I suppose is comprehensive enough to provide a main idea beyond it. Akhil and other interested parties, please try to review and provide

feedback

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

ASAP,
as related changes would take some time and we still like to hit 19.11

deadline.

Quoted 2 lines
Quoted 2 lines

Konstantin

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_crypto_sym.h rte_crypto_cipher_operation_strings[]; · line 104

Quoted 8 lines

diff --git a/lib/librte_cryptodev/rte_crypto_sym.h
b/lib/librte_cryptodev/rte_crypto_sym.h index bc8da2466..c03069e23 100644

lib/librte_cryptodev/rte_crypto_sym.h +0 -0
@@ -103,6 +103,9 @@ rte_crypto_cipher_operation_strings[];
  *
  * This structure contains data relating to Cipher (Encryption and

Decryption)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

lib/librte_cryptodev/rte_crypto_sym.h rte_crypto_cipher_operation_strings[]; · line 106

Quoted 2 lines
lib/librte_cryptodev/rte_crypto_sym.h +1 -0
  *  use to create a session.
+ * Actually I was wrong saying that we don't have free space inside

xforms.

Quoted 1 line
Quoted 1 line
Quoted 1 line

lib/librte_cryptodev/rte_crypto_sym.h rte_crypto_cipher_operation_strings[]; · line 107

Quoted 1 line
lib/librte_cryptodev/rte_crypto_sym.h +1 -0
+ * Making key struct packed (see below) allow us to regain 6B that could

be

Quoted 32 lines
Quoted 25 lines
Quoted 25 lines
Quoted 12 lines
lib/librte_cryptodev/rte_crypto_sym.h +4 -1
+ * used for future extensions.
  */
 struct rte_crypto_cipher_xform {
        enum rte_crypto_cipher_operation op;
@@ -116,7 +119,25 @@ struct rte_crypto_cipher_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /**

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 134

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +10 -0
+         * offset for cipher to start within user provided data buffer.
+        * Fan suggested another (and less space consuming way) -
+         * reuse iv.offset space below, by changing:
+        * struct {uint16_t offset, length;} iv;
+        * to uunamed union:
+        * union {
+        *      struct {uint16_t offset, length;} iv;
+        *      struct {uint16_t iv_len, crypto_offset} cpu_crypto_param;
+        * };
+        * Both approaches seems ok to me in general.

No strong opinions here. OK with this one.

Quoted 7 lines
Quoted 3 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 137

Quoted 3 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+        * Comments/suggestions are welcome.
+         */
+       uint16_t offset;

After another thought - it is probably a bit better to have offset as a separate field.
In that case we can use the same xforms to create both type of sessions.

Quoted 8 lines

ok

Quoted 7 lines
Quoted 7 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform {

Quoted 7 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+
+       uint8_t reserved1[4];
+
        /**< Cipher key
         *
         * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation,

key.data

Quoted 56 lines
Quoted 56 lines

will

Quoted 41 lines
Quoted 41 lines
lib/librte_cryptodev/rte_crypto_sym.h +9 -2
@@ -284,7 +305,7 @@ struct rte_crypto_auth_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
        /**< Authentication key data.
         * The authentication key length MUST be less than or equal to the
         * block size of the algorithm. It is the callers responsibility to
@@ -292,6 +313,8 @@ struct rte_crypto_auth_xform {
         * (for example RFC 2104, FIPS 198a).
         */
 
+       uint8_t reserved1[6];
+
        struct {
                uint16_t offset;
                /**< Starting point for Initialisation Vector or Counter,
@@ -376,7 +399,12 @@ struct rte_crypto_aead_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /** offset for cipher to start within data buffer */
+       uint16_t cipher_offset;
+
+       uint8_t reserved1[4];
 
        struct {
                uint16_t offset;
lib/librte_cryptodev/rte_cryptodev.h +0 -0
index e175b838c..c0c7bfed7 100644
@@ -1272,6 +1272,101 @@ void *
 rte_cryptodev_sym_session_get_user_data(
                                        struct rte_cryptodev_sym_session *sess);
 
Quoted 9 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1279

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+/*
+ * After several thoughts decided not to try to squeeze CPU_CRYPTO
+ * into existing rte_crypto_sym_session structure/API, but instead
+ * introduce an extentsion to it via new fully opaque
+ * struct rte_crypto_cpu_sym_session and additional related API.

What all things do we need to squeeze?
In this proposal I do not see the new struct cpu_sym_session defined here.

The plan is to have it totally opaque to the user, i.e. just:
struct rte_crypto_cpu_sym_session;
in public header files.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

I believe you will have same lib API/struct for cpu_sym_session and

sym_session.

I thought about such way, but there are few things that looks clumsy to me:
1. Right now there is no 'type' (or so) field inside rte_cryptodev_sym_session, so it is not possible to easy distinguish what session do you have: lksd_sym or cpu_sym.
In theory, there is a hole of 4B inside rte_cryptodev_sym_session, so we can

add

Quoted 11 lines
Quoted 11 lines

some extra field
here, but in that case we wouldn't be able to use the same xform for both lksd_sym or cpu_sym
(which seems really plausible thing for me).
2. Majority of rte_cryptodev_sym_session fields I think are unnecessary for rte_crypto_cpu_sym_session:
sess_data[], opaque_data, user_data, nb_drivers.
All that consumes space, that could be used somewhere else instead.
3. I am a bit reluctant to touch existing rte_cryptodev API - to avoid any breakages I can't foresee right now.
From other side - if we'll add new functions/structs for cpu_sym_session we

can

Quoted 2 lines
Quoted 2 lines

mark it
and keep it for some time as experimental, so further changes (if needed)

would

Quoted 5 lines
Quoted 2 lines

still be possible.

OK let us assume that you have a separate structure. But I have a few queries:
1. how can multiple drivers use a same session

Quoted 8 lines

As a short answer: they can't.
It is pretty much the same approach as with rte_security - each device needs to create/init its own session.
So upper layer would need to maintain its own array (or so) for such case. Though the question is why would you like to have same session over multiple SW backed devices?
As it would be anyway just a synchronous function call that will be executed on the same cpu.

Quoted 2 lines

I may have single FAT tunnel which may be distributed over multiple Cores, and each core is affined to a different SW device.

If it is pure SW, then we don't need multiple devices for such scenario. Device in that case is pure abstraction that we can skip.

Quoted 4 lines

So a single session may be accessed by multiple devices.

One more example would be depending on packet sizes, I may switch between HW/SW PMDs with the same session.

Sure, but then we'll have multiple sessions.
BTW, we have same thing now - these private session pointers are just stored inside the same rte_crypto_sym_session.
And if user wants to support this model, he would also need to store <dev_id, queue_id> pair for each HW device anyway.

Quoted 22 lines
Quoted 13 lines
Quoted 3 lines

2. Can somebody use the scheduler pmd for scheduling the different type of

payloads for the same session?

In theory yes.
Though for that scheduler pmd should have inside it's rte_crypto_cpu_sym_session an array of pointers to the underlying devices sessions.

Quoted 2 lines

With your proposal the APIs would be very specific to your use case only.

Quoted 8 lines

Yes in some way.
I consider that API specific for SW backed crypto PMDs.
I can hardly see how any 'real HW' PMDs (lksd-none, lksd-proto) will benefit from it.
Current crypto-op API is very much HW oriented.
Which is ok, that's for it was intended for, but I think we also need one that would be designed
for SW backed implementation in mind.

Quoted 7 lines

We may re-use your API for HW PMDs as well which do not have requirement of Crypto-op/mbuf etc.
The return type of your new process API may have a status which say 'processed' Or can be say 'enqueued'. So if it is 'enqueued', we may have a new API for raw Bufs dequeue as well.

This requirement can be for any hardware PMDs like QAT as well.

I don't think it is a good idea to extend this API for async (lookaside) devices. You'll need to:
- provide dev_id and queue_id for each process(enqueue) and dequeuer operation.
- provide IOVA for all buffers passing to that function (data buffers, digest, IV, aad).
- On dequeue provide some way to associate dequed data and digest buffers with crypto-session that was used (and probably with mbuf).
So most likely we'll end up with another just version of our current crypto-op structure.
If you'd like to get rid of mbufs dependency within current crypto-op API that understandable, but I don't think we should have same API for both sync (CPU) and async (lookaside) cases. It doesn't seem feasible at all and voids whole purpose of that patch.

Quoted 68 lines

That is why a dev-ops would be a better option.

Quoted 55 lines
Quoted 2 lines

When you would add more functionality to this sync API/struct, it will end up

being the same API/struct.

Quoted 2 lines

Let us see how close/ far we are from the existing APIs when the actual

implementation is done.

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is

supported(from feature flag) and

Quoted 1 line

Have relevant fields in xform(the newly added ones which are packed as

per

Quoted 19 lines
Quoted 19 lines

your suggestions) set,

Quoted 18 lines

It will create that type of session.

Quoted 14 lines
lib/librte_cryptodev/rte_cryptodev.h +14 -0
+ * Main points:
+ * - Current crypto-dev API is reasonably mature and it is desirable
+ *   to keep it unchanged (API/ABI stability). From other side, this
+ *   new sync API is new one and probably would require extra changes.
+ *   Having it as a new one allows to mark it as experimental, without
+ *   affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more flexibility
+ *   to the PMD writers and again allows to avoid ABI breakages in future.
+ * - process() function per set of xforms
+ *   allows to expose different process() functions for different
+ *   xform combinations. PMD writer can decide, does he wants to
+ *   push all supported algorithms into one process() function,
+ *   or spread it across several ones.
+ *   I.E. More flexibility for PMD writer.
Quoted 1 line
Quoted 1 line
Quoted 1 line

Which process function should be chosen is internal to PMD, how would

that

Quoted 2 lines
Quoted 2 lines

info

Quoted 1 line

be visible to the application or the library. These will get stored in the

session

Quoted 6 lines
Quoted 6 lines

private

Quoted 1 line

data. It would be upto the PMD writer, to store the per session process

function in

Quoted 3 lines

the session private data.

Process function would be a dev ops just like enc/deq operations and it

should

Quoted 6 lines
Quoted 6 lines

call

Quoted 1 line

The respective process API stored in the session private data.

That model (via devops) is possible, but has several drawbacks from my perspective:

Quoted 6 lines
Quoted 4 lines

1. It means we'll need to pass dev_id as a parameter to process() function. Though in fact dev_id is not a relevant information for us here (all we need is pointer to the session and pointer to the fuction to call) and I tried to avoid using it in data-path functions for that API.

You have a single vdev, but someone may have multiple vdevs for each thread,

Quoted 8 lines

or may

Quoted 1 line

Have same dev with multiple queues for each core.

That's fine. As I said above it is a SW backed implementation.
Each session has to be a separate entity that contains all necessary information (keys, alg/mode info, etc.) to process input buffers.
Plus we need the actual function pointer to call.
I just don't see what for we need a dev_id in that situation.

To iterate the session private data in the session.

Quoted 8 lines
Quoted 5 lines

Again, here we don't need care about queues and their pinning to cores. If let say someone would like to process buffers from the same IPsec SA on 2 different cores in parallel, he can just create 2 sessions for the same xform, give one to thread #1 and second to thread #2.
After that both threads are free to call process(this_thread_ses, ...) at will.

Say you have a 16core device to handle 100G of traffic on a single tunnel. Will we make 16 sessions with same parameters?

Absolutely same question we can ask for current crypto-op API.
You have lookaside crypto-dev with 16 HW queues, each queue is serviced by different CPU. For the same SA, do you need a separate session per queue, or is it ok to reuse current one? AFAIK, right now this is a grey area not clearly defined.
For crypto-devs I am aware - user can reuse the same session (as PMD uses it read-only). But again, right now I think it is not clearly defined and is implementation specific.

Quoted 45 lines
Quoted 33 lines
Quoted 7 lines
Quoted 7 lines

2. As you pointed in that case it will be just one process() function per device. So if PMD would like to have several process() functions for different type of sessions
(let say one per alg) first thing it has to do inside it's process() - read session

data

Quoted 10 lines
Quoted 10 lines

and
based on that, do a jump/call to particular internal sub-routine. Something like:
driver_id = get_pmd_driver_id();
priv_ses = ses->sess_data[driver_id];
Then either:
switch(priv_sess->alg) {case XXX: process_XXX(priv_sess, ...);break;...} OR
priv_ses->process(priv_sess, ...);

Quoted 3 lines
Quoted 3 lines

to select and call the proper function.
Looks like totally unnecessary overhead to me.
Though if we'll have ability to query/extract some sort session_ops based on

the

Quoted 4 lines
Quoted 2 lines

xform -
we can avoid this extra de-refererence+jump/call thing.

What is the issue in the priv_ses->process(); approach?

Nothing at all.
What I am saying that schema with dev_ops dev[dev_id]->dev_ops.process(ses->priv_ses[driver_id], ...)
|
|-> priv_ses->process(...)

Quoted 4 lines

Has bigger overhead then just:
process(ses,...);

So what for to introduce extra-level of indirection here?

Explained above.

Quoted 5 lines
Quoted 3 lines

I don't understand what are you saving by not doing this.
In any case you would need to identify which session correspond to which

process().

Quoted 8 lines
Quoted 3 lines

Yes, sure, but I think we can make user to store information that relationship, in a way he likes: store process() pointer for each session, or group sessions that share the same process() somehow, or...

So whatever relationship that user will make and store will make its life complicated. If we can hide that information in the driver, then what is the issue in that and user Will not need to worry. He would just call the process() and driver will choose which Process need to be called.

Driver can do that at config/init time.
Then at run-time we can avoid that choice at all and call already chosen function.

Quoted 155 lines

I think we should have a POC around this and see the difference in the cycle count. IMO it would be negligible and we would end up making a generic API set which can be used by others as well.

Quoted 150 lines
Quoted 2 lines

For that you would be doing it somewhere in your data path.

Why at data-path?
Only once at session creation/initialization time. Or might be even once per group of sessions.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I am not sure if you would need a new session init API for this as nothing

would

Quoted 11 lines
Quoted 11 lines

be visible to

Quoted 10 lines

the app or lib.

lib/librte_cryptodev/rte_cryptodev.h void * · line 1298

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+ * - Not storing process() pointer inside the session -
+ *   Allows user to choose does he want to store a process() pointer
+ *   per session, or per group of sessions for that device that share
+ *   the same input xforms. I.E. extra flexibility for the user,
+ *   plus allows us to keep cpu_sym_session totally opaque, see above.

If multiple sessions need to be processed via the same process function, PMD would save the same process in all the sessions, I don't think there

would

Quoted 86 lines
Quoted 86 lines
Quoted 1 line

be any perf overhead with that.

I think it would, see above.

Quoted 73 lines
Quoted 73 lines
lib/librte_cryptodev/rte_cryptodev.h +71 -0
+ * Sketched usage model:
+ * ....
+ * /* control path, alloc/init session */
+ * int32_t sz = rte_crypto_cpu_sym_session_size(dev_id, &xform);
+ * struct rte_crypto_cpu_sym_session *ses = user_alloc(..., sz);
+ * rte_crypto_cpu_sym_process_t process =
+ *     rte_crypto_cpu_sym_session_func(dev_id, &xform);
+ * rte_crypto_cpu_sym_session_init(dev_id, ses, &xform);
+ * ...
+ * /* data-path*/
+ * process(ses, ....);
+ * ....
+ * /* control path, termiante/free session */
+ * rte_crypto_cpu_sym_session_fini(dev_id, ses);
+ */
+
+/**
+ * vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_crypto_vec {
+       struct iovec *vec;
+       uint32_t num;
+};
+
+/*
+ * Data-path bulk process crypto function.
+ */
+typedef void (*rte_crypto_cpu_sym_process_t)(
+               struct rte_crypto_cpu_sym_session *sess,
+               struct rte_crypto_vec buf[], void *iv[], void *aad[],
+               void *digest[], int status[], uint32_t num);
+/*
+ * for given device return process function specific to input xforms
+ * on error - return NULL and set rte_errno value.
+ * Note that for same input xfroms for the same device should return
+ * the same process function.
+ */
+__rte_experimental
+rte_crypto_cpu_sym_process_t
+rte_crypto_cpu_sym_session_func(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Return required session size in bytes for given set of xforms.
+ * if xforms == NULL, then return the max possible session size,
+ * that would fit session for any supported by the device algorithm.
+ * if CPU mode is not supported at all, or requeted in xform
+ * algorithm is not supported, then return -ENOTSUP.
+ */
+__rte_experimental
+int
+rte_crypto_cpu_sym_session_size(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Initialize session.
+ * It is caller responsibility to allocate enough space for it.
+ * See rte_crypto_cpu_sym_session_size above.
+ */
+__rte_experimental
+int rte_crypto_cpu_sym_session_init(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+__rte_experimental
+void
+rte_crypto_cpu_sym_session_fini(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+
 #ifdef __cplusplus
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +0 -0
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
index defe05ea0..ed7e63fab 100644
@@ -310,6 +310,20 @@ typedef void

(*cryptodev_sym_free_session_t)(struct

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 311

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
 rte_cryptodev *dev,
 typedef void (*cryptodev_asym_free_session_t)(struct rte_cryptodev

*dev,

Quoted 37 lines
Quoted 37 lines
Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void (*cryptodev_sym_free_session_t)(struct rte_cryptodev *dev,

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +1 -0
                struct rte_cryptodev_asym_session *sess);
 
+typedef int (*cryptodev_cpu_sym_session_size_t) (struct rte_cryptodev

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 317

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef int (*cryptodev_cpu_sym_session_init_t) (struct rte_cryptodev

*dev,

Quoted 4 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 321

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +4 -0
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef void (*cryptodev_cpu_sym_session_fini_t) (struct rte_cryptodev

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 324

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       struct rte_crypto_cpu_sym_session *sess);
+
+typedef rte_crypto_cpu_sym_process_t

(*cryptodev_cpu_sym_session_func_t)

Quoted 20 lines
Quoted 20 lines

(

lib/librte_cryptodev/rte_cryptodev_pmd.h +8 -0
+                       struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
        cryptodev_configure_t dev_configure;    /**< Configure device. */
@@ -343,6 +357,11 @@ struct rte_cryptodev_ops {
        /**< Clear a Crypto sessions private data. */
        cryptodev_asym_free_session_t asym_session_clear;
        /**< Clear a Crypto sessions private data. */
+
+       cryptodev_cpu_sym_session_size_t sym_cpu_session_get_size;
+       cryptodev_cpu_sym_session_func_t sym_cpu_session_get_func;
+       cryptodev_cpu_sym_session_init_t sym_cpu_session_init;
+       cryptodev_cpu_sym_session_fini_t sym_cpu_session_fini;
 };
 
 
#23 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 349 lines

Hi Akhil,

Quoted 12 lines
Quoted 12 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto

workload

Quoted 4 lines
Quoted 4 lines

using

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

same
algorithm, key, and direction being processed by CPU

cycles

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

synchronously.

Quoted 1 line

This flexible action type does not require external

hardware

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

involvement,

Quoted 1 line
Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously,

Quoted 8 lines
Quoted 8 lines

and is

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

more

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

performant
than Cryptodev SW PMD due to the saved cycles on

removed

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

"async

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

mode

Quoted 1 line

simulation" as well as 3 cacheline access of the

crypto

Quoted 11 lines
Quoted 11 lines

ops.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Does that mean application will not call the

cryptodev_enqueue_burst

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

and

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

corresponding dequeue burst.

Yes, instead it just call

rte_security_process_cpu_crypto_bulk(...)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It would be a new API something like process_packets

and

Quoted 9 lines
Quoted 9 lines

it

Quoted 6 lines
Quoted 6 lines

will

Quoted 5 lines
Quoted 5 lines

have

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data

buffers,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

not

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

mbufs.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the

conventional

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto lib

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

only.
As far as I can understand, you are not doing any

protocol

Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

processing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

or

Quoted 3 lines
Quoted 3 lines

any

Quoted 2 lines

value add
To the crypto processing. IMO, you just need a

synchronous

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

crypto

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

processing

Quoted 2 lines

API which
Can be defined in cryptodev, you don't need to re-

create a

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

crypto

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

session

Quoted 3 lines
Quoted 3 lines

in

Quoted 2 lines

the name of
Security session in the driver just to do a synchronous

processing.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I suppose your question is why not to have
rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes

in

Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines

existing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

cryptodev

Quoted 3 lines

API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

need

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

some

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

extra

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be

Quoted 8 lines
Quoted 8 lines

something

Quoted 7 lines
Quoted 7 lines

extra

Quoted 6 lines
Quoted 6 lines

in

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

future).

Quoted 2 lines

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that

slowdown

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

current

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't

change

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

from

Quoted 8 lines
Quoted 8 lines

packet

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which

you

Quoted 9 lines
Quoted 9 lines

can

Quoted 4 lines
Quoted 4 lines

have

Quoted 3 lines
Quoted 3 lines

it as

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be

beneficial

Quoted 3 lines
Quoted 3 lines

in

Quoted 2 lines
Quoted 2 lines

case of

Quoted 1 line
Quoted 1 line
Quoted 1 line

other crypto cases as well.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

We can have cipher offset at both places(crypto-op and

cipher_xform). It

Quoted 6 lines
Quoted 6 lines

will

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types

for

Quoted 5 lines
Quoted 5 lines

CPU

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

related

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

stuff here?

I also thought of adding new xforms, but that wont serve the

purpose

Quoted 6 lines
Quoted 6 lines

for

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

may be all the cases.

Quoted 1 line
Quoted 1 line

You would be needing all information currently available in the

current

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

xforms.

Quoted 1 line
Quoted 1 line

So if you are adding new fields in the new xform, the size will be

more

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

than

Quoted 4 lines
Quoted 4 lines

that of the union of xforms.

Quoted 3 lines
Quoted 3 lines

ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done,

then

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

that

Quoted 4 lines
Quoted 4 lines

can be done for each of the

Quoted 3 lines
Quoted 1 line

Xforms and we can have a solution to this issue.

I think that we can re-use iv.offset for our purposes (for crypto

offset).

Quoted 10 lines
Quoted 10 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

So for now we can make that path work without any ABI breakage. Fan, please feel free to correct me here, if I missed something. If in future we would need to add some extra information it might require ABI breakage, though by now I don't envision anything

particular to

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

add.

Quoted 3 lines

Anyway, if there is no objection to go that way, we can try to make these changes for v2.

Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Actually, after looking at it more deeply it appears not that easy as I

thought

Quoted 4 lines
Quoted 4 lines

it

Quoted 3 lines
Quoted 3 lines

would be :)
Below is a very draft version of proposed API additions. I think it avoids ABI breakages right now and provides enough

flexibility

Quoted 3 lines
Quoted 3 lines

for

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

future extensions (if any).
For now, it doesn't address your comments about naming

conventions

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

(_CPU_

Quoted 2 lines
Quoted 2 lines

vs _SYNC_) , etc.
but I suppose is comprehensive enough to provide a main idea

beyond it.

Quoted 16 lines
Quoted 16 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Akhil and other interested parties, please try to review and provide

feedback

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

ASAP,
as related changes would take some time and we still like to hit 19.11

deadline.

Quoted 2 lines
Quoted 2 lines

Konstantin

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_crypto_sym.h rte_crypto_cipher_operation_strings[]; · line 104

Quoted 8 lines

diff --git a/lib/librte_cryptodev/rte_crypto_sym.h
b/lib/librte_cryptodev/rte_crypto_sym.h index bc8da2466..c03069e23 100644

lib/librte_cryptodev/rte_crypto_sym.h +0 -0
@@ -103,6 +103,9 @@ rte_crypto_cipher_operation_strings[];
  *
  * This structure contains data relating to Cipher (Encryption and

Decryption)

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

lib/librte_cryptodev/rte_crypto_sym.h rte_crypto_cipher_operation_strings[]; · line 106

Quoted 2 lines
lib/librte_cryptodev/rte_crypto_sym.h +1 -0
  *  use to create a session.
+ * Actually I was wrong saying that we don't have free space inside

xforms.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
lib/librte_cryptodev/rte_crypto_sym.h +1 -0
+ * Making key struct packed (see below) allow us to regain 6B that

could

Quoted 26 lines
Quoted 26 lines

be

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 12 lines
lib/librte_cryptodev/rte_crypto_sym.h +4 -1
+ * used for future extensions.
  */
 struct rte_crypto_cipher_xform {
        enum rte_crypto_cipher_operation op;
@@ -116,7 +119,25 @@ struct rte_crypto_cipher_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /**

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 134

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +10 -0
+         * offset for cipher to start within user provided data buffer.
+        * Fan suggested another (and less space consuming way) -
+         * reuse iv.offset space below, by changing:
+        * struct {uint16_t offset, length;} iv;
+        * to uunamed union:
+        * union {
+        *      struct {uint16_t offset, length;} iv;
+        *      struct {uint16_t iv_len, crypto_offset} cpu_crypto_param;
+        * };
+        * Both approaches seems ok to me in general.

No strong opinions here. OK with this one.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 137

Quoted 3 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+        * Comments/suggestions are welcome.
+         */
+       uint16_t offset;

After another thought - it is probably a bit better to have offset as a

separate

Quoted 12 lines
Quoted 12 lines
Quoted 10 lines
Quoted 2 lines

field.
In that case we can use the same xforms to create both type of sessions.

ok

Quoted 7 lines
Quoted 7 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 143

Quoted 7 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+
+       uint8_t reserved1[4];
+
        /**< Cipher key
         *
         * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation,

key.data

Quoted 1 line
Quoted 1 line

will

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_auth_xform { · line 310

Quoted 8 lines
lib/librte_cryptodev/rte_crypto_sym.h +1 -1
@@ -284,7 +305,7 @@ struct rte_crypto_auth_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
        /**< Authentication key data.
         * The authentication key length MUST be less than or equal to

the

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
lib/librte_cryptodev/rte_crypto_sym.h +8 -1
         * block size of the algorithm. It is the callers responsibility to
@@ -292,6 +313,8 @@ struct rte_crypto_auth_xform {
         * (for example RFC 2104, FIPS 198a).
         */
 
+       uint8_t reserved1[6];
+
        struct {
                uint16_t offset;
                /**< Starting point for Initialisation Vector or Counter,
@@ -376,7 +399,12 @@ struct rte_crypto_aead_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /** offset for cipher to start within data buffer */
+       uint16_t cipher_offset;
+
+       uint8_t reserved1[4];
 
        struct {
                uint16_t offset;
lib/librte_cryptodev/rte_cryptodev.h +0 -0
index e175b838c..c0c7bfed7 100644
@@ -1272,6 +1272,101 @@ void *
 rte_cryptodev_sym_session_get_user_data(
                                        struct rte_cryptodev_sym_session *sess);
 
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1279

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+/*
+ * After several thoughts decided not to try to squeeze CPU_CRYPTO
+ * into existing rte_crypto_sym_session structure/API, but instead
+ * introduce an extentsion to it via new fully opaque
+ * struct rte_crypto_cpu_sym_session and additional related API.

What all things do we need to squeeze?
In this proposal I do not see the new struct cpu_sym_session defined

here.

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

The plan is to have it totally opaque to the user, i.e. just:
struct rte_crypto_cpu_sym_session;
in public header files.

Quoted 1 line

I believe you will have same lib API/struct for cpu_sym_session and

sym_session.

I thought about such way, but there are few things that looks clumsy to

me:

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

1. Right now there is no 'type' (or so) field inside

rte_cryptodev_sym_session,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

so it is not possible to easy distinguish what session do you have:

lksd_sym or

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

cpu_sym.
In theory, there is a hole of 4B inside rte_cryptodev_sym_session, so we

can

Quoted 3 lines
Quoted 3 lines

add

Quoted 2 lines
Quoted 2 lines

some extra field
here, but in that case we wouldn't be able to use the same xform for

both

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

lksd_sym or cpu_sym
(which seems really plausible thing for me).
2. Majority of rte_cryptodev_sym_session fields I think are unnecessary

for

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

rte_crypto_cpu_sym_session:
sess_data[], opaque_data, user_data, nb_drivers.
All that consumes space, that could be used somewhere else instead.
3. I am a bit reluctant to touch existing rte_cryptodev API - to avoid any breakages I can't foresee right now.
From other side - if we'll add new functions/structs for cpu_sym_session

we

Quoted 8 lines
Quoted 8 lines

can

Quoted 2 lines
Quoted 2 lines

mark it
and keep it for some time as experimental, so further changes (if needed)

would

Quoted 4 lines
Quoted 2 lines

still be possible.

OK let us assume that you have a separate structure. But I have a few

queries:

Quoted 4 lines
Quoted 4 lines
Quoted 1 line

1. how can multiple drivers use a same session

As a short answer: they can't.
It is pretty much the same approach as with rte_security - each device needs

to

Quoted 3 lines
Quoted 3 lines

create/init its own session.
So upper layer would need to maintain its own array (or so) for such case. Though the question is why would you like to have same session over

multiple

Quoted 2 lines
Quoted 2 lines

SW backed devices?
As it would be anyway just a synchronous function call that will be executed

Quoted 8 lines

on

Quoted 4 lines
Quoted 1 line

the same cpu.

I may have single FAT tunnel which may be distributed over multiple Cores, and each core is affined to a different SW device.

If it is pure SW, then we don't need multiple devices for such scenario. Device in that case is pure abstraction that we can skip.

Yes agreed, but that liberty is given to the application whether it need multiple devices with single queue or a single device with multiple queues.
I think that independence should not be broken in this new API.

Quoted 7 lines
Quoted 5 lines

So a single session may be accessed by multiple devices.

One more example would be depending on packet sizes, I may switch between HW/SW PMDs with the same session.

Sure, but then we'll have multiple sessions.

No, the session will be same and it will have multiple private data for each of the PMD.

Quoted 5 lines

BTW, we have same thing now - these private session pointers are just stored inside the same rte_crypto_sym_session.
And if user wants to support this model, he would also need to store <dev_id, queue_id>
pair for each HW device anyway.

Yes agreed, but how is that thing happening in your new struct, you cannot support that.

Quoted 35 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

2. Can somebody use the scheduler pmd for scheduling the different type

of

Quoted 19 lines
Quoted 10 lines

payloads for the same session?

In theory yes.
Though for that scheduler pmd should have inside it's rte_crypto_cpu_sym_session an array of pointers to the underlying devices sessions.

Quoted 2 lines

With your proposal the APIs would be very specific to your use case only.

Quoted 8 lines

Yes in some way.
I consider that API specific for SW backed crypto PMDs.
I can hardly see how any 'real HW' PMDs (lksd-none, lksd-proto) will benefit from it.
Current crypto-op API is very much HW oriented.
Which is ok, that's for it was intended for, but I think we also need one that would be designed
for SW backed implementation in mind.

Quoted 1 line

We may re-use your API for HW PMDs as well which do not have requirement

of

Quoted 2 lines

Crypto-op/mbuf etc.
The return type of your new process API may have a status which say

'processed'

Quoted 1 line

Or can be say 'enqueued'. So if it is 'enqueued', we may have a new API for

raw

Quoted 3 lines

Bufs dequeue as well.

This requirement can be for any hardware PMDs like QAT as well.

Quoted 16 lines

I don't think it is a good idea to extend this API for async (lookaside) devices. You'll need to:
- provide dev_id and queue_id for each process(enqueue) and dequeuer
operation.
- provide IOVA for all buffers passing to that function (data buffers, digest, IV,
aad).
- On dequeue provide some way to associate dequed data and digest buffers
with
crypto-session that was used (and probably with mbuf).
So most likely we'll end up with another just version of our current crypto-op
structure.
If you'd like to get rid of mbufs dependency within current crypto-op API that understandable,
but I don't think we should have same API for both sync (CPU) and async (lookaside) cases.
It doesn't seem feasible at all and voids whole purpose of that patch.

At this moment we are not much concerned about the dequeue API and about the HW PMD support. It is just that the new API should be generic enough to be used in some future scenarios as well. I am just highlighting the possible usecases which can be there in future.

What is the issue that you face in making a dev-op for this new API. Do you see any performance impact with that?

Quoted 85 lines
Quoted 5 lines

That is why a dev-ops would be a better option.

Quoted 2 lines
Quoted 2 lines

When you would add more functionality to this sync API/struct, it will end

up

Quoted 9 lines
Quoted 9 lines

being the same API/struct.

Quoted 2 lines

Let us see how close/ far we are from the existing APIs when the actual

implementation is done.

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is

supported(from feature flag) and

Quoted 1 line

Have relevant fields in xform(the newly added ones which are packed

as

Quoted 9 lines
Quoted 9 lines

per

Quoted 8 lines
Quoted 8 lines

your suggestions) set,

Quoted 7 lines

It will create that type of session.

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ * Main points:
+ * - Current crypto-dev API is reasonably mature and it is desirable
+ *   to keep it unchanged (API/ABI stability). From other side, this
+ *   new sync API is new one and probably would require extra

changes.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1287

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ *   Having it as a new one allows to mark it as experimental, without
+ *   affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more flexibility
+ *   to the PMD writers and again allows to avoid ABI breakages in

future.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1293

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev.h +6 -0
+ * - process() function per set of xforms
+ *   allows to expose different process() functions for different
+ *   xform combinations. PMD writer can decide, does he wants to
+ *   push all supported algorithms into one process() function,
+ *   or spread it across several ones.
+ *   I.E. More flexibility for PMD writer.

Which process function should be chosen is internal to PMD, how

would

Quoted 17 lines
Quoted 17 lines

that

Quoted 2 lines
Quoted 2 lines

info

Quoted 1 line

be visible to the application or the library. These will get stored in the

session

Quoted 6 lines
Quoted 6 lines

private

Quoted 1 line

data. It would be upto the PMD writer, to store the per session process

function in

Quoted 3 lines

the session private data.

Process function would be a dev ops just like enc/deq operations and it

should

Quoted 6 lines
Quoted 6 lines

call

Quoted 1 line

The respective process API stored in the session private data.

That model (via devops) is possible, but has several drawbacks from my perspective:

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

1. It means we'll need to pass dev_id as a parameter to process() function. Though in fact dev_id is not a relevant information for us here (all we need is pointer to the session and pointer to the fuction to call) and I tried to avoid using it in data-path functions for that API.

You have a single vdev, but someone may have multiple vdevs for each

thread,

Quoted 5 lines
Quoted 5 lines

or may

Quoted 1 line

Have same dev with multiple queues for each core.

That's fine. As I said above it is a SW backed implementation. Each session has to be a separate entity that contains all necessary

information

Quoted 6 lines
Quoted 3 lines

(keys, alg/mode info, etc.) to process input buffers. Plus we need the actual function pointer to call.
I just don't see what for we need a dev_id in that situation.

To iterate the session private data in the session.

Quoted 8 lines
Quoted 5 lines

Again, here we don't need care about queues and their pinning to cores. If let say someone would like to process buffers from the same IPsec SA on 2 different cores in parallel, he can just create 2 sessions for the same xform, give one to thread #1 and second to thread #2.
After that both threads are free to call process(this_thread_ses, ...) at will.

Say you have a 16core device to handle 100G of traffic on a single tunnel. Will we make 16 sessions with same parameters?

Quoted 10 lines

Absolutely same question we can ask for current crypto-op API.
You have lookaside crypto-dev with 16 HW queues, each queue is serviced by different CPU.
For the same SA, do you need a separate session per queue, or is it ok to reuse current one?
AFAIK, right now this is a grey area not clearly defined.
For crypto-devs I am aware - user can reuse the same session (as PMD uses it read-only).
But again, right now I think it is not clearly defined and is implementation specific.

User can use the same session, that is what I am also insisting, but it may have separate Session private data. Cryptodev session create API provide that functionality and we can Leverage that.

BTW, I can see a v2 to this RFC which is still based on security library. When do you plan To submit the patches for crypto based APIs. We have RC1 merge deadline for this patchset on 21st Oct.

As per my understanding you only need a new dev-op for sync support. Session APIs Will remain the same and you will have some extra fields packed in xform structs.

The PMD will need to maintain a pointer to the per session process function while creating Session and will be used by the dev-op API at runtime without any extra check at runtime.

Quoted 229 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

2. As you pointed in that case it will be just one process() function per

device.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

So if PMD would like to have several process() functions for different type

of

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

sessions
(let say one per alg) first thing it has to do inside it's process() - read

session

Quoted 11 lines
Quoted 11 lines

data

Quoted 10 lines
Quoted 10 lines

and
based on that, do a jump/call to particular internal sub-routine. Something like:
driver_id = get_pmd_driver_id();
priv_ses = ses->sess_data[driver_id];
Then either:
switch(priv_sess->alg) {case XXX: process_XXX(priv_sess, ...);break;...} OR
priv_ses->process(priv_sess, ...);

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

to select and call the proper function.
Looks like totally unnecessary overhead to me.
Though if we'll have ability to query/extract some sort session_ops based

on

Quoted 22 lines
Quoted 12 lines

the

Quoted 4 lines
Quoted 2 lines

xform -
we can avoid this extra de-refererence+jump/call thing.

What is the issue in the priv_ses->process(); approach?

Nothing at all.
What I am saying that schema with dev_ops dev[dev_id]->dev_ops.process(ses->priv_ses[driver_id], ...)
|
|-> priv_ses->process(...)

Quoted 4 lines

Has bigger overhead then just:
process(ses,...);

So what for to introduce extra-level of indirection here?

Explained above.

Quoted 3 lines
Quoted 3 lines

I don't understand what are you saving by not doing this.
In any case you would need to identify which session correspond to which

Quoted 7 lines
Quoted 5 lines

process().

Yes, sure, but I think we can make user to store information that relationship, in a way he likes: store process() pointer for each session, or group sessions that share the same process() somehow, or...

So whatever relationship that user will make and store will make its life

complicated.

Quoted 1 line

If we can hide that information in the driver, then what is the issue in that and

user

Quoted 1 line

Will not need to worry. He would just call the process() and driver will choose

which

Quoted 1 line

Process need to be called.

Driver can do that at config/init time.
Then at run-time we can avoid that choice at all and call already chosen function.

Quoted 2 lines

I think we should have a POC around this and see the difference in the cycle

count.

Quoted 1 line

IMO it would be negligible and we would end up making a generic API set

which

Quoted 13 lines

can be used by others as well.

Quoted 11 lines
Quoted 2 lines

For that you would be doing it somewhere in your data path.

Why at data-path?
Only once at session creation/initialization time. Or might be even once per group of sessions.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I am not sure if you would need a new session init API for this as

nothing

Quoted 9 lines
Quoted 9 lines

would

Quoted 8 lines
Quoted 8 lines

be visible to

Quoted 7 lines

the app or lib.

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+ * - Not storing process() pointer inside the session -
+ *   Allows user to choose does he want to store a process() pointer
+ *   per session, or per group of sessions for that device that share
+ *   the same input xforms. I.E. extra flexibility for the user,
+ *   plus allows us to keep cpu_sym_session totally opaque, see

above.

Quoted 37 lines
Quoted 37 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

If multiple sessions need to be processed via the same process function, PMD would save the same process in all the sessions, I don't think there

would

Quoted 33 lines
Quoted 33 lines
Quoted 1 line

be any perf overhead with that.

I think it would, see above.

Quoted 29 lines
Quoted 29 lines
lib/librte_cryptodev/rte_cryptodev.h +28 -0
+ * Sketched usage model:
+ * ....
+ * /* control path, alloc/init session */
+ * int32_t sz = rte_crypto_cpu_sym_session_size(dev_id, &xform);
+ * struct rte_crypto_cpu_sym_session *ses = user_alloc(..., sz);
+ * rte_crypto_cpu_sym_process_t process =
+ *     rte_crypto_cpu_sym_session_func(dev_id, &xform);
+ * rte_crypto_cpu_sym_session_init(dev_id, ses, &xform);
+ * ...
+ * /* data-path*/
+ * process(ses, ....);
+ * ....
+ * /* control path, termiante/free session */
+ * rte_crypto_cpu_sym_session_fini(dev_id, ses);
+ */
+
+/**
+ * vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_crypto_vec {
+       struct iovec *vec;
+       uint32_t num;
+};
+
+/*
+ * Data-path bulk process crypto function.
+ */
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1334

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +8 -0
+typedef void (*rte_crypto_cpu_sym_process_t)(
+               struct rte_crypto_cpu_sym_session *sess,
+               struct rte_crypto_vec buf[], void *iv[], void *aad[],
+               void *digest[], int status[], uint32_t num);
+/*
+ * for given device return process function specific to input xforms
+ * on error - return NULL and set rte_errno value.
+ * Note that for same input xfroms for the same device should

return

Quoted 45 lines
Quoted 45 lines
Quoted 45 lines
Quoted 45 lines
Quoted 36 lines
Quoted 36 lines
lib/librte_cryptodev/rte_cryptodev.h +35 -0
+ * the same process function.
+ */
+__rte_experimental
+rte_crypto_cpu_sym_process_t
+rte_crypto_cpu_sym_session_func(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Return required session size in bytes for given set of xforms.
+ * if xforms == NULL, then return the max possible session size,
+ * that would fit session for any supported by the device algorithm.
+ * if CPU mode is not supported at all, or requeted in xform
+ * algorithm is not supported, then return -ENOTSUP.
+ */
+__rte_experimental
+int
+rte_crypto_cpu_sym_session_size(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Initialize session.
+ * It is caller responsibility to allocate enough space for it.
+ * See rte_crypto_cpu_sym_session_size above.
+ */
+__rte_experimental
+int rte_crypto_cpu_sym_session_init(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+__rte_experimental
+void
+rte_crypto_cpu_sym_session_fini(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+
 #ifdef __cplusplus
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +0 -0
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
index defe05ea0..ed7e63fab 100644
@@ -310,6 +310,20 @@ typedef void

(*cryptodev_sym_free_session_t)(struct

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 311

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
 rte_cryptodev *dev,
 typedef void (*cryptodev_asym_free_session_t)(struct rte_cryptodev

*dev,

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +1 -0
                struct rte_cryptodev_asym_session *sess);
 
+typedef int (*cryptodev_cpu_sym_session_size_t) (struct

rte_cryptodev

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 317

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef int (*cryptodev_cpu_sym_session_init_t) (struct

rte_cryptodev

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

*dev,

Quoted 4 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 321

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +4 -0
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef void (*cryptodev_cpu_sym_session_fini_t) (struct

rte_cryptodev

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 324

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       struct rte_crypto_cpu_sym_session *sess);
+
+typedef rte_crypto_cpu_sym_process_t

(*cryptodev_cpu_sym_session_func_t)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h

Quoted 7 lines

(

lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
        cryptodev_configure_t dev_configure;    /**< Configure device.

*/

Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +5 -0
@@ -343,6 +357,11 @@ struct rte_cryptodev_ops {
        /**< Clear a Crypto sessions private data. */
        cryptodev_asym_free_session_t asym_session_clear;
        /**< Clear a Crypto sessions private data. */
+
+       cryptodev_cpu_sym_session_size_t sym_cpu_session_get_size;
+       cryptodev_cpu_sym_session_func_t sym_cpu_session_get_func;
+       cryptodev_cpu_sym_session_init_t sym_cpu_session_init;
+       cryptodev_cpu_sym_session_fini_t sym_cpu_session_fini;
 };
 
 
#24 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Quoted 367 lines
Quoted 345 lines
Quoted 12 lines
Quoted 12 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This action type allows the burst of symmetric crypto

workload

Quoted 4 lines
Quoted 4 lines

using

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

the

Quoted 2 lines
Quoted 2 lines

same
algorithm, key, and direction being processed by CPU

cycles

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

synchronously.
This flexible action type does not require external

hardware

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

involvement,

Quoted 1 line
Quoted 1 line

having the crypto workload processed synchronously,

Quoted 8 lines
Quoted 8 lines

and is

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

more

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

performant
than Cryptodev SW PMD due to the saved cycles on

removed

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

"async

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

mode
simulation" as well as 3 cacheline access of the

crypto

Quoted 11 lines
Quoted 11 lines

ops.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Does that mean application will not call the

cryptodev_enqueue_burst

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

and

Quoted 3 lines
Quoted 3 lines

corresponding dequeue burst.

Yes, instead it just call

rte_security_process_cpu_crypto_bulk(...)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It would be a new API something like process_packets

and

Quoted 9 lines
Quoted 9 lines

it

Quoted 6 lines
Quoted 6 lines

will

Quoted 5 lines
Quoted 5 lines

have

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

the

Quoted 3 lines
Quoted 3 lines

crypto processed packets while returning from the API?

Yes, though the plan is that API will operate on raw data

buffers,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

not

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

mbufs.

Quoted 3 lines
Quoted 3 lines

I still do not understand why we cannot do with the

conventional

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto lib

Quoted 2 lines
Quoted 2 lines

only.
As far as I can understand, you are not doing any

protocol

Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

processing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

or

Quoted 3 lines
Quoted 3 lines

any
value add
To the crypto processing. IMO, you just need a

synchronous

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

crypto

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

processing
API which
Can be defined in cryptodev, you don't need to re-

create a

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

crypto

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

session

Quoted 3 lines
Quoted 3 lines

in
the name of
Security session in the driver just to do a synchronous

processing.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I suppose your question is why not to have
rte_crypot_process_cpu_crypto_bulk(...) instead?
The main reason is that would require disruptive changes

in

Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines

existing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

cryptodev
API
(would cause ABI/API breakage).
Session for RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO

need

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

some

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

extra

Quoted 3 lines
Quoted 3 lines

information
that normal crypto_sym_xform doesn't contain
(cipher offset from the start of the buffer, might be

Quoted 8 lines
Quoted 8 lines

something

Quoted 7 lines
Quoted 7 lines

extra

Quoted 6 lines
Quoted 6 lines

in

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

future).

Cipher offset will be part of rte_crypto_op.

fill/read (+ alloc/free) is one of the main things that

slowdown

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

current

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crypto-op

Quoted 2 lines
Quoted 2 lines

approach.
That's why the general idea - have all data that wouldn't

change

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

from

Quoted 8 lines
Quoted 8 lines

packet

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

to

Quoted 6 lines
Quoted 2 lines

packet
included into the session and setup it once at session_init().

I agree that you cannot use crypto-op.
You can have the new API in crypto.
As per the current patch, you only need cipher_offset which

you

Quoted 9 lines
Quoted 9 lines

can

Quoted 4 lines
Quoted 4 lines

have

Quoted 3 lines
Quoted 3 lines

it as

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

a parameter until

Quoted 1 line

You get it approved in the crypto xform. I believe it will be

beneficial

Quoted 3 lines
Quoted 3 lines

in

Quoted 2 lines
Quoted 2 lines

case of

Quoted 1 line
Quoted 1 line
Quoted 1 line

other crypto cases as well.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

We can have cipher offset at both places(crypto-op and

cipher_xform). It

Quoted 6 lines
Quoted 6 lines

will

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

give flexibility to the user to

Quoted 1 line

override it.

After having another thought on your proposal:
Probably we can introduce new rte_crypto_sym_xform_types

for

Quoted 5 lines
Quoted 5 lines

CPU

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

related

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

stuff here?

I also thought of adding new xforms, but that wont serve the

purpose

Quoted 6 lines
Quoted 6 lines

for

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

may be all the cases.

Quoted 1 line
Quoted 1 line

You would be needing all information currently available in the

current

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

xforms.

Quoted 1 line
Quoted 1 line

So if you are adding new fields in the new xform, the size will be

more

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

than

Quoted 4 lines
Quoted 4 lines

that of the union of xforms.

Quoted 3 lines
Quoted 3 lines

ABI breakage would still be there.

If you think a valid compression of the AEAD xform can be done,

then

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

that

Quoted 4 lines
Quoted 4 lines

can be done for each of the

Quoted 3 lines
Quoted 1 line

Xforms and we can have a solution to this issue.

I think that we can re-use iv.offset for our purposes (for crypto

offset).

Quoted 10 lines
Quoted 10 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

So for now we can make that path work without any ABI breakage. Fan, please feel free to correct me here, if I missed something. If in future we would need to add some extra information it might require ABI breakage, though by now I don't envision anything

particular to

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

add.

Quoted 3 lines

Anyway, if there is no objection to go that way, we can try to make these changes for v2.

Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Actually, after looking at it more deeply it appears not that easy as I

thought

Quoted 4 lines
Quoted 4 lines

it

Quoted 3 lines
Quoted 3 lines

would be :)
Below is a very draft version of proposed API additions. I think it avoids ABI breakages right now and provides enough

flexibility

Quoted 3 lines
Quoted 3 lines

for

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

future extensions (if any).
For now, it doesn't address your comments about naming

conventions

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

(_CPU_

Quoted 2 lines
Quoted 2 lines

vs _SYNC_) , etc.
but I suppose is comprehensive enough to provide a main idea

beyond it.

Quoted 16 lines
Quoted 16 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Akhil and other interested parties, please try to review and provide

feedback

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

ASAP,
as related changes would take some time and we still like to hit 19.11

deadline.

Quoted 2 lines
Quoted 2 lines

Konstantin

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_crypto_sym.h rte_crypto_cipher_operation_strings[]; · line 104

Quoted 8 lines

diff --git a/lib/librte_cryptodev/rte_crypto_sym.h
b/lib/librte_cryptodev/rte_crypto_sym.h index bc8da2466..c03069e23 100644

lib/librte_cryptodev/rte_crypto_sym.h +0 -0
@@ -103,6 +103,9 @@ rte_crypto_cipher_operation_strings[];
  *
  * This structure contains data relating to Cipher (Encryption and

Decryption)

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

lib/librte_cryptodev/rte_crypto_sym.h rte_crypto_cipher_operation_strings[]; · line 106

Quoted 2 lines
lib/librte_cryptodev/rte_crypto_sym.h +1 -0
  *  use to create a session.
+ * Actually I was wrong saying that we don't have free space inside

xforms.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
lib/librte_cryptodev/rte_crypto_sym.h +1 -0
+ * Making key struct packed (see below) allow us to regain 6B that

could

Quoted 26 lines
Quoted 26 lines

be

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 12 lines
lib/librte_cryptodev/rte_crypto_sym.h +4 -1
+ * used for future extensions.
  */
 struct rte_crypto_cipher_xform {
        enum rte_crypto_cipher_operation op;
@@ -116,7 +119,25 @@ struct rte_crypto_cipher_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /**

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 134

Quoted 10 lines
lib/librte_cryptodev/rte_crypto_sym.h +10 -0
+         * offset for cipher to start within user provided data buffer.
+        * Fan suggested another (and less space consuming way) -
+         * reuse iv.offset space below, by changing:
+        * struct {uint16_t offset, length;} iv;
+        * to uunamed union:
+        * union {
+        *      struct {uint16_t offset, length;} iv;
+        *      struct {uint16_t iv_len, crypto_offset} cpu_crypto_param;
+        * };
+        * Both approaches seems ok to me in general.

No strong opinions here. OK with this one.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 137

Quoted 3 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+        * Comments/suggestions are welcome.
+         */
+       uint16_t offset;

After another thought - it is probably a bit better to have offset as a

separate

Quoted 12 lines
Quoted 12 lines
Quoted 10 lines
Quoted 2 lines

field.
In that case we can use the same xforms to create both type of sessions.

ok

Quoted 7 lines
Quoted 7 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_cipher_xform { · line 143

Quoted 7 lines
lib/librte_cryptodev/rte_crypto_sym.h +3 -0
+
+       uint8_t reserved1[4];
+
        /**< Cipher key
         *
         * For the RTE_CRYPTO_CIPHER_AES_F8 mode of operation,

key.data

Quoted 1 line
Quoted 1 line

will

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_crypto_sym.h struct rte_crypto_auth_xform { · line 310

Quoted 8 lines
lib/librte_cryptodev/rte_crypto_sym.h +1 -1
@@ -284,7 +305,7 @@ struct rte_crypto_auth_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
        /**< Authentication key data.
         * The authentication key length MUST be less than or equal to

the

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
lib/librte_cryptodev/rte_crypto_sym.h +8 -1
         * block size of the algorithm. It is the callers responsibility to
@@ -292,6 +313,8 @@ struct rte_crypto_auth_xform {
         * (for example RFC 2104, FIPS 198a).
         */
 
+       uint8_t reserved1[6];
+
        struct {
                uint16_t offset;
                /**< Starting point for Initialisation Vector or Counter,
@@ -376,7 +399,12 @@ struct rte_crypto_aead_xform {
        struct {
                const uint8_t *data;    /**< pointer to key data */
                uint16_t length;        /**< key length in bytes */
-       } key;
+       } __attribute__((__packed__)) key;
+
+       /** offset for cipher to start within data buffer */
+       uint16_t cipher_offset;
+
+       uint8_t reserved1[4];
 
        struct {
                uint16_t offset;
lib/librte_cryptodev/rte_cryptodev.h +0 -0
index e175b838c..c0c7bfed7 100644
@@ -1272,6 +1272,101 @@ void *
 rte_cryptodev_sym_session_get_user_data(
                                        struct rte_cryptodev_sym_session *sess);
 
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1279

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+/*
+ * After several thoughts decided not to try to squeeze CPU_CRYPTO
+ * into existing rte_crypto_sym_session structure/API, but instead
+ * introduce an extentsion to it via new fully opaque
+ * struct rte_crypto_cpu_sym_session and additional related API.

What all things do we need to squeeze?
In this proposal I do not see the new struct cpu_sym_session defined

here.

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

The plan is to have it totally opaque to the user, i.e. just:
struct rte_crypto_cpu_sym_session;
in public header files.

Quoted 1 line

I believe you will have same lib API/struct for cpu_sym_session and

sym_session.

I thought about such way, but there are few things that looks clumsy to

me:

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

1. Right now there is no 'type' (or so) field inside

rte_cryptodev_sym_session,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

so it is not possible to easy distinguish what session do you have:

lksd_sym or

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

cpu_sym.
In theory, there is a hole of 4B inside rte_cryptodev_sym_session, so we

can

Quoted 3 lines
Quoted 3 lines

add

Quoted 2 lines
Quoted 2 lines

some extra field
here, but in that case we wouldn't be able to use the same xform for

both

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

lksd_sym or cpu_sym
(which seems really plausible thing for me).
2. Majority of rte_cryptodev_sym_session fields I think are unnecessary

for

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

rte_crypto_cpu_sym_session:
sess_data[], opaque_data, user_data, nb_drivers.
All that consumes space, that could be used somewhere else instead.
3. I am a bit reluctant to touch existing rte_cryptodev API - to avoid any breakages I can't foresee right now.
From other side - if we'll add new functions/structs for cpu_sym_session

we

Quoted 8 lines
Quoted 8 lines

can

Quoted 2 lines
Quoted 2 lines

mark it
and keep it for some time as experimental, so further changes (if needed)

would

Quoted 4 lines
Quoted 2 lines

still be possible.

OK let us assume that you have a separate structure. But I have a few

queries:

Quoted 4 lines
Quoted 4 lines
Quoted 1 line

1. how can multiple drivers use a same session

As a short answer: they can't.
It is pretty much the same approach as with rte_security - each device needs

to

Quoted 3 lines
Quoted 3 lines

create/init its own session.
So upper layer would need to maintain its own array (or so) for such case. Though the question is why would you like to have same session over

multiple

Quoted 2 lines
Quoted 2 lines

SW backed devices?
As it would be anyway just a synchronous function call that will be executed

Quoted 8 lines

on

Quoted 4 lines
Quoted 1 line

the same cpu.

I may have single FAT tunnel which may be distributed over multiple Cores, and each core is affined to a different SW device.

If it is pure SW, then we don't need multiple devices for such scenario. Device in that case is pure abstraction that we can skip.

Yes agreed, but that liberty is given to the application whether it need multiple devices with single queue or a single device with multiple queues.
I think that independence should not be broken in this new API.

Quoted 7 lines
Quoted 5 lines

So a single session may be accessed by multiple devices.

One more example would be depending on packet sizes, I may switch between HW/SW PMDs with the same session.

Sure, but then we'll have multiple sessions.

No, the session will be same and it will have multiple private data for each of the PMD.

Quoted 7 lines
Quoted 5 lines

BTW, we have same thing now - these private session pointers are just stored inside the same rte_crypto_sym_session.
And if user wants to support this model, he would also need to store <dev_id, queue_id>
pair for each HW device anyway.

Yes agreed, but how is that thing happening in your new struct, you cannot support that.

User can store all these info in his own struct.
That's exactly what we have right now.
Let say ipsec-secgw has to store for each IPsec SA:
pointer to crypto-session and/or pointer to security session plus (for lookaside-devices) cdev_id_qp that allows it to extract dev_id + queue_id information.
As I understand that works for now, as each ipsec_sa uses only one dev+queue. Though if someone would like to use multiple devices/queues for the same SA - he would need to have an array of these <dev+queue> pairs. So even right now rte_cryptodev_sym_session is not self-consistent and requires extra information to be maintained by user.

Quoted 53 lines
Quoted 36 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

2. Can somebody use the scheduler pmd for scheduling the different type

of

Quoted 19 lines
Quoted 10 lines

payloads for the same session?

In theory yes.
Though for that scheduler pmd should have inside it's rte_crypto_cpu_sym_session an array of pointers to the underlying devices sessions.

Quoted 2 lines

With your proposal the APIs would be very specific to your use case only.

Quoted 8 lines

Yes in some way.
I consider that API specific for SW backed crypto PMDs.
I can hardly see how any 'real HW' PMDs (lksd-none, lksd-proto) will benefit from it.
Current crypto-op API is very much HW oriented.
Which is ok, that's for it was intended for, but I think we also need one that would be designed
for SW backed implementation in mind.

Quoted 1 line

We may re-use your API for HW PMDs as well which do not have requirement

of

Quoted 2 lines

Crypto-op/mbuf etc.
The return type of your new process API may have a status which say

'processed'

Quoted 1 line

Or can be say 'enqueued'. So if it is 'enqueued', we may have a new API for

raw

Quoted 3 lines

Bufs dequeue as well.

This requirement can be for any hardware PMDs like QAT as well.

Quoted 16 lines

I don't think it is a good idea to extend this API for async (lookaside) devices. You'll need to:
- provide dev_id and queue_id for each process(enqueue) and dequeuer
operation.
- provide IOVA for all buffers passing to that function (data buffers, digest, IV,
aad).
- On dequeue provide some way to associate dequed data and digest buffers
with
crypto-session that was used (and probably with mbuf).
So most likely we'll end up with another just version of our current crypto-op
structure.
If you'd like to get rid of mbufs dependency within current crypto-op API that understandable,
but I don't think we should have same API for both sync (CPU) and async (lookaside) cases.
It doesn't seem feasible at all and voids whole purpose of that patch.

Quoted 4 lines

At this moment we are not much concerned about the dequeue API and about the HW PMD support. It is just that the new API should be generic enough to be used in some future scenarios as well. I am just highlighting the possible usecases which can be there in future.

Sorry, but I strongly disagree with such approach.
We should stop adding/modifying API 'just in case' and because 'it might be useful for some future HW'. Inside DPDK we already do have too many dev level APIs without any implementations. That's quite bad practice and very dis-orienting for end-users.
I think to justify API additions/changes we need at least one proper implementation for it, or at least some strong evidence that people are really committed to support it in nearest future. BTW, that what TB agreed on, nearly a year ago.

This new API (if we'll go ahead with it of course) would stay experimental for some time anyway to make sure we don't miss anything needed (I think for about a year time-frame). So if you guys *really* want to extend it support _async_ devices too -
I am open for modifications/additions here.
Though personally I think such addition would over-complicate things and we'll end up with another reincarnation of current crypto-op.
We actually discussed it internally, and decided to drop that idea because of that. Again, my opinion - for lookaside devices it might be better to try to optimize current crypto-op path (remove mbuf requirement, probably add ability to
group by session on enqueue/dequeue, etc.).

Quoted 3 lines

What is the issue that you face in making a dev-op for this new API. Do you see any performance impact with that?

There are two main things:
1. user would need to maintain and provide for each process() call dev_id+queue_id. That's means extra (and totally unnecessary for SW) overhead.
2. yes I would expect some perf overhead too - it would be extra call or branch. Again as it would be data-dependency - most likely cpu wouldn't be able to pipeline it efficiently:

rte_crypto_sym_process(uint8_t dev_id, uint16 qp_id, rte_crypto_sym_session *ses, ...)

{
     struct rte_cryptodev *dev = &rte_cryptodevs[dev_id];
     return (*dev->process)(sess->data[dev->driver_id, ...);
}

driver_specific_process(driver_specific_sym_session *sess) {
return sess->process(sess, ...) ;
}

I didn't make any exact measurements but sure it would be slower than just:
session_udata->process(session->udata->sess, ...);
Again it would be much more noticeable on low end cpus.
Let say here: http://mails.dpdk.org/archives/dev/2019-September/144350.html
Jerin claims 1.5-3% drop for introducing extra call via hiding eth_dev contents - I suppose we would have something similar here.
I do realize that in majority of cases crypto is more expensive then RX/TX, but still.

If it would be a really unavoidable tradeoff (support already existing API, or so) I wouldn't mind, but I don't see any real need for it right now.

Quoted 97 lines
Quoted 86 lines
Quoted 6 lines

That is why a dev-ops would be a better option.

Quoted 2 lines
Quoted 2 lines

When you would add more functionality to this sync API/struct, it will end

up

Quoted 9 lines
Quoted 9 lines

being the same API/struct.

Quoted 2 lines

Let us see how close/ far we are from the existing APIs when the actual

implementation is done.

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is

supported(from feature flag) and

Quoted 1 line

Have relevant fields in xform(the newly added ones which are packed

as

Quoted 9 lines
Quoted 9 lines

per

Quoted 8 lines
Quoted 8 lines

your suggestions) set,

Quoted 7 lines

It will create that type of session.

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ * Main points:
+ * - Current crypto-dev API is reasonably mature and it is desirable
+ *   to keep it unchanged (API/ABI stability). From other side, this
+ *   new sync API is new one and probably would require extra

changes.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1287

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev.h +4 -0
+ *   Having it as a new one allows to mark it as experimental, without
+ *   affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more flexibility
+ *   to the PMD writers and again allows to avoid ABI breakages in

future.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1293

Quoted 6 lines
lib/librte_cryptodev/rte_cryptodev.h +6 -0
+ * - process() function per set of xforms
+ *   allows to expose different process() functions for different
+ *   xform combinations. PMD writer can decide, does he wants to
+ *   push all supported algorithms into one process() function,
+ *   or spread it across several ones.
+ *   I.E. More flexibility for PMD writer.

Which process function should be chosen is internal to PMD, how

would

Quoted 17 lines
Quoted 17 lines

that

Quoted 2 lines
Quoted 2 lines

info

Quoted 1 line

be visible to the application or the library. These will get stored in the

session

Quoted 6 lines
Quoted 6 lines

private

Quoted 1 line

data. It would be upto the PMD writer, to store the per session process

function in

Quoted 3 lines

the session private data.

Process function would be a dev ops just like enc/deq operations and it

should

Quoted 6 lines
Quoted 6 lines

call

Quoted 1 line

The respective process API stored in the session private data.

That model (via devops) is possible, but has several drawbacks from my perspective:

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

1. It means we'll need to pass dev_id as a parameter to process() function. Though in fact dev_id is not a relevant information for us here (all we need is pointer to the session and pointer to the fuction to call) and I tried to avoid using it in data-path functions for that API.

You have a single vdev, but someone may have multiple vdevs for each

thread,

Quoted 5 lines
Quoted 5 lines

or may

Quoted 1 line

Have same dev with multiple queues for each core.

That's fine. As I said above it is a SW backed implementation. Each session has to be a separate entity that contains all necessary

information

Quoted 6 lines
Quoted 3 lines

(keys, alg/mode info, etc.) to process input buffers. Plus we need the actual function pointer to call.
I just don't see what for we need a dev_id in that situation.

To iterate the session private data in the session.

Quoted 8 lines
Quoted 5 lines

Again, here we don't need care about queues and their pinning to cores. If let say someone would like to process buffers from the same IPsec SA on 2 different cores in parallel, he can just create 2 sessions for the same xform, give one to thread #1 and second to thread #2.
After that both threads are free to call process(this_thread_ses, ...) at will.

Say you have a 16core device to handle 100G of traffic on a single tunnel. Will we make 16 sessions with same parameters?

Quoted 10 lines

Absolutely same question we can ask for current crypto-op API.
You have lookaside crypto-dev with 16 HW queues, each queue is serviced by different CPU.
For the same SA, do you need a separate session per queue, or is it ok to reuse current one?
AFAIK, right now this is a grey area not clearly defined.
For crypto-devs I am aware - user can reuse the same session (as PMD uses it read-only).
But again, right now I think it is not clearly defined and is implementation specific.

Quoted 3 lines

User can use the same session, that is what I am also insisting, but it may have separate Session private data. Cryptodev session create API provide that functionality and we can Leverage that.

rte_cryptodev_sym_session. sess_data[] is indexed by driver_id, which means we can't use the same rte_cryptodev_sym_session to hold sessions for both sync and async mode for the same device. Off course we can add a hard requirement that any driver that wants to support process() has to create sessions that can handle both process and enqueue/dequeue, but then again what for to create such overhead?

BTW, to be honest, I don't consider current rte_cryptodev_sym_session construct for multiple device_ids:

__extension__ struct {
                void *data;
                uint16_t refcnt;
        } sess_data[0];
        /**< Driver specific session material, variable size */

as an advantage.
It looks too error prone for me:
1. Simultaneous session initialization/de-initialization for devices with the same driver_id is not possible.
2. It assumes that all device driver will be loaded before we start to create session pools.

Right now it seems ok, as no-one requires such functionality, but I don't know how it will be in future. For me rte_security session model, where for each security context user have to create new session looks much more robust.

Quoted 2 lines

BTW, I can see a v2 to this RFC which is still based on security library.

Yes, v2 was concentrated on fixing found issues, some code restructuring, i.e. - changes that would be needed anyway whatever API aproach we'll choose.

Quoted 3 lines

When do you plan
To submit the patches for crypto based APIs. We have RC1 merge deadline for this patchset on 21st Oct.

We'd like to start working on it ASAP, but it seems we still have a major disagreement about how this crypto-dev API should look like.
Which makes me think - should we return to our original proposal via rte_security? It still looks to me like clean and straightforward way to enable this new API, and probably wouldn't cause that much controversy.
What do you think?

Quoted 236 lines

As per my understanding you only need a new dev-op for sync support. Session APIs Will remain the same and you will have some extra fields packed in xform structs.

The PMD will need to maintain a pointer to the per session process function while creating Session and will be used by the dev-op API at runtime without any extra check at runtime.

Quoted 229 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

2. As you pointed in that case it will be just one process() function per

device.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

So if PMD would like to have several process() functions for different type

of

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

sessions
(let say one per alg) first thing it has to do inside it's process() - read

session

Quoted 11 lines
Quoted 11 lines

data

Quoted 10 lines
Quoted 10 lines

and
based on that, do a jump/call to particular internal sub-routine. Something like:
driver_id = get_pmd_driver_id();
priv_ses = ses->sess_data[driver_id];
Then either:
switch(priv_sess->alg) {case XXX: process_XXX(priv_sess, ...);break;...} OR
priv_ses->process(priv_sess, ...);

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

to select and call the proper function.
Looks like totally unnecessary overhead to me.
Though if we'll have ability to query/extract some sort session_ops based

on

Quoted 22 lines
Quoted 12 lines

the

Quoted 4 lines
Quoted 2 lines

xform -
we can avoid this extra de-refererence+jump/call thing.

What is the issue in the priv_ses->process(); approach?

Nothing at all.
What I am saying that schema with dev_ops dev[dev_id]->dev_ops.process(ses->priv_ses[driver_id], ...)
|
|-> priv_ses->process(...)

Quoted 4 lines

Has bigger overhead then just:
process(ses,...);

So what for to introduce extra-level of indirection here?

Explained above.

Quoted 3 lines
Quoted 3 lines

I don't understand what are you saving by not doing this.
In any case you would need to identify which session correspond to which

Quoted 7 lines
Quoted 5 lines

process().

Yes, sure, but I think we can make user to store information that relationship, in a way he likes: store process() pointer for each session, or group sessions that share the same process() somehow, or...

So whatever relationship that user will make and store will make its life

complicated.

Quoted 1 line

If we can hide that information in the driver, then what is the issue in that and

user

Quoted 1 line

Will not need to worry. He would just call the process() and driver will choose

which

Quoted 1 line

Process need to be called.

Driver can do that at config/init time.
Then at run-time we can avoid that choice at all and call already chosen function.

Quoted 2 lines

I think we should have a POC around this and see the difference in the cycle

count.

Quoted 1 line

IMO it would be negligible and we would end up making a generic API set

which

Quoted 13 lines

can be used by others as well.

Quoted 11 lines
Quoted 2 lines

For that you would be doing it somewhere in your data path.

Why at data-path?
Only once at session creation/initialization time. Or might be even once per group of sessions.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I am not sure if you would need a new session init API for this as

nothing

Quoted 9 lines
Quoted 9 lines

would

Quoted 8 lines
Quoted 8 lines

be visible to

Quoted 7 lines

the app or lib.

Quoted 5 lines
lib/librte_cryptodev/rte_cryptodev.h +5 -0
+ * - Not storing process() pointer inside the session -
+ *   Allows user to choose does he want to store a process() pointer
+ *   per session, or per group of sessions for that device that share
+ *   the same input xforms. I.E. extra flexibility for the user,
+ *   plus allows us to keep cpu_sym_session totally opaque, see

above.

Quoted 37 lines
Quoted 37 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

If multiple sessions need to be processed via the same process function, PMD would save the same process in all the sessions, I don't think there

would

Quoted 33 lines
Quoted 33 lines
Quoted 1 line

be any perf overhead with that.

I think it would, see above.

Quoted 29 lines
Quoted 29 lines
lib/librte_cryptodev/rte_cryptodev.h +28 -0
+ * Sketched usage model:
+ * ....
+ * /* control path, alloc/init session */
+ * int32_t sz = rte_crypto_cpu_sym_session_size(dev_id, &xform);
+ * struct rte_crypto_cpu_sym_session *ses = user_alloc(..., sz);
+ * rte_crypto_cpu_sym_process_t process =
+ *     rte_crypto_cpu_sym_session_func(dev_id, &xform);
+ * rte_crypto_cpu_sym_session_init(dev_id, ses, &xform);
+ * ...
+ * /* data-path*/
+ * process(ses, ....);
+ * ....
+ * /* control path, termiante/free session */
+ * rte_crypto_cpu_sym_session_fini(dev_id, ses);
+ */
+
+/**
+ * vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_crypto_vec {
+       struct iovec *vec;
+       uint32_t num;
+};
+
+/*
+ * Data-path bulk process crypto function.
+ */
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev.h void * · line 1334

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +8 -0
+typedef void (*rte_crypto_cpu_sym_process_t)(
+               struct rte_crypto_cpu_sym_session *sess,
+               struct rte_crypto_vec buf[], void *iv[], void *aad[],
+               void *digest[], int status[], uint32_t num);
+/*
+ * for given device return process function specific to input xforms
+ * on error - return NULL and set rte_errno value.
+ * Note that for same input xfroms for the same device should

return

Quoted 45 lines
Quoted 45 lines
Quoted 45 lines
Quoted 45 lines
Quoted 36 lines
Quoted 36 lines
lib/librte_cryptodev/rte_cryptodev.h +35 -0
+ * the same process function.
+ */
+__rte_experimental
+rte_crypto_cpu_sym_process_t
+rte_crypto_cpu_sym_session_func(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Return required session size in bytes for given set of xforms.
+ * if xforms == NULL, then return the max possible session size,
+ * that would fit session for any supported by the device algorithm.
+ * if CPU mode is not supported at all, or requeted in xform
+ * algorithm is not supported, then return -ENOTSUP.
+ */
+__rte_experimental
+int
+rte_crypto_cpu_sym_session_size(uint8_t dev_id,
+                       const struct rte_crypto_sym_xform *xforms);
+
+/*
+ * Initialize session.
+ * It is caller responsibility to allocate enough space for it.
+ * See rte_crypto_cpu_sym_session_size above.
+ */
+__rte_experimental
+int rte_crypto_cpu_sym_session_init(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+__rte_experimental
+void
+rte_crypto_cpu_sym_session_fini(uint8_t dev_id,
+                       struct rte_crypto_cpu_sym_session *sess);
+
+
 #ifdef __cplusplus
Quoted 8 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void

Quoted 8 lines
lib/librte_cryptodev/rte_cryptodev.h +0 -0
 }
 #endif
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
index defe05ea0..ed7e63fab 100644
@@ -310,6 +310,20 @@ typedef void

(*cryptodev_sym_free_session_t)(struct

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 311

Quoted 2 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +0 -0
 rte_cryptodev *dev,
 typedef void (*cryptodev_asym_free_session_t)(struct rte_cryptodev

*dev,

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +1 -0
                struct rte_cryptodev_asym_session *sess);
 
+typedef int (*cryptodev_cpu_sym_session_size_t) (struct

rte_cryptodev

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 317

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef int (*cryptodev_cpu_sym_session_init_t) (struct

rte_cryptodev

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

*dev,

Quoted 4 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 321

Quoted 4 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +4 -0
+                       struct rte_crypto_cpu_sym_session *sess,
+                       const struct rte_crypto_sym_xform *xforms);
+
+typedef void (*cryptodev_cpu_sym_session_fini_t) (struct

rte_cryptodev

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

*dev,

Quoted 3 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h typedef void · line 324

Quoted 3 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       struct rte_crypto_cpu_sym_session *sess);
+
+typedef rte_crypto_cpu_sym_process_t

(*cryptodev_cpu_sym_session_func_t)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

lib/librte_cryptodev/rte_cryptodev_pmd.h

Quoted 7 lines

(

lib/librte_cryptodev/rte_cryptodev_pmd.h +3 -0
+                       struct rte_cryptodev *dev,
+                       const struct rte_crypto_sym_xform *xforms);
+
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
        cryptodev_configure_t dev_configure;    /**< Configure device.

*/

Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
lib/librte_cryptodev/rte_cryptodev_pmd.h +5 -0
@@ -343,6 +357,11 @@ struct rte_cryptodev_ops {
        /**< Clear a Crypto sessions private data. */
        cryptodev_asym_free_session_t asym_session_clear;
        /**< Clear a Crypto sessions private data. */
+
+       cryptodev_cpu_sym_session_size_t sym_cpu_session_get_size;
+       cryptodev_cpu_sym_session_func_t sym_cpu_session_get_func;
+       cryptodev_cpu_sym_session_init_t sym_cpu_session_init;
+       cryptodev_cpu_sym_session_fini_t sym_cpu_session_fini;
 };
 
 
#25 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 3 lines

Hi Akhil,

..[snip]

Quoted 49 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

OK let us assume that you have a separate structure. But I have a few

queries:

Quoted 4 lines
Quoted 4 lines
Quoted 1 line

1. how can multiple drivers use a same session

As a short answer: they can't.
It is pretty much the same approach as with rte_security - each device

needs

Quoted 7 lines
Quoted 7 lines

to

Quoted 3 lines
Quoted 3 lines

create/init its own session.
So upper layer would need to maintain its own array (or so) for such case. Though the question is why would you like to have same session over

multiple

Quoted 2 lines
Quoted 2 lines

SW backed devices?
As it would be anyway just a synchronous function call that will be

executed

Quoted 6 lines
Quoted 6 lines

on

Quoted 4 lines
Quoted 1 line

the same cpu.

I may have single FAT tunnel which may be distributed over multiple Cores, and each core is affined to a different SW device.

Quoted 10 lines
Quoted 2 lines

If it is pure SW, then we don't need multiple devices for such scenario. Device in that case is pure abstraction that we can skip.

Yes agreed, but that liberty is given to the application whether it need multiple devices with single queue or a single device with multiple queues.
I think that independence should not be broken in this new API.

Quoted 4 lines
Quoted 4 lines

So a single session may be accessed by multiple devices.

One more example would be depending on packet sizes, I may switch

between

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

HW/SW PMDs with the same session.

Sure, but then we'll have multiple sessions.

No, the session will be same and it will have multiple private data for each of

the PMD.

Quoted 2 lines
Quoted 2 lines

BTW, we have same thing now - these private session pointers are just

stored

Quoted 6 lines
Quoted 4 lines

inside the same rte_crypto_sym_session.
And if user wants to support this model, he would also need to store <dev_id, queue_id>
pair for each HW device anyway.

Yes agreed, but how is that thing happening in your new struct, you cannot

support that.

Quoted 11 lines

User can store all these info in his own struct.
That's exactly what we have right now.
Let say ipsec-secgw has to store for each IPsec SA:
pointer to crypto-session and/or pointer to security session plus (for lookaside-devices) cdev_id_qp that allows it to extract dev_id + queue_id information.
As I understand that works for now, as each ipsec_sa uses only one dev+queue. Though if someone would like to use multiple devices/queues for the same SA - he would need to have an array of these <dev+queue> pairs. So even right now rte_cryptodev_sym_session is not self-consistent and requires extra information to be maintained by user.

Why are you increasing the complexity for the user application.
The new APIs and struct should be such that it need to do minimum changes in the stack so that stack is portable on multiple vendors.
You should try to hide as much complexity in the driver or lib to give the user simple APIs.

Having a same session for multiple devices was added by Intel only for some use cases. And we had split that session create API into 2. Now if those are not useful shall we move back to the single API. I think @Doherty, Declan and @De Lara Guarch, Pablo can comment on this.

Quoted 80 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

2. Can somebody use the scheduler pmd for scheduling the different

type

Quoted 10 lines
Quoted 10 lines

of

Quoted 9 lines
Quoted 9 lines

payloads for the same session?

In theory yes.
Though for that scheduler pmd should have inside it's rte_crypto_cpu_sym_session an array of pointers to the underlying devices sessions.

Quoted 2 lines

With your proposal the APIs would be very specific to your use case

only.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Yes in some way.
I consider that API specific for SW backed crypto PMDs.
I can hardly see how any 'real HW' PMDs (lksd-none, lksd-proto) will

benefit

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

from it.
Current crypto-op API is very much HW oriented.
Which is ok, that's for it was intended for, but I think we also need one

that

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

would be designed
for SW backed implementation in mind.

We may re-use your API for HW PMDs as well which do not have

requirement

Quoted 8 lines
Quoted 8 lines

of

Quoted 2 lines

Crypto-op/mbuf etc.
The return type of your new process API may have a status which say

'processed'

Quoted 1 line

Or can be say 'enqueued'. So if it is 'enqueued', we may have a new API for

raw

Quoted 2 lines

Bufs dequeue as well.

Quoted 7 lines
Quoted 7 lines
Quoted 1 line

This requirement can be for any hardware PMDs like QAT as well.

I don't think it is a good idea to extend this API for async (lookaside) devices. You'll need to:
- provide dev_id and queue_id for each process(enqueue) and dequeuer
operation.
- provide IOVA for all buffers passing to that function (data buffers, digest,

IV,

Quoted 12 lines
Quoted 11 lines

aad).
- On dequeue provide some way to associate dequed data and digest buffers
with
crypto-session that was used (and probably with mbuf).
So most likely we'll end up with another just version of our current crypto-op
structure.
If you'd like to get rid of mbufs dependency within current crypto-op API that understandable,
but I don't think we should have same API for both sync (CPU) and async (lookaside) cases.
It doesn't seem feasible at all and voids whole purpose of that patch.

Quoted 1 line

At this moment we are not much concerned about the dequeue API and about

the

Quoted 1 line

HW PMD support. It is just that the new API should be generic enough to be

used in

Quoted 1 line

some future scenarios as well. I am just highlighting the possible usecases

which can

Quoted 1 line

be there in future.

Sorry, but I strongly disagree with such approach.
We should stop adding/modifying API 'just in case' and because 'it might be useful for some future HW'.
Inside DPDK we already do have too many dev level APIs without any implementations.
That's quite bad practice and very dis-orienting for end-users.
I think to justify API additions/changes we need at least one proper implementation for it,
or at least some strong evidence that people are really committed to support it in nearest future.
BTW, that what TB agreed on, nearly a year ago.

Quoted 13 lines

This new API (if we'll go ahead with it of course) would stay experimental for some time anyway
to make sure we don't miss anything needed (I think for about a year time- frame).
So if you guys *really* want to extend it support _async_ devices too - I am open for modifications/additions here.
Though personally I think such addition would over-complicate things and we'll end up with
another reincarnation of current crypto-op.
We actually discussed it internally, and decided to drop that idea because of that. Again, my opinion - for lookaside devices it might be better to try to optimize current crypto-op path (remove mbuf requirement, probably add ability to group by session on enqueue/dequeue, etc.).

I agree that the new API is experimental and can be modified later. So no issues in that, but we can keep some things in mind while defining APIs. These were some comments from my side, if those are impacting the current scenario, you can drop those. We will take care of those later.

Quoted 10 lines
Quoted 3 lines

What is the issue that you face in making a dev-op for this new API. Do you see

any

Quoted 1 line

performance impact with that?

There are two main things:
1. user would need to maintain and provide for each process() call dev_id+queue_id.
That's means extra (and totally unnecessary for SW) overhead.

You are using a crypto device for performing the processing, you must use dev_id to identify which SW device it is. This is how the DPDK Framework works.
.

Quoted 27 lines

2. yes I would expect some perf overhead too - it would be extra call or branch. Again as it would be data-dependency - most likely cpu wouldn't be able to pipeline
it efficiently:

rte_crypto_sym_process(uint8_t dev_id, uint16 qp_id, rte_crypto_sym_session *ses, ...)

{
     struct rte_cryptodev *dev = &rte_cryptodevs[dev_id];
     return (*dev->process)(sess->data[dev->driver_id, ...);
}

driver_specific_process(driver_specific_sym_session *sess)

{
   return sess->process(sess, ...) ;
}

I didn't make any exact measurements but sure it would be slower than just:
session_udata->process(session->udata->sess, ...);
Again it would be much more noticeable on low end cpus.
Let say here:
http://mails.dpdk.org/archives/dev/2019-September/144350.html
Jerin claims 1.5-3% drop for introducing extra call via hiding eth_dev contents - I suppose we would have something similar here.
I do realize that in majority of cases crypto is more expensive then RX/TX, but still.

Quoted 2 lines

If it would be a really unavoidable tradeoff (support already existing API, or so) I wouldn't mind, but I don't see any real need for it right now.

Calling session_udata->process(session->udata->sess, ...); from the application and Application need to maintain for each PMD the process() API in its memory will make the application not portable to other vendors.

What we are doing here is defining another way to create sessions for the same stuff that is already done. This make applications non-portable and confusing for the application writer.

I would say you should do some profiling first. As you also mentioned crypto workload is more Cycle consuming, it will not impact this case.

Quoted 129 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

That is why a dev-ops would be a better option.

Quoted 2 lines
Quoted 2 lines

When you would add more functionality to this sync API/struct, it will

end

Quoted 4 lines
Quoted 4 lines

up

Quoted 3 lines
Quoted 3 lines

being the same API/struct.

Quoted 2 lines

Let us see how close/ far we are from the existing APIs when the

actual

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

implementation is done.

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is

supported(from feature flag) and

Quoted 1 line

Have relevant fields in xform(the newly added ones which are

packed

Quoted 8 lines
Quoted 8 lines

as

Quoted 7 lines
Quoted 7 lines

per

Quoted 6 lines
Quoted 6 lines

your suggestions) set,

Quoted 5 lines

It will create that type of session.

Quoted 2 lines

+ * Main points:
+ * - Current crypto-dev API is reasonably mature and it is

desirable

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ * to keep it unchanged (API/ABI stability). From other side, this
+ * new sync API is new one and probably would require extra

changes.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * Having it as a new one allows to mark it as experimental,

without

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

+ * affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more flexibility
+ * to the PMD writers and again allows to avoid ABI breakages

in

Quoted 4 lines
Quoted 4 lines

future.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

+ * - process() function per set of xforms
+ * allows to expose different process() functions for different
+ * xform combinations. PMD writer can decide, does he wants

to

Quoted 9 lines
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
+ *   push all supported algorithms into one process() function,
+ *   or spread it across several ones.
+ *   I.E. More flexibility for PMD writer.

Which process function should be chosen is internal to PMD, how

would

Quoted 3 lines
Quoted 3 lines

that

Quoted 2 lines
Quoted 2 lines

info

Quoted 1 line

be visible to the application or the library. These will get stored in

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

session

Quoted 2 lines
Quoted 2 lines

private

Quoted 1 line

data. It would be upto the PMD writer, to store the per session

process

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

function in

Quoted 3 lines

the session private data.

Process function would be a dev ops just like enc/deq operations

and it

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

should

Quoted 4 lines
Quoted 4 lines

call

Quoted 1 line

The respective process API stored in the session private data.

That model (via devops) is possible, but has several drawbacks from

my

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

perspective:

1. It means we'll need to pass dev_id as a parameter to process()

function.

Quoted 12 lines
Quoted 12 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

Though in fact dev_id is not a relevant information for us here (all we need is pointer to the session and pointer to the fuction to call) and I tried to avoid using it in data-path functions for that API.

You have a single vdev, but someone may have multiple vdevs for each

thread,

Quoted 5 lines
Quoted 5 lines

or may

Quoted 1 line

Have same dev with multiple queues for each core.

That's fine. As I said above it is a SW backed implementation. Each session has to be a separate entity that contains all necessary

information

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

(keys, alg/mode info, etc.) to process input buffers. Plus we need the actual function pointer to call.
I just don't see what for we need a dev_id in that situation.

To iterate the session private data in the session.

Quoted 2 lines

Again, here we don't need care about queues and their pinning to cores. If let say someone would like to process buffers from the same IPsec SA

on 2

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

different cores in parallel, he can just create 2 sessions for the same

xform,

Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

give one to thread #1 and second to thread #2.
After that both threads are free to call process(this_thread_ses, ...) at will.

Say you have a 16core device to handle 100G of traffic on a single tunnel. Will we make 16 sessions with same parameters?

Absolutely same question we can ask for current crypto-op API. You have lookaside crypto-dev with 16 HW queues, each queue is serviced by different CPU.
For the same SA, do you need a separate session per queue, or is it ok to

reuse

Quoted 8 lines
Quoted 6 lines

current one?
AFAIK, right now this is a grey area not clearly defined.
For crypto-devs I am aware - user can reuse the same session (as PMD uses it read-only).
But again, right now I think it is not clearly defined and is implementation specific.

User can use the same session, that is what I am also insisting, but it may have

separate

Quoted 1 line

Session private data. Cryptodev session create API provide that functionality

and we can

Quoted 1 line

Leverage that.

rte_cryptodev_sym_session. sess_data[] is indexed by driver_id, which means we can't use
the same rte_cryptodev_sym_session to hold sessions for both sync and async mode
for the same device. Off course we can add a hard requirement that any driver that wants to
support process() has to create sessions that can handle both process and enqueue/dequeue,
but then again what for to create such overhead?

Quoted 8 lines

BTW, to be honest, I don't consider current rte_cryptodev_sym_session construct for multiple device_ids:

__extension__ struct {
                void *data;
                uint16_t refcnt;
        } sess_data[0];
        /**< Driver specific session material, variable size */

Yes I also feel the same. I was also not in favor of this when it was introduced. Please go ahead and remove this. I have no issues with that.

Quoted 7 lines

as an advantage.
It looks too error prone for me:
1. Simultaneous session initialization/de-initialization for devices with the same driver_id is not possible.
2. It assumes that all device driver will be loaded before we start to create session pools.

Quoted 5 lines

Right now it seems ok, as no-one requires such functionality, but I don't know how it will be in future.
For me rte_security session model, where for each security context user have to create new session
looks much more robust.

Agreed

Quoted 12 lines
Quoted 3 lines

BTW, I can see a v2 to this RFC which is still based on security library.

Yes, v2 was concentrated on fixing found issues, some code restructuring, i.e. - changes that would be needed anyway whatever API aproach we'll choose.

Quoted 2 lines

When do you plan
To submit the patches for crypto based APIs. We have RC1 merge deadline for

this

Quoted 1 line

patchset on 21st Oct.

Quoted 8 lines

We'd like to start working on it ASAP, but it seems we still have a major disagreement
about how this crypto-dev API should look like.
Which makes me think - should we return to our original proposal via rte_security?
It still looks to me like clean and straightforward way to enable this new API, and probably wouldn't cause that much controversy.
What do you think?

I cannot spend more time discussing on this until RC1 date. I have some other stuff pending. You can send the patches early next week with the approach that I mentioned or else we can discuss this post RC1(which would mean deferring to 20.02).

But moving back to security is not acceptable to me. The code should be put where it is intended and not where it is easy to put. You are not doing any rte_security stuff.

Regards,
Akhil

#26 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Thanks for the review and comments!
Knowing you are extremely busy. Here is my point in brief:
I think placing the CPU synchronous crypto in the rte_security make sense, as

1. rte_security contains inline crypto and lookaside crypto action type already, adding cpu_crypto action type is reasonable.
2. rte_security contains the security features may not supported by all devices, such as crypto, ipsec, and PDCP. cpu_crypto follow this category, again crypto.
3. placing CPU synchronous crypto API in rte_security is natural - as inline mode works synchronously, too. However cryptodev doesn't.
4. placing CPU synchronous crypto API in rte_security helps boosting SW crypto performance, I have already provided a simple perf test inside the unit test in the patchset for the user to try out - just comparing its output against DPDK crypto perf app output.
5. placing CPU synchronous crypto API in cryptodev will never serve HW lookaside crypto PMDs, as making them to work synchronously have huge performance penalty. However Cryptodev framework's existing design is providing APIs that will work in all crypto PMDs (rte_cryptodev_enqueue_burst / dequeue_burst for example), this does not fit in cryptodev's principle.
6. placing CPU synchronous crypto API in cryptodev confuses the user, as:
- the session created for async mode may not work in sync mode
- both enqueue/dequeue and cpu_crypto_process does the same crypto processing, but one PMD may support only one API (set), the other may support another, and the third PMD supports both. We have to provide another API to let the user query which one to support which.
- two completely different code paths for async/sync mode.
7. You said in the end of the email - placing CPU synchronous crypto API into rte_security is not acceptable as it does not do any rte_security stuff - crypto isn't? You may call this a quibble, but in my idea, in the patchset both PMDs' implementations did offload the work to the CPU's special circuit designed dedicated to accelerate the crypto processing.

To me cryptodev is the one CPU synchronous crypto API should not go into, rte_security is.

Regards,
Fan

Quoted 480 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Friday, October 11, 2019 2:24 PM
To: Ananyev, Konstantin <[email protected]>; '[email protected]' <[email protected]>; De Lara Guarch, Pablo <[email protected]>; 'Thomas Monjalon' <[email protected]>; Zhang, Roy Fan <[email protected]>; Doherty, Declan <[email protected]>
Cc: 'Anoob Joseph' <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 3 lines

Hi Akhil,

..[snip]

Quoted 12 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

OK let us assume that you have a separate structure. But I have a few

queries:

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

1. how can multiple drivers use a same session

As a short answer: they can't.
It is pretty much the same approach as with rte_security - each device

needs

Quoted 3 lines
Quoted 3 lines

to

Quoted 2 lines
Quoted 2 lines

create/init its own session.
So upper layer would need to maintain its own array (or so) for such

case.

Quoted 18 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines

Though the question is why would you like to have same session over

multiple

Quoted 3 lines
Quoted 3 lines

SW backed devices?
As it would be anyway just a synchronous function call that will be

executed

Quoted 11 lines
Quoted 8 lines

on

Quoted 4 lines
Quoted 1 line

the same cpu.

I may have single FAT tunnel which may be distributed over multiple Cores, and each core is affined to a different SW device.

If it is pure SW, then we don't need multiple devices for such scenario. Device in that case is pure abstraction that we can skip.

Yes agreed, but that liberty is given to the application whether it need multiple devices with single queue or a single device with multiple

queues.

Quoted 26 lines
Quoted 6 lines

I think that independence should not be broken in this new API.

Quoted 5 lines
Quoted 5 lines

So a single session may be accessed by multiple devices.

One more example would be depending on packet sizes, I may switch

between

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

HW/SW PMDs with the same session.

Sure, but then we'll have multiple sessions.

No, the session will be same and it will have multiple private data for each of

the PMD.

Quoted 3 lines
Quoted 3 lines

BTW, we have same thing now - these private session pointers are just

stored

Quoted 6 lines
Quoted 3 lines

inside the same rte_crypto_sym_session.
And if user wants to support this model, he would also need to store <dev_id, queue_id> pair for each HW device anyway.

Yes agreed, but how is that thing happening in your new struct, you cannot

support that.

Quoted 9 lines

User can store all these info in his own struct.
That's exactly what we have right now.
Let say ipsec-secgw has to store for each IPsec SA:
pointer to crypto-session and/or pointer to security session plus (for lookaside-devices) cdev_id_qp that allows it to extract dev_id + queue_id information.
As I understand that works for now, as each ipsec_sa uses only one dev+queue. Though if someone would like to use multiple devices/queues for the same SA - he would need to have an array of these <dev+queue>

pairs.

Quoted 2 lines

So even right now rte_cryptodev_sym_session is not self-consistent and requires extra information to be maintained by user.

Why are you increasing the complexity for the user application. The new APIs and struct should be such that it need to do minimum changes in the stack so that stack is portable on multiple vendors.
You should try to hide as much complexity in the driver or lib to give the user simple APIs.

Having a same session for multiple devices was added by Intel only for some use cases.
And we had split that session create API into 2. Now if those are not useful shall we move back to the single API. I think @Doherty, Declan and @De Lara Guarch, Pablo can comment on this.

Quoted 30 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

2. Can somebody use the scheduler pmd for scheduling the different

type

Quoted 11 lines
Quoted 11 lines

of

Quoted 10 lines
Quoted 10 lines

payloads for the same session?

In theory yes.
Though for that scheduler pmd should have inside it's rte_crypto_cpu_sym_session an array of pointers to the underlying devices sessions.

Quoted 3 lines

With your proposal the APIs would be very specific to your use case

only.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Yes in some way.
I consider that API specific for SW backed crypto PMDs. I can hardly see how any 'real HW' PMDs (lksd-none, lksd-proto) will

benefit

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

from it.
Current crypto-op API is very much HW oriented. Which is ok, that's for it was intended for, but I think we also need one

Quoted 19 lines

that

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

would be designed
for SW backed implementation in mind.

We may re-use your API for HW PMDs as well which do not have

requirement

Quoted 13 lines
Quoted 13 lines

of

Quoted 3 lines

Crypto-op/mbuf etc.
The return type of your new process API may have a status which say

'processed'

Quoted 2 lines

Or can be say 'enqueued'. So if it is 'enqueued', we may have a new API for

raw

Quoted 3 lines

Bufs dequeue as well.

This requirement can be for any hardware PMDs like QAT as well.

I don't think it is a good idea to extend this API for async (lookaside)

devices.

Quoted 40 lines
Quoted 5 lines
Quoted 5 lines

You'll need to:
- provide dev_id and queue_id for each process(enqueue) and
dequeuer operation.
- provide IOVA for all buffers passing to that function (data
buffers, digest,

IV,

Quoted 12 lines
Quoted 11 lines

aad).
- On dequeue provide some way to associate dequed data and digest
buffers with
crypto-session that was used (and probably with mbuf).
So most likely we'll end up with another just version of our
current crypto-op structure.
If you'd like to get rid of mbufs dependency within current crypto-op API that understandable, but I don't think we should have same API for both sync (CPU) and async
(lookaside) cases.
It doesn't seem feasible at all and voids whole purpose of that patch.

Quoted 2 lines

At this moment we are not much concerned about the dequeue API and about

the

Quoted 2 lines

HW PMD support. It is just that the new API should be generic enough to be

used in

Quoted 2 lines

some future scenarios as well. I am just highlighting the possible usecases

which can

Quoted 1 line

be there in future.

Sorry, but I strongly disagree with such approach.
We should stop adding/modifying API 'just in case' and because 'it might be useful for some future HW'.
Inside DPDK we already do have too many dev level APIs without any implementations.
That's quite bad practice and very dis-orienting for end-users. I think to justify API additions/changes we need at least one proper implementation for it, or at least some strong evidence that people are really committed to support it in nearest future. BTW, that what TB agreed on, nearly a year ago.

Quoted 8 lines

This new API (if we'll go ahead with it of course) would stay experimental for some time anyway to make sure we don't miss anything needed (I think for about a year time- frame).
So if you guys *really* want to extend it support _async_ devices too
- I am open for modifications/additions here.
Though personally I think such addition would over-complicate things and we'll end up with another reincarnation of current crypto-op. We actually discussed it internally, and decided to drop that idea because

of that.

Quoted 3 lines

Again, my opinion - for lookaside devices it might be better to try to optimize current crypto-op path (remove mbuf requirement, probably add ability to group by session on enqueue/dequeue, etc.).

I agree that the new API is experimental and can be modified later. So no issues in that, but we can keep some things in mind while defining APIs. These were some comments from my side, if those are impacting the current scenario, you can drop those. We will take care of those later.

Quoted 11 lines
Quoted 4 lines

What is the issue that you face in making a dev-op for this new API. Do you see

any

Quoted 1 line

performance impact with that?

There are two main things:
1. user would need to maintain and provide for each process() call dev_id+queue_id.
That's means extra (and totally unnecessary for SW) overhead.

You are using a crypto device for performing the processing, you must use dev_id to identify which SW device it is. This is how the DPDK Framework works.
.

Quoted 1 line

2. yes I would expect some perf overhead too - it would be extra call or

branch.

Quoted 8 lines

Again as it would be data-dependency - most likely cpu wouldn't be able to pipeline it efficiently:

rte_crypto_sym_process(uint8_t dev_id, uint16 qp_id,

rte_crypto_sym_session *ses, ...) {
     struct rte_cryptodev *dev = &rte_cryptodevs[dev_id];
     return (*dev->process)(sess->data[dev->driver_id, ...); }
Quoted 5 lines
driver_specific_process(driver_specific_sym_session *sess) {
   return sess->process(sess, ...) ;
}

I didn't make any exact measurements but sure it would be slower than

just:

Quoted 11 lines

session_udata->process(session->udata->sess, ...); Again it would be much more noticeable on low end cpus.
Let say here:
http://mails.dpdk.org/archives/dev/2019-September/144350.html
Jerin claims 1.5-3% drop for introducing extra call via hiding eth_dev contents - I suppose we would have something similar here. I do realize that in majority of cases crypto is more expensive then RX/TX, but still.

If it would be a really unavoidable tradeoff (support already existing API, or so) I wouldn't mind, but I don't see any real need for it right now.

Calling session_udata->process(session->udata->sess, ...); from the application and Application need to maintain for each PMD the process() API in its memory will make the application not portable to other vendors.

What we are doing here is defining another way to create sessions for the same stuff that is already done. This make applications non-portable and confusing for the application writer.

I would say you should do some profiling first. As you also mentioned crypto workload is more Cycle consuming, it will not impact this case.

Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

That is why a dev-ops would be a better option.

Quoted 3 lines
Quoted 3 lines

When you would add more functionality to this sync API/struct, it will

end

Quoted 5 lines
Quoted 5 lines

up

Quoted 4 lines
Quoted 4 lines

being the same API/struct.

Quoted 3 lines

Let us see how close/ far we are from the existing APIs when the

actual

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

implementation is done.

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is

supported(from feature flag) and

Quoted 2 lines

Have relevant fields in xform(the newly added ones which are

packed

Quoted 2 lines
Quoted 2 lines

as

Quoted 1 line
Quoted 1 line

per

Quoted 9 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

your suggestions) set,

Quoted 6 lines

It will create that type of session.

Quoted 3 lines

+ * Main points:
+ * - Current crypto-dev API is reasonably mature and
+ it is

desirable

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * to keep it unchanged (API/ABI stability). From other

side, this

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * new sync API is new one and probably would require

extra

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

changes.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * Having it as a new one allows to mark it as experimental,

without

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ * affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more

flexibility

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * to the PMD writers and again allows to avoid ABI

breakages

Quoted 4 lines

in

Quoted 3 lines
Quoted 3 lines

future.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ * - process() function per set of xforms
+ * allows to expose different process() functions for

different

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * xform combinations. PMD writer can decide, does he

wants

Quoted 2 lines

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * push all supported algorithms into one process()

function,

Quoted 26 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

+ * or spread it across several ones.
+ * I.E. More flexibility for PMD writer.

Which process function should be chosen is internal to PMD, how

would

Quoted 4 lines
Quoted 4 lines

that

Quoted 3 lines
Quoted 3 lines

info

Quoted 2 lines

be visible to the application or the library. These will get stored in

the

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

session

Quoted 3 lines
Quoted 3 lines

private

Quoted 2 lines

data. It would be upto the PMD writer, to store the per session

process

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

function in

Quoted 4 lines

the session private data.

Process function would be a dev ops just like enc/deq operations

and it

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

should

Quoted 3 lines
Quoted 3 lines

call

Quoted 1 line

The respective process API stored in the session private data.

Quoted 11 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

That model (via devops) is possible, but has several drawbacks from

my

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

perspective:

1. It means we'll need to pass dev_id as a parameter to process()

function.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Though in fact dev_id is not a relevant information for us here (all we need is pointer to the session and pointer to the fuction to call) and I tried to avoid using it in data-path

functions for that API.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

You have a single vdev, but someone may have multiple vdevs for each

thread,

Quoted 6 lines
Quoted 6 lines

or may

Quoted 1 line

Have same dev with multiple queues for each core.

That's fine. As I said above it is a SW backed implementation. Each session has to be a separate entity that contains all necessary

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

information

Quoted 7 lines
Quoted 3 lines

(keys, alg/mode info, etc.) to process input buffers. Plus we need the actual function pointer to call.
I just don't see what for we need a dev_id in that situation.

To iterate the session private data in the session.

Quoted 1 line

Again, here we don't need care about queues and their pinning to

cores.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

If let say someone would like to process buffers from the same IPsec SA

on 2

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

different cores in parallel, he can just create 2 sessions for the same

xform,

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

give one to thread #1 and second to thread #2.
After that both threads are free to call process(this_thread_ses, ...)

at will.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Say you have a 16core device to handle 100G of traffic on a single

tunnel.

Quoted 30 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line

Will we make 16 sessions with same parameters?

Absolutely same question we can ask for current crypto-op API. You have lookaside crypto-dev with 16 HW queues, each queue is serviced by different CPU.
For the same SA, do you need a separate session per queue, or is it ok to

reuse

Quoted 9 lines
Quoted 6 lines

current one?
AFAIK, right now this is a grey area not clearly defined. For crypto-devs I am aware - user can reuse the same session (as PMD uses it read-only).
But again, right now I think it is not clearly defined and is implementation specific.

User can use the same session, that is what I am also insisting, but it may have

separate

Quoted 2 lines

Session private data. Cryptodev session create API provide that functionality

and we can

Quoted 1 line

Leverage that.

rte_cryptodev_sym_session. sess_data[] is indexed by driver_id, which means we can't use the same rte_cryptodev_sym_session to hold sessions for both sync and async mode for the same device. Off course we can add a hard requirement that any driver that wants to support process() has to create sessions that can handle both process and enqueue/dequeue, but then again what for to create such overhead?

Quoted 8 lines

BTW, to be honest, I don't consider current rte_cryptodev_sym_session construct for multiple device_ids:

__extension__ struct {
                void *data;
                uint16_t refcnt;
        } sess_data[0];
        /**< Driver specific session material, variable size */

Yes I also feel the same. I was also not in favor of this when it was introduced. Please go ahead and remove this. I have no issues with that.

Quoted 11 lines

as an advantage.
It looks too error prone for me:
1. Simultaneous session initialization/de-initialization for devices with the same driver_id is not possible.
2. It assumes that all device driver will be loaded before we start to create session pools.

Right now it seems ok, as no-one requires such functionality, but I don't know how it will be in future.
For me rte_security session model, where for each security context user have to create new session looks much more robust.

Agreed

Quoted 6 lines
Quoted 3 lines

BTW, I can see a v2 to this RFC which is still based on security library.

Yes, v2 was concentrated on fixing found issues, some code restructuring, i.e. - changes that would be needed anyway whatever API

aproach we'll choose.

Quoted 7 lines
Quoted 4 lines

When do you plan
To submit the patches for crypto based APIs. We have RC1 merge deadline for

this

Quoted 1 line

patchset on 21st Oct.

Quoted 7 lines

We'd like to start working on it ASAP, but it seems we still have a major disagreement about how this crypto-dev API should look like. Which makes me think - should we return to our original proposal via rte_security?
It still looks to me like clean and straightforward way to enable this new API, and probably wouldn't cause that much controversy. What do you think?

I cannot spend more time discussing on this until RC1 date. I have some other stuff pending.
You can send the patches early next week with the approach that I mentioned or else we can discuss this post RC1(which would mean deferring to 20.02).

But moving back to security is not acceptable to me. The code should be put where it is intended and not where it is easy to put. You are not doing any rte_security stuff.

Regards,
Akhil

#27 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Quoted 26 lines

Hi Akhil,

Thanks for the review and comments!
Knowing you are extremely busy. Here is my point in brief:
I think placing the CPU synchronous crypto in the rte_security make sense, as

1. rte_security contains inline crypto and lookaside crypto action type already, adding cpu_crypto action type is reasonable.
2. rte_security contains the security features may not supported by all devices, such as crypto, ipsec, and PDCP. cpu_crypto follow this category, again crypto.
3. placing CPU synchronous crypto API in rte_security is natural - as inline mode works synchronously, too. However cryptodev doesn't.
4. placing CPU synchronous crypto API in rte_security helps boosting SW crypto performance, I have already provided a simple perf test inside the unit test in the patchset for the user to try out - just comparing its output against DPDK crypto perf app output.
5. placing CPU synchronous crypto API in cryptodev will never serve HW lookaside crypto PMDs, as making them to work synchronously have huge performance penalty. However Cryptodev framework's existing design is providing APIs that will work in all crypto PMDs (rte_cryptodev_enqueue_burst / dequeue_burst for example), this does not fit in cryptodev's principle.
6. placing CPU synchronous crypto API in cryptodev confuses the user, as:
- the session created for async mode may not work in sync mode
- both enqueue/dequeue and cpu_crypto_process does the same crypto processing, but one PMD may support only one API (set),
the other may support another, and the third PMD supports both. We have to provide another API to let the user query which one to support which.
- two completely different code paths for async/sync mode.
7. You said in the end of the email - placing CPU synchronous crypto API into rte_security is not acceptable as it does not do any rte_security stuff - crypto isn't? You may call this a quibble, but in my idea, in the patchset both PMDs' implementations did offload the work to the CPU's special circuit designed dedicated to accelerate the crypto processing.

Quoted 1 line

To me cryptodev is the one CPU synchronous crypto API should not go into, rte_security is.

I also don't understand why rte_security is not an option here.
We do have inline-crypto right now, why we can't have cpu-crypto with new process() API here? Actually would like to hear more opinions from the community here -
what other interested parties think is the best way for introducing cpu-crypto specific API?

Konstantin

Quoted 484 lines

Regards,
Fan

Quoted 480 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Friday, October 11, 2019 2:24 PM
To: Ananyev, Konstantin <[email protected]>; '[email protected]' <[email protected]>; De Lara Guarch, Pablo <[email protected]>; 'Thomas Monjalon' <[email protected]>; Zhang, Roy Fan <[email protected]>; Doherty, Declan <[email protected]>
Cc: 'Anoob Joseph' <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 3 lines

Hi Akhil,

..[snip]

Quoted 12 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

OK let us assume that you have a separate structure. But I have a few

queries:

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

1. how can multiple drivers use a same session

As a short answer: they can't.
It is pretty much the same approach as with rte_security - each device

needs

Quoted 3 lines
Quoted 3 lines

to

Quoted 2 lines
Quoted 2 lines

create/init its own session.
So upper layer would need to maintain its own array (or so) for such

case.

Quoted 18 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines

Though the question is why would you like to have same session over

multiple

Quoted 3 lines
Quoted 3 lines

SW backed devices?
As it would be anyway just a synchronous function call that will be

executed

Quoted 11 lines
Quoted 8 lines

on

Quoted 4 lines
Quoted 1 line

the same cpu.

I may have single FAT tunnel which may be distributed over multiple Cores, and each core is affined to a different SW device.

If it is pure SW, then we don't need multiple devices for such scenario. Device in that case is pure abstraction that we can skip.

Yes agreed, but that liberty is given to the application whether it need multiple devices with single queue or a single device with multiple

queues.

Quoted 26 lines
Quoted 6 lines

I think that independence should not be broken in this new API.

Quoted 5 lines
Quoted 5 lines

So a single session may be accessed by multiple devices.

One more example would be depending on packet sizes, I may switch

between

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

HW/SW PMDs with the same session.

Sure, but then we'll have multiple sessions.

No, the session will be same and it will have multiple private data for each of

the PMD.

Quoted 3 lines
Quoted 3 lines

BTW, we have same thing now - these private session pointers are just

stored

Quoted 6 lines
Quoted 3 lines

inside the same rte_crypto_sym_session.
And if user wants to support this model, he would also need to store <dev_id, queue_id> pair for each HW device anyway.

Yes agreed, but how is that thing happening in your new struct, you cannot

support that.

Quoted 9 lines

User can store all these info in his own struct.
That's exactly what we have right now.
Let say ipsec-secgw has to store for each IPsec SA:
pointer to crypto-session and/or pointer to security session plus (for lookaside-devices) cdev_id_qp that allows it to extract dev_id + queue_id information.
As I understand that works for now, as each ipsec_sa uses only one dev+queue. Though if someone would like to use multiple devices/queues for the same SA - he would need to have an array of these <dev+queue>

pairs.

Quoted 2 lines

So even right now rte_cryptodev_sym_session is not self-consistent and requires extra information to be maintained by user.

Why are you increasing the complexity for the user application. The new APIs and struct should be such that it need to do minimum changes in the stack so that stack is portable on multiple vendors.
You should try to hide as much complexity in the driver or lib to give the user simple APIs.

Having a same session for multiple devices was added by Intel only for some use cases.
And we had split that session create API into 2. Now if those are not useful shall we move back to the single API. I think @Doherty, Declan and @De Lara Guarch, Pablo can comment on this.

Quoted 30 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

2. Can somebody use the scheduler pmd for scheduling the different

type

Quoted 11 lines
Quoted 11 lines

of

Quoted 10 lines
Quoted 10 lines

payloads for the same session?

In theory yes.
Though for that scheduler pmd should have inside it's rte_crypto_cpu_sym_session an array of pointers to the underlying devices sessions.

Quoted 3 lines

With your proposal the APIs would be very specific to your use case

only.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Yes in some way.
I consider that API specific for SW backed crypto PMDs. I can hardly see how any 'real HW' PMDs (lksd-none, lksd-proto) will

benefit

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

from it.
Current crypto-op API is very much HW oriented. Which is ok, that's for it was intended for, but I think we also need one

Quoted 19 lines

that

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

would be designed
for SW backed implementation in mind.

We may re-use your API for HW PMDs as well which do not have

requirement

Quoted 13 lines
Quoted 13 lines

of

Quoted 3 lines

Crypto-op/mbuf etc.
The return type of your new process API may have a status which say

'processed'

Quoted 2 lines

Or can be say 'enqueued'. So if it is 'enqueued', we may have a new API for

raw

Quoted 3 lines

Bufs dequeue as well.

This requirement can be for any hardware PMDs like QAT as well.

I don't think it is a good idea to extend this API for async (lookaside)

devices.

Quoted 40 lines
Quoted 5 lines
Quoted 5 lines

You'll need to:
- provide dev_id and queue_id for each process(enqueue) and
dequeuer operation.
- provide IOVA for all buffers passing to that function (data
buffers, digest,

IV,

Quoted 12 lines
Quoted 11 lines

aad).
- On dequeue provide some way to associate dequed data and digest
buffers with
crypto-session that was used (and probably with mbuf).
So most likely we'll end up with another just version of our
current crypto-op structure.
If you'd like to get rid of mbufs dependency within current crypto-op API that understandable, but I don't think we should have same API for both sync (CPU) and async
(lookaside) cases.
It doesn't seem feasible at all and voids whole purpose of that patch.

Quoted 2 lines

At this moment we are not much concerned about the dequeue API and about

the

Quoted 2 lines

HW PMD support. It is just that the new API should be generic enough to be

used in

Quoted 2 lines

some future scenarios as well. I am just highlighting the possible usecases

which can

Quoted 1 line

be there in future.

Sorry, but I strongly disagree with such approach.
We should stop adding/modifying API 'just in case' and because 'it might be useful for some future HW'.
Inside DPDK we already do have too many dev level APIs without any implementations.
That's quite bad practice and very dis-orienting for end-users. I think to justify API additions/changes we need at least one proper implementation for it, or at least some strong evidence that people are really committed to support it in nearest future. BTW, that what TB agreed on, nearly a year ago.

Quoted 8 lines

This new API (if we'll go ahead with it of course) would stay experimental for some time anyway to make sure we don't miss anything needed (I think for about a year time- frame).
So if you guys *really* want to extend it support _async_ devices too
- I am open for modifications/additions here.
Though personally I think such addition would over-complicate things and we'll end up with another reincarnation of current crypto-op. We actually discussed it internally, and decided to drop that idea because

of that.

Quoted 3 lines

Again, my opinion - for lookaside devices it might be better to try to optimize current crypto-op path (remove mbuf requirement, probably add ability to group by session on enqueue/dequeue, etc.).

I agree that the new API is experimental and can be modified later. So no issues in that, but we can keep some things in mind while defining APIs. These were some comments from my side, if those are impacting the current scenario, you can drop those. We will take care of those later.

Quoted 11 lines
Quoted 4 lines

What is the issue that you face in making a dev-op for this new API. Do you see

any

Quoted 1 line

performance impact with that?

There are two main things:
1. user would need to maintain and provide for each process() call dev_id+queue_id.
That's means extra (and totally unnecessary for SW) overhead.

You are using a crypto device for performing the processing, you must use dev_id to identify which SW device it is. This is how the DPDK Framework works.
.

Quoted 1 line

2. yes I would expect some perf overhead too - it would be extra call or

branch.

Quoted 8 lines

Again as it would be data-dependency - most likely cpu wouldn't be able to pipeline it efficiently:

rte_crypto_sym_process(uint8_t dev_id, uint16 qp_id,

rte_crypto_sym_session *ses, ...) {
     struct rte_cryptodev *dev = &rte_cryptodevs[dev_id];
     return (*dev->process)(sess->data[dev->driver_id, ...); }
Quoted 5 lines
driver_specific_process(driver_specific_sym_session *sess) {
   return sess->process(sess, ...) ;
}

I didn't make any exact measurements but sure it would be slower than

just:

Quoted 11 lines

session_udata->process(session->udata->sess, ...); Again it would be much more noticeable on low end cpus.
Let say here:
http://mails.dpdk.org/archives/dev/2019-September/144350.html
Jerin claims 1.5-3% drop for introducing extra call via hiding eth_dev contents - I suppose we would have something similar here. I do realize that in majority of cases crypto is more expensive then RX/TX, but still.

If it would be a really unavoidable tradeoff (support already existing API, or so) I wouldn't mind, but I don't see any real need for it right now.

Calling session_udata->process(session->udata->sess, ...); from the application and Application need to maintain for each PMD the process() API in its memory will make the application not portable to other vendors.

What we are doing here is defining another way to create sessions for the same stuff that is already done. This make applications non-portable and confusing for the application writer.

I would say you should do some profiling first. As you also mentioned crypto workload is more Cycle consuming, it will not impact this case.

Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

That is why a dev-ops would be a better option.

Quoted 3 lines
Quoted 3 lines

When you would add more functionality to this sync API/struct, it will

end

Quoted 5 lines
Quoted 5 lines

up

Quoted 4 lines
Quoted 4 lines

being the same API/struct.

Quoted 3 lines

Let us see how close/ far we are from the existing APIs when the

actual

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

implementation is done.

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

I am not sure if that would be needed.
It would be internal to the driver that if synchronous processing is

supported(from feature flag) and

Quoted 2 lines

Have relevant fields in xform(the newly added ones which are

packed

Quoted 2 lines
Quoted 2 lines

as

Quoted 1 line
Quoted 1 line

per

Quoted 9 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

your suggestions) set,

Quoted 6 lines

It will create that type of session.

Quoted 3 lines

+ * Main points:
+ * - Current crypto-dev API is reasonably mature and
+ it is

desirable

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * to keep it unchanged (API/ABI stability). From other

side, this

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * new sync API is new one and probably would require

extra

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

changes.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * Having it as a new one allows to mark it as experimental,

without

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ * affecting existing one.
+ * - Fully opaque cpu_sym_session structure gives more

flexibility

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * to the PMD writers and again allows to avoid ABI

breakages

Quoted 4 lines

in

Quoted 3 lines
Quoted 3 lines

future.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ * - process() function per set of xforms
+ * allows to expose different process() functions for

different

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * xform combinations. PMD writer can decide, does he

wants

Quoted 2 lines

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * push all supported algorithms into one process()

function,

Quoted 26 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

+ * or spread it across several ones.
+ * I.E. More flexibility for PMD writer.

Which process function should be chosen is internal to PMD, how

would

Quoted 4 lines
Quoted 4 lines

that

Quoted 3 lines
Quoted 3 lines

info

Quoted 2 lines

be visible to the application or the library. These will get stored in

the

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

session

Quoted 3 lines
Quoted 3 lines

private

Quoted 2 lines

data. It would be upto the PMD writer, to store the per session

process

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

function in

Quoted 4 lines

the session private data.

Process function would be a dev ops just like enc/deq operations

and it

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

should

Quoted 3 lines
Quoted 3 lines

call

Quoted 1 line

The respective process API stored in the session private data.

Quoted 11 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

That model (via devops) is possible, but has several drawbacks from

my

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

perspective:

1. It means we'll need to pass dev_id as a parameter to process()

function.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Though in fact dev_id is not a relevant information for us here (all we need is pointer to the session and pointer to the fuction to call) and I tried to avoid using it in data-path

functions for that API.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

You have a single vdev, but someone may have multiple vdevs for each

thread,

Quoted 6 lines
Quoted 6 lines

or may

Quoted 1 line

Have same dev with multiple queues for each core.

That's fine. As I said above it is a SW backed implementation. Each session has to be a separate entity that contains all necessary

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

information

Quoted 7 lines
Quoted 3 lines

(keys, alg/mode info, etc.) to process input buffers. Plus we need the actual function pointer to call.
I just don't see what for we need a dev_id in that situation.

To iterate the session private data in the session.

Quoted 1 line

Again, here we don't need care about queues and their pinning to

cores.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

If let say someone would like to process buffers from the same IPsec SA

on 2

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

different cores in parallel, he can just create 2 sessions for the same

xform,

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

give one to thread #1 and second to thread #2.
After that both threads are free to call process(this_thread_ses, ...)

at will.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Say you have a 16core device to handle 100G of traffic on a single

tunnel.

Quoted 30 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line

Will we make 16 sessions with same parameters?

Absolutely same question we can ask for current crypto-op API. You have lookaside crypto-dev with 16 HW queues, each queue is serviced by different CPU.
For the same SA, do you need a separate session per queue, or is it ok to

reuse

Quoted 9 lines
Quoted 6 lines

current one?
AFAIK, right now this is a grey area not clearly defined. For crypto-devs I am aware - user can reuse the same session (as PMD uses it read-only).
But again, right now I think it is not clearly defined and is implementation specific.

User can use the same session, that is what I am also insisting, but it may have

separate

Quoted 2 lines

Session private data. Cryptodev session create API provide that functionality

and we can

Quoted 1 line

Leverage that.

rte_cryptodev_sym_session. sess_data[] is indexed by driver_id, which means we can't use the same rte_cryptodev_sym_session to hold sessions for both sync and async mode for the same device. Off course we can add a hard requirement that any driver that wants to support process() has to create sessions that can handle both process and enqueue/dequeue, but then again what for to create such overhead?

Quoted 8 lines

BTW, to be honest, I don't consider current rte_cryptodev_sym_session construct for multiple device_ids:

__extension__ struct {
                void *data;
                uint16_t refcnt;
        } sess_data[0];
        /**< Driver specific session material, variable size */

Yes I also feel the same. I was also not in favor of this when it was introduced. Please go ahead and remove this. I have no issues with that.

Quoted 11 lines

as an advantage.
It looks too error prone for me:
1. Simultaneous session initialization/de-initialization for devices with the same driver_id is not possible.
2. It assumes that all device driver will be loaded before we start to create session pools.

Right now it seems ok, as no-one requires such functionality, but I don't know how it will be in future.
For me rte_security session model, where for each security context user have to create new session looks much more robust.

Agreed

Quoted 6 lines
Quoted 3 lines

BTW, I can see a v2 to this RFC which is still based on security library.

Yes, v2 was concentrated on fixing found issues, some code restructuring, i.e. - changes that would be needed anyway whatever API

aproach we'll choose.

Quoted 7 lines
Quoted 4 lines

When do you plan
To submit the patches for crypto based APIs. We have RC1 merge deadline for

this

Quoted 1 line

patchset on 21st Oct.

Quoted 7 lines

We'd like to start working on it ASAP, but it seems we still have a major disagreement about how this crypto-dev API should look like. Which makes me think - should we return to our original proposal via rte_security?
It still looks to me like clean and straightforward way to enable this new API, and probably wouldn't cause that much controversy. What do you think?

I cannot spend more time discussing on this until RC1 date. I have some other stuff pending.
You can send the patches early next week with the approach that I mentioned or else we can discuss this post RC1(which would mean deferring to 20.02).

But moving back to security is not acceptable to me. The code should be put where it is intended and not where it is easy to put. You are not doing any rte_security stuff.

Regards,
Akhil

#28 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Quoted 43 lines
Quoted 11 lines

Hi Akhil,

Thanks for the review and comments!
Knowing you are extremely busy. Here is my point in brief:
I think placing the CPU synchronous crypto in the rte_security make sense, as

1. rte_security contains inline crypto and lookaside crypto action type already,

adding cpu_crypto action type is reasonable.

Quoted 1 line

2. rte_security contains the security features may not supported by all devices,

such as crypto, ipsec, and PDCP. cpu_crypto follow this

Quoted 2 lines

category, again crypto.
3. placing CPU synchronous crypto API in rte_security is natural - as inline

mode works synchronously, too. However cryptodev doesn't.

Quoted 1 line

4. placing CPU synchronous crypto API in rte_security helps boosting SW

crypto performance, I have already provided a simple perf test

Quoted 1 line

inside the unit test in the patchset for the user to try out - just comparing its

output against DPDK crypto perf app output.

Quoted 1 line

5. placing CPU synchronous crypto API in cryptodev will never serve HW

lookaside crypto PMDs, as making them to work synchronously

Quoted 1 line

have huge performance penalty. However Cryptodev framework's existing

design is providing APIs that will work in all crypto PMDs

Quoted 1 line

(rte_cryptodev_enqueue_burst / dequeue_burst for example), this does not fit

in cryptodev's principle.

Quoted 3 lines

6. placing CPU synchronous crypto API in cryptodev confuses the user, as:
- the session created for async mode may not work in sync mode
- both enqueue/dequeue and cpu_crypto_process does the same crypto

processing, but one PMD may support only one API (set),

Quoted 1 line

the other may support another, and the third PMD supports both. We have to

provide another API to let the user query which one to

Quoted 3 lines

support which.
- two completely different code paths for async/sync mode.
7. You said in the end of the email - placing CPU synchronous crypto API into

rte_security is not acceptable as it does not do any

Quoted 1 line

rte_security stuff - crypto isn't? You may call this a quibble, but in my idea, in

the patchset both PMDs' implementations did offload the work

Quoted 1 line

to the CPU's special circuit designed dedicated to accelerate the crypto

processing.

Quoted 2 lines

To me cryptodev is the one CPU synchronous crypto API should not go into,

Quoted 8 lines

rte_security is.

I also don't understand why rte_security is not an option here. We do have inline-crypto right now, why we can't have cpu-crypto with new process() API here?
Actually would like to hear more opinions from the community here - what other interested parties think is the best way for introducing cpu-crypto specific API?

I have raised this concern in the weekly status meeting as well. But it looks like nobody is interested.

#29 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Quoted 63 lines

-----Original Message-----
From: Akhil Goyal [mailto:[email protected]]
Sent: Tuesday, October 15, 2019 4:02 PM
To: Ananyev, Konstantin <[email protected]>; Zhang, Roy Fan <[email protected]>; '[email protected]' <[email protected]>; De Lara Guarch, Pablo <[email protected]>; 'Thomas Monjalon' <[email protected]>; Doherty, Declan <[email protected]>
Cc: 'Anoob Joseph' <[email protected]>; Jerin Jacob <[email protected]>; Hemant Agrawal <[email protected]>
Subject: RE: [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Quoted 41 lines
Quoted 9 lines

Hi Akhil,

Thanks for the review and comments!
Knowing you are extremely busy. Here is my point in brief:
I think placing the CPU synchronous crypto in the rte_security make sense, as

1. rte_security contains inline crypto and lookaside crypto action type already,

adding cpu_crypto action type is reasonable.

Quoted 1 line

2. rte_security contains the security features may not supported by all devices,

such as crypto, ipsec, and PDCP. cpu_crypto follow this

Quoted 2 lines

category, again crypto.
3. placing CPU synchronous crypto API in rte_security is natural - as inline

mode works synchronously, too. However cryptodev doesn't.

Quoted 1 line

4. placing CPU synchronous crypto API in rte_security helps boosting SW

crypto performance, I have already provided a simple perf test

Quoted 1 line

inside the unit test in the patchset for the user to try out - just comparing its

output against DPDK crypto perf app output.

Quoted 1 line

5. placing CPU synchronous crypto API in cryptodev will never serve HW

lookaside crypto PMDs, as making them to work synchronously

Quoted 1 line

have huge performance penalty. However Cryptodev framework's existing

design is providing APIs that will work in all crypto PMDs

Quoted 1 line

(rte_cryptodev_enqueue_burst / dequeue_burst for example), this does not fit

in cryptodev's principle.

Quoted 3 lines

6. placing CPU synchronous crypto API in cryptodev confuses the user, as:
- the session created for async mode may not work in sync mode
- both enqueue/dequeue and cpu_crypto_process does the same crypto

processing, but one PMD may support only one API (set),

Quoted 1 line

the other may support another, and the third PMD supports both. We have to

provide another API to let the user query which one to

Quoted 3 lines

support which.
- two completely different code paths for async/sync mode.
7. You said in the end of the email - placing CPU synchronous crypto API into

rte_security is not acceptable as it does not do any

Quoted 1 line

rte_security stuff - crypto isn't? You may call this a quibble, but in my idea, in

the patchset both PMDs' implementations did offload the work

Quoted 1 line

to the CPU's special circuit designed dedicated to accelerate the crypto

processing.

Quoted 2 lines

To me cryptodev is the one CPU synchronous crypto API should not go into,

Quoted 8 lines

rte_security is.

I also don't understand why rte_security is not an option here. We do have inline-crypto right now, why we can't have cpu-crypto with new process() API here?
Actually would like to hear more opinions from the community here - what other interested parties think is the best way for introducing cpu-crypto specific API?

Quoted 2 lines

I have raised this concern in the weekly status meeting as well. But it looks like nobody is interested.

That's really a pity...
CC-ing it to TB members, hopefully someone would be interested, or at least can forward to interested person. Konstantin

#30 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

In reply to Zhang, Roy Fan

Hi Fan,

Quoted 9 lines

Hi Akhil,

Thanks for the review and comments!
Knowing you are extremely busy. Here is my point in brief:
I think placing the CPU synchronous crypto in the rte_security make sense, as

1. rte_security contains inline crypto and lookaside crypto action type already, adding cpu_crypto action type is reasonable.

The argument here is not about cpu-crypto, any SW PMD is nothing but a cpu-crypto. Hence cryptodev already support that.
Here we are concerned only with synchronous processing for crypto workloads.

Quoted 2 lines

2. rte_security contains the security features may not supported by all devices, such as crypto, ipsec, and PDCP. cpu_crypto follow this category, again crypto.

I do not get the intent of this comment. Looking at your patchset, what I get is, You need a synchronous API for crypto workloads.
If sync processing is required for security payloads, we can add a sync API there as well. I have made that comment before also. We can have sync API in both security and cryptodev.

Quoted 2 lines

3. placing CPU synchronous crypto API in rte_security is natural - as inline mode works synchronously, too. However cryptodev doesn't.

It is a valid use case for all the cryptodev SW PMDs
that there should be a synchronous API for crypto processing and that is what your usecase need.

Quoted 4 lines

4. placing CPU synchronous crypto API in rte_security helps boosting SW crypto performance, I have already provided a simple perf test inside the unit test in the patchset for the user to try out - just comparing its output against DPDK crypto perf app output.

I don't expect any performance difference while moving this from security to cryptodev. Have you done any profiling?

Quoted 5 lines

5. placing CPU synchronous crypto API in cryptodev will never serve HW lookaside crypto PMDs, as making them to work synchronously have huge performance penalty. However Cryptodev framework's existing design is providing APIs that will work in all crypto PMDs (rte_cryptodev_enqueue_burst / dequeue_burst for example), this does not fit in cryptodev's principle.

Agreed that it is not for HW PMDs, however the op may be null in those cases. Why it is against the cryptodev principle? There are some ops which PMDs may or may not support.

Quoted 2 lines

6. placing CPU synchronous crypto API in cryptodev confuses the user, as:
- the session created for async mode may not work in sync mode

Why? The whole idea for my conversations on this patchset talks about that. Same session should work for both sync and async processing.

Quoted 4 lines

- both enqueue/dequeue and cpu_crypto_process does the same crypto
processing, but one PMD may support only one API (set), the other may support another, and the third PMD supports both. We have to provide another API to let the user query which one to support which.

This should be based on a Feature flag. It would be upto the application developer To decide which (sync/async) processing is required for which type of flows that it is configuring.

Quoted 6 lines

- two completely different code paths for async/sync mode.
7. You said in the end of the email - placing CPU synchronous crypto API into rte_security is not acceptable as it does not do any rte_security stuff - crypto isn't? You may call this a quibble, but in my idea, in the patchset both PMDs' implementations did offload the work to the CPU's special circuit designed dedicated to accelerate the crypto processing.

This is specific to Intel SW PMDs only. IMO, if you are talking about SW PMDs, openssl can also benefit from this.

Quoted 7 lines

To me cryptodev is the one CPU synchronous crypto API should not go into, rte_security is.

Regards,
Fan

Regards,
Akhil

#31 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

In reply to Akhil Goyal

Hi Akhil,

Quoted 24 lines
Quoted 16 lines
Quoted 1 line

User can use the same session, that is what I am also insisting, but it may have

separate

Quoted 1 line

Session private data. Cryptodev session create API provide that functionality

and we can

Quoted 1 line

Leverage that.

rte_cryptodev_sym_session. sess_data[] is indexed by driver_id, which means we can't use
the same rte_cryptodev_sym_session to hold sessions for both sync and async mode
for the same device. Off course we can add a hard requirement that any driver that wants to
support process() has to create sessions that can handle both process and enqueue/dequeue,
but then again what for to create such overhead?

Quoted 8 lines

BTW, to be honest, I don't consider current rte_cryptodev_sym_session construct for multiple device_ids:

__extension__ struct {
                void *data;
                uint16_t refcnt;
        } sess_data[0];
        /**< Driver specific session material, variable size */
Quoted 2 lines

Yes I also feel the same. I was also not in favor of this when it was introduced. Please go ahead and remove this. I have no issues with that.

If you are not happy with that structure, and admit there are issues with it, why do you push for reusing it for cpu-crypto API?
Why not to take step back, take into account current drawbacks and define something that (hopefully) would suite us better? Again new API will be experimental for some time, so we'll
have some opportunity to see does it works and if not fix it.

About removing data[] from existing rte_cryptodev_sym_session - Personally would like to do that, but the change seems to be too massive. Definitely not ready for such effort right now.

Quoted 36 lines
Quoted 8 lines

as an advantage.
It looks too error prone for me:
1. Simultaneous session initialization/de-initialization for devices with the same driver_id is not possible.
2. It assumes that all device driver will be loaded before we start to create session pools.

Quoted 5 lines

Right now it seems ok, as no-one requires such functionality, but I don't know how it will be in future.
For me rte_security session model, where for each security context user have to create new session
looks much more robust.

Agreed

Quoted 12 lines
Quoted 3 lines

BTW, I can see a v2 to this RFC which is still based on security library.

Yes, v2 was concentrated on fixing found issues, some code restructuring, i.e. - changes that would be needed anyway whatever API aproach we'll choose.

Quoted 2 lines

When do you plan
To submit the patches for crypto based APIs. We have RC1 merge deadline for

this

Quoted 1 line

patchset on 21st Oct.

Quoted 8 lines

We'd like to start working on it ASAP, but it seems we still have a major disagreement
about how this crypto-dev API should look like.
Which makes me think - should we return to our original proposal via rte_security?
It still looks to me like clean and straightforward way to enable this new API, and probably wouldn't cause that much controversy.
What do you think?

Quoted 7 lines

I cannot spend more time discussing on this until RC1 date. I have some other stuff pending. You can send the patches early next week with the approach that I mentioned or else we can discuss this post RC1(which would mean deferring to 20.02).

But moving back to security is not acceptable to me. The code should be put where it is intended and not where it is easy to put. You are not doing any rte_security stuff.

Ok, then my suggestion:
Let's at least write down all points about crypto-dev approach where we disagree and then probably try to resolve them one by one.... If we fail to make an agreement/progress in next week or so, (and no more reviews from the community)
will have bring that subject to TB meeting to decide.
Sounds fair to you?

List is below.
Please add/correct me, if I missed something.

Konstantin

1. extra input parameters to create/init rte_(cpu)_sym_session.

Will leverage existing 6B gap inside rte_crypto_*_xform between 'algo' and 'key' fields. New fields will be optional and would be used by PMD only when cpu-crypto session is requested. For lksd-crypto session PMD is free to ignore these fields.
No ABI breakage is required.

Hopefully no controversy here with #1.

2. cpu-crypto create/init.
a) Our suggestion - introduce new API for that:
- rte_crypto_cpu_sym_init() that would init completely opaque rte_crypto_cpu_sym_session.
- struct rte_crypto_cpu_sym_session_ops {(*process)(...); (*clear); /*whatever else we'll need *'};
- rte_crypto_cpu_sym_get_ops(const struct rte_crypto_sym_xform *xforms)
that would return const struct rte_crypto_cpu_sym_session_ops *based on input xforms.
Advantages:
1) totally opaque data structure (no ABI breakages in future), PMD writer is totally free
with it format and contents.
2) each session entity is self-contained, user doesn't need to bring along dev_id etc.
dev_id is needed only at init stage, after that user will use session ops to perform
all operations on that session (process(), clear(), etc.).
3) User can decide does he wants to store ops[] pointer on a per session basis,
or on a per group of same sessions, or...
4) No mandatory mempools for private sessions. User can allocate memory for cpu-crypto
session whenever he likes.
Disadvantages:
5) Extra changes in control path
6) User has to store session_ops pointer explicitly.
b) Your suggestion - reuse existing rte_cryptodev_sym_session_init() and existing rte_cryptodev_sym_session structure.
Advantages:
1) allows to reuse same struct and init/create/clear() functions.
Probably less changes in control path.
Disadvantages:
2) rte_cryptodev_sym_session. sess_data[] is indexed by driver_id, which means that
we can't use the same rte_cryptodev_sym_session to hold private sessions pointers
for both sync and async mode for the same device.
So wthe only option we have - make PMD devops->sym_session_configure()
always create a session that can work in both cpu and lksd modes.
For some implementations that would probably mean that under the hood PMD would create
2 different session structs (sync/async) and then use one or another depending on from what API been called. Seems doable, but ...:
- will contradict with statement from 1:
" New fields will be optional and would be used by PMD only when cpu-crypto session is requested."
Now it becomes mandatory for all apps to specify cpu-crypto related parameters too,
even if they don't plan to use that mode - i.e. behavior change, existing app change.
- might cause extra space overhead.
3) not possible to store device (not driver) specific data within the session, but I think it is not really needed right now. So probably minor compared to 2.b.2.

Actually #3 follows from #2, but decided to have them separated.

3. process() parameters/behavior
a) Our suggestion: user stores ptr to session ops (or to (*process) itself) and just does:
session_ops->process(sess, ...);
Advantages:
1) fastest possible execution path
2) no need to carry on dev_id for data-path
Disadvantages:
3) user has to carry on session_ops pointer explicitly
b) Your suggestion: add (*cpu_process) inside rte_cryptodev_ops and then:
rte_crypto_cpu_sym_process(uint8_t dev_id, rte_cryptodev_sym_session *sess, /*data parameters*/) {...
rte_cryptodevs[dev_id].dev_ops->cpu_process(ses, ...);
/*and then inside PMD specifc process: */
pmd_private_session = sess->sess_data[this_pmd_driver_id].data;
/* and then most likely either */ pmd_private_session->process(pmd_private_session, ...);
/* or jump based on session/input data */
Advantages:
1) don't see any...
Disadvantages:
2) User has to carry on dev_id inside data-path
3) Extra level of indirection (plus data dependency) - both for data and instructions. Possible slowdown compared to a) (not measured).

#32 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Quoted 117 lines
Quoted 26 lines
Quoted 18 lines
Quoted 3 lines

User can use the same session, that is what I am also insisting, but it may have

separate

Quoted 1 line

Session private data. Cryptodev session create API provide that functionality

and we can

Quoted 1 line

Leverage that.

rte_cryptodev_sym_session. sess_data[] is indexed by driver_id, which means we can't use
the same rte_cryptodev_sym_session to hold sessions for both sync and async mode
for the same device. Off course we can add a hard requirement that any driver that wants to
support process() has to create sessions that can handle both process and enqueue/dequeue,
but then again what for to create such overhead?

Quoted 8 lines

BTW, to be honest, I don't consider current rte_cryptodev_sym_session construct for multiple device_ids:

__extension__ struct {
                void *data;
                uint16_t refcnt;
        } sess_data[0];
        /**< Driver specific session material, variable size */
Quoted 2 lines

Yes I also feel the same. I was also not in favor of this when it was introduced. Please go ahead and remove this. I have no issues with that.

If you are not happy with that structure, and admit there are issues with it, why do you push for reusing it for cpu-crypto API?
Why not to take step back, take into account current drawbacks and define something that (hopefully) would suite us better? Again new API will be experimental for some time, so we'll
have some opportunity to see does it works and if not fix it.

About removing data[] from existing rte_cryptodev_sym_session - Personally would like to do that, but the change seems to be too massive. Definitely not ready for such effort right now.

Quoted 36 lines
Quoted 8 lines

as an advantage.
It looks too error prone for me:
1. Simultaneous session initialization/de-initialization for devices with the same driver_id is not possible.
2. It assumes that all device driver will be loaded before we start to create session pools.

Quoted 5 lines

Right now it seems ok, as no-one requires such functionality, but I don't know how it will be in future.
For me rte_security session model, where for each security context user have to create new session
looks much more robust.

Agreed

Quoted 12 lines
Quoted 3 lines

BTW, I can see a v2 to this RFC which is still based on security library.

Yes, v2 was concentrated on fixing found issues, some code restructuring, i.e. - changes that would be needed anyway whatever API aproach we'll choose.

Quoted 2 lines

When do you plan
To submit the patches for crypto based APIs. We have RC1 merge deadline for

this

Quoted 1 line

patchset on 21st Oct.

Quoted 8 lines

We'd like to start working on it ASAP, but it seems we still have a major disagreement
about how this crypto-dev API should look like.
Which makes me think - should we return to our original proposal via rte_security?
It still looks to me like clean and straightforward way to enable this new API, and probably wouldn't cause that much controversy.
What do you think?

Quoted 7 lines

I cannot spend more time discussing on this until RC1 date. I have some other stuff pending. You can send the patches early next week with the approach that I mentioned or else we can discuss this post RC1(which would mean deferring to 20.02).

But moving back to security is not acceptable to me. The code should be put where it is intended and not where it is easy to put. You are not doing any rte_security stuff.

Ok, then my suggestion:
Let's at least write down all points about crypto-dev approach where we disagree and then probably try to resolve them one by one.... If we fail to make an agreement/progress in next week or so, (and no more reviews from the community)
will have bring that subject to TB meeting to decide.
Sounds fair to you?

List is below.
Please add/correct me, if I missed something.

Konstantin

1. extra input parameters to create/init rte_(cpu)_sym_session.

Will leverage existing 6B gap inside rte_crypto_*_xform between 'algo' and 'key' fields. New fields will be optional and would be used by PMD only when cpu-crypto session is requested. For lksd-crypto session PMD is free to ignore these fields.
No ABI breakage is required.

Hopefully no controversy here with #1.

2. cpu-crypto create/init.
a) Our suggestion - introduce new API for that:
- rte_crypto_cpu_sym_init() that would init completely opaque rte_crypto_cpu_sym_session.
- struct rte_crypto_cpu_sym_session_ops {(*process)(...); (*clear); /*whatever else we'll need *'};
- rte_crypto_cpu_sym_get_ops(const struct rte_crypto_sym_xform *xforms)
that would return const struct rte_crypto_cpu_sym_session_ops *based on input xforms.
Advantages:
1) totally opaque data structure (no ABI breakages in future), PMD writer is totally free with it format and contents.
2) each session entity is self-contained, user doesn't need to bring along dev_id etc. dev_id is needed only at init stage, after that user will use session ops to perform

Quoted 8 lines

all operations on that session (process(), clear(), etc.).
3) User can decide does he wants to store ops[] pointer on a per session basis, or on a per group of same sessions, or...
4) No mandatory mempools for private sessions. User can allocate memory for cpu-crypto session whenever he likes.
Disadvantages:
5) Extra changes in control path
6) User has to store session_ops pointer explicitly.

After another thought if 2.a.6 is really that big deal we can have small shim layer on top:

rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops * const ops; } OR even
rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops ops; }

And merge rte_crypto_cpu_sym_init() and rte_crypto_cpu_sym_get_ops() into one (init).

Then process() can become a wrapper:
rte_crypto_cpu_sym_process(ses, ...) {return ses->ops->process(ses->ses, ...);} OR
rte_crypto_cpu_sym_process(ses, ...) {return ses->ops.process(ses->ses, ...);}

if that would help to reach consensus - works for me.

Quoted 47 lines

b) Your suggestion - reuse existing rte_cryptodev_sym_session_init() and existing rte_cryptodev_sym_session structure.
Advantages:
1) allows to reuse same struct and init/create/clear() functions.
Probably less changes in control path.
Disadvantages:
2) rte_cryptodev_sym_session. sess_data[] is indexed by driver_id, which means that
we can't use the same rte_cryptodev_sym_session to hold private sessions pointers
for both sync and async mode for the same device.
So wthe only option we have - make PMD devops->sym_session_configure()
always create a session that can work in both cpu and lksd modes.
For some implementations that would probably mean that under the hood PMD would create
2 different session structs (sync/async) and then use one or another depending on from what API been called. Seems doable, but ...:
- will contradict with statement from 1:
" New fields will be optional and would be used by PMD only when cpu-crypto session is requested."
Now it becomes mandatory for all apps to specify cpu-crypto related parameters too,
even if they don't plan to use that mode - i.e. behavior change, existing app change.
- might cause extra space overhead.
3) not possible to store device (not driver) specific data within the session, but I think it is not really needed right now. So probably minor compared to 2.b.2.

Actually #3 follows from #2, but decided to have them separated.

3. process() parameters/behavior
a) Our suggestion: user stores ptr to session ops (or to (*process) itself) and just does:
session_ops->process(sess, ...);
Advantages:
1) fastest possible execution path
2) no need to carry on dev_id for data-path
Disadvantages:
3) user has to carry on session_ops pointer explicitly
b) Your suggestion: add (*cpu_process) inside rte_cryptodev_ops and then:
rte_crypto_cpu_sym_process(uint8_t dev_id, rte_cryptodev_sym_session *sess, /*data parameters*/) {...
rte_cryptodevs[dev_id].dev_ops->cpu_process(ses, ...);
/*and then inside PMD specifc process: */
pmd_private_session = sess->sess_data[this_pmd_driver_id].data;
/* and then most likely either */ pmd_private_session->process(pmd_private_session, ...);
/* or jump based on session/input data */
Advantages:
1) don't see any...
Disadvantages:
2) User has to carry on dev_id inside data-path
3) Extra level of indirection (plus data dependency) - both for data and instructions. Possible slowdown compared to a) (not measured).

#33 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

In reply to Ananyev, Konstantin

Hi Konstantin,

Added my comments inline with your draft.

Quoted 15 lines

Hi Akhil,

Quoted 10 lines
Quoted 8 lines

BTW, to be honest, I don't consider current rte_cryptodev_sym_session construct for multiple device_ids:

__extension__ struct {
                void *data;
                uint16_t refcnt;
        } sess_data[0];
        /**< Driver specific session material, variable size */

Yes I also feel the same. I was also not in favor of this when it was introduced. Please go ahead and remove this. I have no issues with that.

Quoted 6 lines

If you are not happy with that structure, and admit there are issues with it, why do you push for reusing it for cpu-crypto API?
Why not to take step back, take into account current drawbacks and define something that (hopefully) would suite us better? Again new API will be experimental for some time, so we'll
have some opportunity to see does it works and if not fix it.

[Akhil] This structure is serving some use case which is agreed upon in the Community, we cannot just remove a feature altogether. Rather it is Intel's Use case only.

Quoted 5 lines

About removing data[] from existing rte_cryptodev_sym_session - Personally would like to do that, but the change seems to be too massive. Definitely not ready for such effort right now.

[snip]..

Quoted 8 lines

Ok, then my suggestion:
Let's at least write down all points about crypto-dev approach where we disagree and then probably try to resolve them one by one.... If we fail to make an agreement/progress in next week or so, (and no more reviews from the community)
will have bring that subject to TB meeting to decide.
Sounds fair to you?

Agreed

Quoted 5 lines

List is below.
Please add/correct me, if I missed something.

Konstantin

Before going into comparison, we should define the requirement as well. What I understood from the patchset,
"You need a synchronous API to perform crypto operations on raw data using SW PMDs" So,
- no crypto-ops,
- no separate enq-deq, only single process API for data path
- Do not need any value addition to the session parameters.
(You would need some parameters from the crypto-op which
Are constant per session and since you wont use crypto-op,
You need some place to store that)

Now as per your mail, the comparison
1. extra input parameters to create/init rte_(cpu)_sym_session.

Will leverage existing 6B gap inside rte_crypto_*_xform between 'algo' and 'key' fields. New fields will be optional and would be used by PMD only when cpu-crypto session is requested. For lksd-crypto session PMD is free to ignore these fields.
No ABI breakage is required.

[Akhil] Agreed, no issues.

2. cpu-crypto create/init.
a) Our suggestion - introduce new API for that:
- rte_crypto_cpu_sym_init() that would init completely opaque rte_crypto_cpu_sym_session.
- struct rte_crypto_cpu_sym_session_ops {(*process)(...); (*clear); /*whatever else we'll need *'};
- rte_crypto_cpu_sym_get_ops(const struct rte_crypto_sym_xform *xforms)
that would return const struct rte_crypto_cpu_sym_session_ops *based on input xforms.
Advantages:
1) totally opaque data structure (no ABI breakages in future), PMD writer is totally free with it format and contents.

[Akhil] It will have breakage at some point till we don't hit the union size. Rather I don't suspect there will be more parameters added.
Or do we really care about the ABI breakage when the argument is about the correct place to add a piece of code or do we really agree to add code anywhere just to avoid that breakage.

2) each session entity is self-contained, user doesn't need to bring along dev_id etc. dev_id is needed only at init stage, after that user will use session ops to perform all operations on that session (process(), clear(), etc.).

[Akhil] There is nothing called as session ops in current DPDK. What you are proposing is a new concept which doesn't have any extra benefit, rather it is adding complexity to have two different code paths for session create.

3) User can decide does he wants to store ops[] pointer on a per session basis, or on a per group of same sessions, or...

[Akhil] Will the user really care which process API should be called from the PMD. Rather it should be driver's responsibility to store that in the session private data which would be opaque to the user. As per my suggestion same process function can be added to multiple sessions or a single session can be managed inside the PMD.

4) No mandatory mempools for private sessions. User can allocate memory for cpu-crypto session whenever he likes.

[Akhil] you mean session private data? You would need that memory anyways, user will be allocating that already. You do not need to manage that.

Disadvantages:
5) Extra changes in control path
6) User has to store session_ops pointer explicitly.

[Akhil] More disadvantages:
- All supporting PMDs will need to maintain TWO types of session for the same crypto processing. Suppose a fix or a new feature(or algo) is added, PMD owner will need to add code in both the session create APIs. Hence more maintenance and error prone.
- Stacks which will be using these new APIs also need to maintain two code path for the same processing while doing session initialization for sync and async

b) Your suggestion - reuse existing rte_cryptodev_sym_session_init() and existing rte_cryptodev_sym_session structure.
Advantages:
1) allows to reuse same struct and init/create/clear() functions.
Probably less changes in control path.
Disadvantages:
2) rte_cryptodev_sym_session. sess_data[] is indexed by driver_id, which means that
we can't use the same rte_cryptodev_sym_session to hold private sessions pointers
for both sync and async mode for the same device.
So the only option we have - make PMD devops->sym_session_configure()
always create a session that can work in both cpu and lksd modes.
For some implementations that would probably mean that under the hood PMD would create 2 different session structs (sync/async) and then use one or another depending on from what API been called. Seems doable, but ...:
- will contradict with statement from 1:
" New fields will be optional and would be used by PMD only when cpu-crypto session is requested."
Now it becomes mandatory for all apps to specify cpu-crypto related parameters too,
even if they don't plan to use that mode - i.e. behavior change, existing app change.
- might cause extra space overhead.

[Akhil] It will not contradict with #1, you will only have few checks in the session init PMD Which support this mode, find appropriate values and set the appropriate process() in it. User should be able to call, legacy enq-deq as well as the new process() without any issue. User would be at runtime will be able to change the datapath.
So this is not a disadvantage, it would be additional flexibility for the user.

3) not possible to store device (not driver) specific data within the session, but I think it is not really needed right now. So probably minor compared to 2.b.2.

[Akhil] So lets omit this for current discussion. And I hope we can find some way to deal with it.

Actually #3 follows from #2, but decided to have them separated.

3. process() parameters/behavior
a) Our suggestion: user stores ptr to session ops (or to (*process) itself) and just does:
session_ops->process(sess, ...);
Advantages:
1) fastest possible execution path
2) no need to carry on dev_id for data-path

[Akhil] I don't see any overhead of carrying dev id, at least it would be inline with the current DPDK methodology.
What you are suggesting is a new way to get the things done without much benefit. Also I don't see any performance difference as crypto workload is heavier than Code cycles, so that wont matter.
So IMO, there is no advantage in your suggestion as well.

Disadvantages:
3) user has to carry on session_ops pointer explicitly
b) Your suggestion: add (*cpu_process) inside rte_cryptodev_ops and then:
rte_crypto_cpu_sym_process(uint8_t dev_id, rte_cryptodev_sym_session *sess, /*data parameters*/) {...
rte_cryptodevs[dev_id].dev_ops->cpu_process(ses, ...);
/*and then inside PMD specifc process: */
pmd_private_session = sess->sess_data[this_pmd_driver_id].data;
/* and then most likely either */ pmd_private_session->process(pmd_private_session, ...);
/* or jump based on session/input data */
Advantages:
1) don't see any...
Disadvantages:
2) User has to carry on dev_id inside data-path
3) Extra level of indirection (plus data dependency) - both for data and instructions. Possible slowdown compared to a) (not measured).

Having said all this, if the disagreements cannot be resolved, you can go for a pmd API specific to your PMDs, because as per my understanding the solution doesn't look scalable to other PMDs. Your approach is aligned only to Intel, will not benefit others like openssl which is used by all vendors.

Regards,
Akhil

#34 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Quoted 23 lines

Added my comments inline with your draft.

Quoted 15 lines

Hi Akhil,

Quoted 10 lines
Quoted 8 lines

BTW, to be honest, I don't consider current rte_cryptodev_sym_session construct for multiple device_ids:

__extension__ struct {
                void *data;
                uint16_t refcnt;
        } sess_data[0];
        /**< Driver specific session material, variable size */

Yes I also feel the same. I was also not in favor of this when it was introduced. Please go ahead and remove this. I have no issues with that.

Quoted 6 lines

If you are not happy with that structure, and admit there are issues with it, why do you push for reusing it for cpu-crypto API?
Why not to take step back, take into account current drawbacks and define something that (hopefully) would suite us better? Again new API will be experimental for some time, so we'll
have some opportunity to see does it works and if not fix it.

Quoted 2 lines

[Akhil] This structure is serving some use case which is agreed upon in the Community, we cannot just remove a feature altogether.

I understand that, but we don't suggest to remove anything that already here. We are talking about extending existing/adding new API.
All our debates around how much we can reuse from existing one and what new needs to be added.

Quoted 18 lines

Rather it is Intel's Use case only.

Quoted 5 lines

About removing data[] from existing rte_cryptodev_sym_session - Personally would like to do that, but the change seems to be too massive. Definitely not ready for such effort right now.

[snip]..

Quoted 8 lines

Ok, then my suggestion:
Let's at least write down all points about crypto-dev approach where we disagree and then probably try to resolve them one by one.... If we fail to make an agreement/progress in next week or so, (and no more reviews from the community)
will have bring that subject to TB meeting to decide.
Sounds fair to you?

Quoted 8 lines

Agreed

Quoted 5 lines

List is below.
Please add/correct me, if I missed something.

Konstantin

Before going into comparison, we should define the requirement as well.

Good point.

Quoted 9 lines

What I understood from the patchset,
"You need a synchronous API to perform crypto operations on raw data using SW PMDs" So,
- no crypto-ops,
- no separate enq-deq, only single process API for data path
- Do not need any value addition to the session parameters.
(You would need some parameters from the crypto-op which
Are constant per session and since you wont use crypto-op,
You need some place to store that)

Yes, this is correct, I think.

Quoted 21 lines

Now as per your mail, the comparison
1. extra input parameters to create/init rte_(cpu)_sym_session.

Will leverage existing 6B gap inside rte_crypto_*_xform between 'algo' and 'key' fields. New fields will be optional and would be used by PMD only when cpu-crypto session is requested. For lksd-crypto session PMD is free to ignore these fields.
No ABI breakage is required.

[Akhil] Agreed, no issues.

2. cpu-crypto create/init.
a) Our suggestion - introduce new API for that:
- rte_crypto_cpu_sym_init() that would init completely opaque rte_crypto_cpu_sym_session.
- struct rte_crypto_cpu_sym_session_ops {(*process)(...); (*clear); /*whatever else we'll need *'};
- rte_crypto_cpu_sym_get_ops(const struct rte_crypto_sym_xform *xforms)
that would return const struct rte_crypto_cpu_sym_session_ops *based on input xforms.
Advantages:
1) totally opaque data structure (no ABI breakages in future), PMD writer is totally free with it format and contents.

Quoted 1 line

[Akhil] It will have breakage at some point till we don't hit the union size.

Not sure, what union you are talking about?

Quoted 4 lines

Rather I don't suspect there will be more parameters added.
Or do we really care about the ABI breakage when the argument is about the correct place to add a piece of code or do we really agree to add code anywhere just to avoid that breakage.

I am talking about maintaining it in future.
if your struct is not seen externally, no chances to introduce ABI breakage.

Quoted 6 lines

2) each session entity is self-contained, user doesn't need to bring along dev_id etc. dev_id is needed only at init stage, after that user will use session ops to perform all operations on that session (process(), clear(), etc.).

[Akhil] There is nothing called as session ops in current DPDK.

True, but it doesn't mean we can't/shouldn't have it.

Quoted 5 lines

What you are proposing
is a new concept which doesn't have any extra benefit, rather it is adding complexity to have two different code paths for session create.

Quoted 7 lines

3) User can decide does he wants to store ops[] pointer on a per session basis, or on a per group of same sessions, or...

[Akhil] Will the user really care which process API should be called from the PMD. Rather it should be driver's responsibility to store that in the session private data which would be opaque to the user. As per my suggestion same process function can be added to multiple sessions or a single session can be managed inside the PMD.

In that case we either need to have a function per session (stored internally), or make decision (branches) at run-time.
But as I said in other mail - I am ok to add small shim structure here:
either rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops ops; } or rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops *ops; } And merge rte_crypto_cpu_sym_init() and rte_crypto_cpu_sym_get_ops() into one (init).

Quoted 6 lines

4) No mandatory mempools for private sessions. User can allocate memory for cpu-crypto session whenever he likes.

[Akhil] you mean session private data?

Yes.

Quoted 2 lines

You would need that memory anyways, user will be allocating that already. You do not need to manage that.

What I am saying - right now user has no choice but to allocate it via mempool. Which is probably not the best options for all cases.

Quoted 10 lines

Disadvantages:
5) Extra changes in control path
6) User has to store session_ops pointer explicitly.

[Akhil] More disadvantages:
- All supporting PMDs will need to maintain TWO types of session for the same crypto processing. Suppose a fix or a new feature(or algo) is added, PMD owner will need to add code in both the session create APIs. Hence more maintenance and error prone.

I think majority of code for both paths will be common, plus even we'll reuse current sym_session_init() - changes in PMD session_init() code will be unavoidable.
But yes, it will be new entry in devops, that PMD will have to support.
Ok to add it as 7) to the list.

Quoted 3 lines

- Stacks which will be using these new APIs also need to maintain two code path for the same processing while doing session initialization for sync and async

That's the same as #5 above, I think.

Quoted 22 lines

b) Your suggestion - reuse existing rte_cryptodev_sym_session_init() and existing rte_cryptodev_sym_session structure.
Advantages:
1) allows to reuse same struct and init/create/clear() functions.
Probably less changes in control path.
Disadvantages:
2) rte_cryptodev_sym_session. sess_data[] is indexed by driver_id, which means that
we can't use the same rte_cryptodev_sym_session to hold private sessions pointers
for both sync and async mode for the same device.
So the only option we have - make PMD devops->sym_session_configure()
always create a session that can work in both cpu and lksd modes.
For some implementations that would probably mean that under the hood PMD would create 2 different session structs (sync/async) and then use one or another depending on from what API been called. Seems doable, but ...:
- will contradict with statement from 1:
" New fields will be optional and would be used by PMD only when cpu-crypto session is requested."
Now it becomes mandatory for all apps to specify cpu-crypto related parameters too,
even if they don't plan to use that mode - i.e. behavior change, existing app change.
- might cause extra space overhead.

Quoted 5 lines

[Akhil] It will not contradict with #1, you will only have few checks in the session init PMD Which support this mode, find appropriate values and set the appropriate process() in it. User should be able to call, legacy enq-deq as well as the new process() without any issue. User would be at runtime will be able to change the datapath.
So this is not a disadvantage, it would be additional flexibility for the user.

Ok, but that's what I am saying - if PMD would *always* have to create a session that can handle both modes (sync/async), then user would *always* have to provide parameters for both modes too. Otherwise if let say user didn't setup sync specific parameters at all, what PMD should do?
- return with error?
- init session that can be used with async path only?
My current assumption is #1.
If #2, then how user will be able to distinguish is that session valid for both modes, or only for one?

Quoted 6 lines

3) not possible to store device (not driver) specific data within the session, but I think it is not really needed right now. So probably minor compared to 2.b.2.

[Akhil] So lets omit this for current discussion. And I hope we can find some way to deal with it.

I don't think there is an easy way to fix that with existing API.

Quoted 11 lines

Actually #3 follows from #2, but decided to have them separated.

3. process() parameters/behavior
a) Our suggestion: user stores ptr to session ops (or to (*process) itself) and just does:
session_ops->process(sess, ...);
Advantages:
1) fastest possible execution path
2) no need to carry on dev_id for data-path

Quoted 2 lines

[Akhil] I don't see any overhead of carrying dev id, at least it would be inline with the current DPDK methodology.

If we'll add process() into rte_cryptodev itself (same as we have enqueue_burst/dequeue_burst), then it will be an ABI breakage.
Also there are discussions to get rid of that approach completely:
http://mails.dpdk.org/archives/dev/2019-September/144674.html
So I am not sure this is a recommended way these days.

Quoted 1 line

What you are suggesting is a new way to get the things done without much benefit.

Would help with ABI stability plus better performance, isn't it enough?

Quoted 2 lines

Also I don't see any performance difference as crypto workload is heavier than Code cycles, so that wont matter.

It depends.
Suppose function call costs you ~30 cycles.
If you have burst of big packets (let say crypto for each will take ~2K cycles) that belong to the same session, then yes you wouldn't notice these extra 30 cycles at all. If you have burst of small packets (let say crypto for each will take ~300 cycles) each belongs to different session, then it will cost you ~10% extra.

Quoted 20 lines

So IMO, there is no advantage in your suggestion as well.

Disadvantages:
3) user has to carry on session_ops pointer explicitly
b) Your suggestion: add (*cpu_process) inside rte_cryptodev_ops and then:
rte_crypto_cpu_sym_process(uint8_t dev_id, rte_cryptodev_sym_session *sess, /*data parameters*/) {...
rte_cryptodevs[dev_id].dev_ops->cpu_process(ses, ...);
/*and then inside PMD specifc process: */
pmd_private_session = sess->sess_data[this_pmd_driver_id].data;
/* and then most likely either */ pmd_private_session->process(pmd_private_session, ...);
/* or jump based on session/input data */
Advantages:
1) don't see any...
Disadvantages:
2) User has to carry on dev_id inside data-path
3) Extra level of indirection (plus data dependency) - both for data and instructions. Possible slowdown compared to a) (not measured).

Quoted 2 lines

Having said all this, if the disagreements cannot be resolved, you can go for a pmd API specific to your PMDs,

I don't think it is good idea.
PMD specific API is sort of deprecated path, also there is no clean way to use it within the libraries.

Quoted 3 lines

because as per my understanding the solution doesn't look scalable to other PMDs. Your approach is aligned only to Intel , will not benefit others like openssl which is used by all vendors.

I feel quite opposite, from my perspective majority of SW backed PMDs will benefit from it. And I don't see anything Intel specific in my proposals above.
About openssl PMD: I am not an expert here, but looking at the code, I think it will fit really well. Look yourself at its internal functions: process_openssl_auth_op/process_openssl_crypto_op, I think they doing exactly the same - they use sync API underneath, and they are session based (AFAIK you don't need any device/queue data, everything that needed for crypto/auth is stored inside session).

Konstantin

#35 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 63 lines

Hi Akhil,

Quoted 11 lines

Added my comments inline with your draft.
[snip]..

Quoted 8 lines

Ok, then my suggestion:
Let's at least write down all points about crypto-dev approach where we disagree and then probably try to resolve them one by one.... If we fail to make an agreement/progress in next week or so, (and no more reviews from the community)
will have bring that subject to TB meeting to decide.
Sounds fair to you?

Quoted 8 lines

Agreed

Quoted 5 lines

List is below.
Please add/correct me, if I missed something.

Konstantin

Before going into comparison, we should define the requirement as well.

Good point.

Quoted 2 lines

What I understood from the patchset,
"You need a synchronous API to perform crypto operations on raw data using

SW PMDs"

Quoted 7 lines

So,
- no crypto-ops,
- no separate enq-deq, only single process API for data path
- Do not need any value addition to the session parameters. (You would need some parameters from the crypto-op which Are constant per session and since you wont use crypto-op, You need some place to store that)

Yes, this is correct, I think.

Quoted 5 lines

Now as per your mail, the comparison
1. extra input parameters to create/init rte_(cpu)_sym_session.

Will leverage existing 6B gap inside rte_crypto_*_xform between 'algo' and

'key' fields.

Quoted 1 line

New fields will be optional and would be used by PMD only when cpu-crypto

session is requested.

Quoted 8 lines

For lksd-crypto session PMD is free to ignore these fields. No ABI breakage is required.

[Akhil] Agreed, no issues.

2. cpu-crypto create/init.
a) Our suggestion - introduce new API for that:
- rte_crypto_cpu_sym_init() that would init completely opaque

rte_crypto_cpu_sym_session.

Quoted 1 line

- struct rte_crypto_cpu_sym_session_ops {(*process)(...); (*clear);

/*whatever else we'll need *'};

Quoted 1 line

- rte_crypto_cpu_sym_get_ops(const struct rte_crypto_sym_xform

*xforms)

Quoted 1 line

that would return const struct rte_crypto_cpu_sym_session_ops *based

on input xforms.

Quoted 8 lines
Quoted 2 lines

Advantages:
1) totally opaque data structure (no ABI breakages in future), PMD

writer is totally free

Quoted 3 lines

with it format and contents.

[Akhil] It will have breakage at some point till we don't hit the union size.

Not sure, what union you are talking about?

Union of xforms in rte_security_session_conf

Quoted 11 lines
Quoted 5 lines

Rather I don't suspect there will be more parameters added.
Or do we really care about the ABI breakage when the argument is about the correct place to add a piece of code or do we really agree to add code anywhere just to avoid that breakage.

I am talking about maintaining it in future.
if your struct is not seen externally, no chances to introduce ABI breakage.

Quoted 2 lines

2) each session entity is self-contained, user doesn't need to bring along

Quoted 8 lines

dev_id etc.

Quoted 1 line

dev_id is needed only at init stage, after that user will use session ops

to perform

Quoted 3 lines

all operations on that session (process(), clear(), etc.).

[Akhil] There is nothing called as session ops in current DPDK.

True, but it doesn't mean we can't/shouldn't have it.

We can have it if it is not adding complexity for the user. Creating 2 different code Paths for user is not desirable for the stack developers.

Quoted 20 lines
Quoted 3 lines

What you are proposing
is a new concept which doesn't have any extra benefit, rather it is adding

complexity

Quoted 4 lines

to have two different code paths for session create.

3) User can decide does he wants to store ops[] pointer on a per session

basis,

Quoted 3 lines

or on a per group of same sessions, or...

[Akhil] Will the user really care which process API should be called from the

PMD.

Quoted 1 line

Rather it should be driver's responsibility to store that in the session private

data

Quoted 1 line

which would be opaque to the user. As per my suggestion same process

function can

Quoted 1 line

be added to multiple sessions or a single session can be managed inside the

PMD.

Quoted 9 lines

In that case we either need to have a function per session (stored internally), or make decision (branches) at run-time.
But as I said in other mail - I am ok to add small shim structure here:
either rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops ops; }
or rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops *ops; }
And merge rte_crypto_cpu_sym_init() and rte_crypto_cpu_sym_get_ops() into one (init).

Again that will be a separate API call from the user perspective which is not good.

Quoted 82 lines
Quoted 4 lines

4) No mandatory mempools for private sessions. User can allocate

memory for cpu-crypto

Quoted 3 lines

session whenever he likes.

[Akhil] you mean session private data?

Yes.

Quoted 2 lines

You would need that memory anyways, user will be allocating that already. You do not need to manage that.

What I am saying - right now user has no choice but to allocate it via mempool. Which is probably not the best options for all cases.

Quoted 8 lines

Disadvantages:
5) Extra changes in control path
6) User has to store session_ops pointer explicitly.

[Akhil] More disadvantages:
- All supporting PMDs will need to maintain TWO types of session for the same crypto processing. Suppose a fix or a new feature(or algo) is added, PMD

owner

Quoted 1 line

will need to add code in both the session create APIs. Hence more

maintenance and

Quoted 1 line

error prone.

I think majority of code for both paths will be common, plus even we'll reuse current sym_session_init() -
changes in PMD session_init() code will be unavoidable.
But yes, it will be new entry in devops, that PMD will have to support. Ok to add it as 7) to the list.

Quoted 3 lines

- Stacks which will be using these new APIs also need to maintain two code path for the same processing while doing session initialization for sync and async

That's the same as #5 above, I think.

Quoted 3 lines

b) Your suggestion - reuse existing rte_cryptodev_sym_session_init() and

existing rte_cryptodev_sym_session

Quoted 6 lines

structure.
Advantages:
1) allows to reuse same struct and init/create/clear() functions. Probably less changes in control path.
Disadvantages:
2) rte_cryptodev_sym_session. sess_data[] is indexed by driver_id,

which means that

Quoted 1 line

we can't use the same rte_cryptodev_sym_session to hold private

sessions pointers

Quoted 5 lines

for both sync and async mode for the same device.
So the only option we have - make PMD devops-
sym_session_configure()
always create a session that can work in both cpu and lksd modes. For some implementations that would probably mean that under the

hood PMD would create

Quoted 1 line

2 different session structs (sync/async) and then use one or another

depending on from what API been called.

Quoted 3 lines

Seems doable, but ...:
- will contradict with statement from 1:
" New fields will be optional and would be used by PMD only when

cpu-crypto session is requested."

Quoted 1 line

Now it becomes mandatory for all apps to specify cpu-crypto

related parameters too,

Quoted 1 line

even if they don't plan to use that mode - i.e. behavior change,

existing app change.

Quoted 3 lines

- might cause extra space overhead.

[Akhil] It will not contradict with #1, you will only have few checks in the

session init PMD

Quoted 1 line

Which support this mode, find appropriate values and set the appropriate

process() in it.

Quoted 1 line

User should be able to call, legacy enq-deq as well as the new process()

without any issue.

Quoted 2 lines

User would be at runtime will be able to change the datapath.
So this is not a disadvantage, it would be additional flexibility for the user.

Quoted 11 lines

Ok, but that's what I am saying - if PMD would *always* have to create a session that can handle
both modes (sync/async), then user would *always* have to provide parameters for both modes too.
Otherwise if let say user didn't setup sync specific parameters at all, what PMD should do?
- return with error?
- init session that can be used with async path only?
My current assumption is #1.
If #2, then how user will be able to distinguish is that session valid for both modes, or only for one?

I would say a 3rd option, do nothing if sync params are not set. Probably have a debug print in the PMD(which support sync mode) to specify that session is not configured properly for sync mode.
Internally the PMD will not store the process() API in the session priv data And while calling the first packet, devops->process will give an assert that session Is not configured for sync mode. The session validation would be done in any case your suggestion or mine. So no extra overhead at runtime.

Quoted 27 lines
Quoted 5 lines

3) not possible to store device (not driver) specific data within the

session, but I think it is not really needed right now.

Quoted 3 lines

So probably minor compared to 2.b.2.

[Akhil] So lets omit this for current discussion. And I hope we can find some

way to deal with it.

I don't think there is an easy way to fix that with existing API.

Quoted 6 lines

Actually #3 follows from #2, but decided to have them separated.

3. process() parameters/behavior
a) Our suggestion: user stores ptr to session ops (or to (*process) itself) and

just does:

Quoted 6 lines

session_ops->process(sess, ...); Advantages:
1) fastest possible execution path
2) no need to carry on dev_id for data-path

[Akhil] I don't see any overhead of carrying dev id, at least it would be inline

with the

Quoted 8 lines
Quoted 1 line

current DPDK methodology.

If we'll add process() into rte_cryptodev itself (same as we have enqueue_burst/dequeue_burst),
then it will be an ABI breakage.
Also there are discussions to get rid of that approach completely:
http://mails.dpdk.org/archives/dev/2019-September/144674.html
So I am not sure this is a recommended way these days.

We can either have it in rte_cryptodev or in rte_cryptodev_ops whichever is good for you.

Whether it is ABI breakage or not, as per your requirements, this is the correct approach. Do you agree with this or not?

Now handling the API/ABI breakage is a separate story. In 19.11 release we Are not much concerned about the ABI breakages, this was discussed in community. So adding a new dev_ops wouldn't have been an issue. Now since we are so close to RC1 deadline, we should come up with some other solution for next release. May be having a pmd API in 20.02 and converting it into formal one in 20.11

Quoted 9 lines
Quoted 2 lines

What you are suggesting is a new way to get the things done without much

benefit.

Would help with ABI stability plus better performance, isn't it enough?

Quoted 2 lines

Also I don't see any performance difference as crypto workload is heavier than Code cycles, so that wont matter.

Quoted 8 lines

It depends.
Suppose function call costs you ~30 cycles.
If you have burst of big packets (let say crypto for each will take ~2K cycles) that belong
to the same session, then yes you wouldn't notice these extra 30 cycles at all. If you have burst of small packets (let say crypto for each will take ~300 cycles) each
belongs to different session, then it will cost you ~10% extra.

Let us do some profiling on openssl with both the approaches and find out the difference.

Quoted 23 lines
Quoted 8 lines

So IMO, there is no advantage in your suggestion as well.

Disadvantages:
3) user has to carry on session_ops pointer explicitly
b) Your suggestion: add (*cpu_process) inside rte_cryptodev_ops and then:
rte_crypto_cpu_sym_process(uint8_t dev_id, rte_cryptodev_sym_session

*sess, /*data parameters*/) {...

Quoted 11 lines
                     rte_cryptodevs[dev_id].dev_ops->cpu_process(ses, ...);
                      /*and then inside PMD specifc process: */
                     pmd_private_session = sess->sess_data[this_pmd_driver_id].data;
                     /* and then most likely either */
                     pmd_private_session->process(pmd_private_session, ...);
                     /* or jump based on session/input data */
	Advantages:
	1) don't see any...
	Disadvantages:
	2) User has to carry on dev_id inside data-path
	3) Extra level of indirection (plus data dependency) - both for data and

instructions.

Quoted 2 lines

Possible slowdown compared to a) (not measured).

Quoted 7 lines
Quoted 1 line

Having said all this, if the disagreements cannot be resolved, you can go for a

pmd API specific

Quoted 1 line

to your PMDs,

I don't think it is good idea.
PMD specific API is sort of deprecated path, also there is no clean way to use it within the libraries.

I know that this is a deprecated path, we can use it until we are not allowed to break ABI/API

Quoted 7 lines
Quoted 2 lines

because as per my understanding the solution doesn't look scalable to other

PMDs.

Quoted 1 line

Your approach is aligned only to Intel , will not benefit others like openssl

which is used by all

Quoted 1 line

vendors.

Quoted 12 lines

I feel quite opposite, from my perspective majority of SW backed PMDs will benefit from it.
And I don't see anything Intel specific in my proposals above. About openssl PMD: I am not an expert here, but looking at the code, I think it will fit really well.
Look yourself at its internal functions:
process_openssl_auth_op/process_openssl_crypto_op,
I think they doing exactly the same - they use sync API underneath, and they are session based
(AFAIK you don't need any device/queue data, everything that needed for crypto/auth is stored inside session).

By vendor specific, I mean,
- no PMD would like to have 2 different variants of session Init APIs for doing the same stuff.
- stacks will become vendor specific while using 2 separate session create APIs. No stack would Like to support 2 variants of session create- one for HW PMDs and one for SW PMDs.

-Akhil

#36 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Quoted 60 lines
Quoted 60 lines
Quoted 11 lines

Added my comments inline with your draft.
[snip]..

Quoted 8 lines

Ok, then my suggestion:
Let's at least write down all points about crypto-dev approach where we disagree and then probably try to resolve them one by one.... If we fail to make an agreement/progress in next week or so, (and no more reviews from the community)
will have bring that subject to TB meeting to decide.
Sounds fair to you?

Quoted 8 lines

Agreed

Quoted 5 lines

List is below.
Please add/correct me, if I missed something.

Konstantin

Before going into comparison, we should define the requirement as well.

Good point.

Quoted 2 lines

What I understood from the patchset,
"You need a synchronous API to perform crypto operations on raw data using

SW PMDs"

Quoted 7 lines

So,
- no crypto-ops,
- no separate enq-deq, only single process API for data path
- Do not need any value addition to the session parameters. (You would need some parameters from the crypto-op which Are constant per session and since you wont use crypto-op, You need some place to store that)

Yes, this is correct, I think.

Quoted 5 lines

Now as per your mail, the comparison
1. extra input parameters to create/init rte_(cpu)_sym_session.

Will leverage existing 6B gap inside rte_crypto_*_xform between 'algo' and

'key' fields.

Quoted 1 line

New fields will be optional and would be used by PMD only when cpu-crypto

session is requested.

Quoted 8 lines

For lksd-crypto session PMD is free to ignore these fields. No ABI breakage is required.

[Akhil] Agreed, no issues.

2. cpu-crypto create/init.
a) Our suggestion - introduce new API for that:
- rte_crypto_cpu_sym_init() that would init completely opaque

rte_crypto_cpu_sym_session.

Quoted 1 line

- struct rte_crypto_cpu_sym_session_ops {(*process)(...); (*clear);

/*whatever else we'll need *'};

Quoted 1 line

- rte_crypto_cpu_sym_get_ops(const struct rte_crypto_sym_xform

*xforms)

Quoted 1 line

that would return const struct rte_crypto_cpu_sym_session_ops *based

on input xforms.

Quoted 2 lines

Advantages:
1) totally opaque data structure (no ABI breakages in future), PMD

Quoted 8 lines
Quoted 6 lines

writer is totally free

Quoted 3 lines

with it format and contents.

[Akhil] It will have breakage at some point till we don't hit the union size.

Not sure, what union you are talking about?

Union of xforms in rte_security_session_conf

Hmm, how does it relates here?
I thought we discussing pure rte_cryptodev_sym_session, no?

Quoted 150 lines
Quoted 12 lines
Quoted 6 lines

Rather I don't suspect there will be more parameters added.
Or do we really care about the ABI breakage when the argument is about the correct place to add a piece of code or do we really agree to add code anywhere just to avoid that breakage.

I am talking about maintaining it in future.
if your struct is not seen externally, no chances to introduce ABI breakage.

Quoted 2 lines

2) each session entity is self-contained, user doesn't need to bring along

Quoted 8 lines

dev_id etc.

Quoted 1 line

dev_id is needed only at init stage, after that user will use session ops

to perform

Quoted 3 lines

all operations on that session (process(), clear(), etc.).

[Akhil] There is nothing called as session ops in current DPDK.

True, but it doesn't mean we can't/shouldn't have it.

We can have it if it is not adding complexity for the user. Creating 2 different code Paths for user is not desirable for the stack developers.

Quoted 20 lines
Quoted 3 lines

What you are proposing
is a new concept which doesn't have any extra benefit, rather it is adding

complexity

Quoted 4 lines

to have two different code paths for session create.

3) User can decide does he wants to store ops[] pointer on a per session

basis,

Quoted 3 lines

or on a per group of same sessions, or...

[Akhil] Will the user really care which process API should be called from the

PMD.

Quoted 1 line

Rather it should be driver's responsibility to store that in the session private

data

Quoted 1 line

which would be opaque to the user. As per my suggestion same process

function can

Quoted 1 line

be added to multiple sessions or a single session can be managed inside the

PMD.

Quoted 9 lines

In that case we either need to have a function per session (stored internally), or make decision (branches) at run-time.
But as I said in other mail - I am ok to add small shim structure here:
either rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops ops; }
or rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops *ops; }
And merge rte_crypto_cpu_sym_init() and rte_crypto_cpu_sym_get_ops() into one (init).

Again that will be a separate API call from the user perspective which is not good.

Quoted 82 lines
Quoted 4 lines

4) No mandatory mempools for private sessions. User can allocate

memory for cpu-crypto

Quoted 3 lines

session whenever he likes.

[Akhil] you mean session private data?

Yes.

Quoted 2 lines

You would need that memory anyways, user will be allocating that already. You do not need to manage that.

What I am saying - right now user has no choice but to allocate it via mempool. Which is probably not the best options for all cases.

Quoted 8 lines

Disadvantages:
5) Extra changes in control path
6) User has to store session_ops pointer explicitly.

[Akhil] More disadvantages:
- All supporting PMDs will need to maintain TWO types of session for the same crypto processing. Suppose a fix or a new feature(or algo) is added, PMD

owner

Quoted 1 line

will need to add code in both the session create APIs. Hence more

maintenance and

Quoted 1 line

error prone.

I think majority of code for both paths will be common, plus even we'll reuse current sym_session_init() -
changes in PMD session_init() code will be unavoidable.
But yes, it will be new entry in devops, that PMD will have to support. Ok to add it as 7) to the list.

Quoted 3 lines

- Stacks which will be using these new APIs also need to maintain two code path for the same processing while doing session initialization for sync and async

That's the same as #5 above, I think.

Quoted 3 lines

b) Your suggestion - reuse existing rte_cryptodev_sym_session_init() and

existing rte_cryptodev_sym_session

Quoted 6 lines

structure.
Advantages:
1) allows to reuse same struct and init/create/clear() functions. Probably less changes in control path.
Disadvantages:
2) rte_cryptodev_sym_session. sess_data[] is indexed by driver_id,

which means that

Quoted 1 line

we can't use the same rte_cryptodev_sym_session to hold private

sessions pointers

Quoted 5 lines

for both sync and async mode for the same device.
So the only option we have - make PMD devops-
sym_session_configure()
always create a session that can work in both cpu and lksd modes. For some implementations that would probably mean that under the

hood PMD would create

Quoted 1 line

2 different session structs (sync/async) and then use one or another

depending on from what API been called.

Quoted 3 lines

Seems doable, but ...:
- will contradict with statement from 1:
" New fields will be optional and would be used by PMD only when

cpu-crypto session is requested."

Quoted 1 line

Now it becomes mandatory for all apps to specify cpu-crypto

related parameters too,

Quoted 1 line

even if they don't plan to use that mode - i.e. behavior change,

existing app change.

Quoted 3 lines

- might cause extra space overhead.

[Akhil] It will not contradict with #1, you will only have few checks in the

session init PMD

Quoted 1 line

Which support this mode, find appropriate values and set the appropriate

process() in it.

Quoted 1 line

User should be able to call, legacy enq-deq as well as the new process()

without any issue.

Quoted 2 lines

User would be at runtime will be able to change the datapath.
So this is not a disadvantage, it would be additional flexibility for the user.

Quoted 11 lines

Ok, but that's what I am saying - if PMD would *always* have to create a session that can handle
both modes (sync/async), then user would *always* have to provide parameters for both modes too.
Otherwise if let say user didn't setup sync specific parameters at all, what PMD should do?
- return with error?
- init session that can be used with async path only?
My current assumption is #1.
If #2, then how user will be able to distinguish is that session valid for both modes, or only for one?

Quoted 3 lines

I would say a 3rd option, do nothing if sync params are not set. Probably have a debug print in the PMD(which support sync mode) to specify that session is not configured properly for sync mode.

So, just print warning and proceed with init session that can be used with async path only? Then it sounds the same as #2 above.
Which actually means that sync mode parameters for sym_session_init() becomes optional. Then we need an API to provide to the user information what modes (sync+async/async only) is supported by that session for given dev_id.
And user would have to query/retain this information at control-path,
and store it somewhere in user-space together with session pointer and dev_ids to use later at data-path (same as we do now for session type).
That definitely requires changes in control-path to start using it.
Plus the fact that this value can differ for different dev_ids for the same session - doesn't make things easier here.

Quoted 4 lines

Internally the PMD will not store the process() API in the session priv data And while calling the first packet, devops->process will give an assert that session Is not configured for sync mode. The session validation would be done in any case your suggestion or mine. So no extra overhead at runtime.

I believe that after session_init() user should get either an error or valid session handler that he can use at runtime.
Pushing session validation to runtime doesn't seem like a good idea.

Quoted 37 lines
Quoted 28 lines
Quoted 6 lines

3) not possible to store device (not driver) specific data within the

session, but I think it is not really needed right now.

Quoted 3 lines

So probably minor compared to 2.b.2.

[Akhil] So lets omit this for current discussion. And I hope we can find some

way to deal with it.

I don't think there is an easy way to fix that with existing API.

Quoted 6 lines

Actually #3 follows from #2, but decided to have them separated.

3. process() parameters/behavior
a) Our suggestion: user stores ptr to session ops (or to (*process) itself) and

just does:

Quoted 6 lines

session_ops->process(sess, ...); Advantages:
1) fastest possible execution path
2) no need to carry on dev_id for data-path

[Akhil] I don't see any overhead of carrying dev id, at least it would be inline

with the

Quoted 8 lines
Quoted 1 line

current DPDK methodology.

If we'll add process() into rte_cryptodev itself (same as we have enqueue_burst/dequeue_burst),
then it will be an ABI breakage.
Also there are discussions to get rid of that approach completely:
http://mails.dpdk.org/archives/dev/2019-September/144674.html
So I am not sure this is a recommended way these days.

Quoted 5 lines

We can either have it in rte_cryptodev or in rte_cryptodev_ops whichever is good for you.

Whether it is ABI breakage or not, as per your requirements, this is the correct approach. Do you agree with this or not?

I think it is possible approach, but not the best one:
it looks quite flakey to me (see all these uncertainty with sym_session_init above), plus introduces extra overhead at data-path.

Quoted 83 lines

Now handling the API/ABI breakage is a separate story. In 19.11 release we Are not much concerned about the ABI breakages, this was discussed in community. So adding a new dev_ops wouldn't have been an issue. Now since we are so close to RC1 deadline, we should come up with some other solution for next release. May be having a pmd API in 20.02 and converting it into formal one in 20.11

Quoted 9 lines
Quoted 2 lines

What you are suggesting is a new way to get the things done without much

benefit.

Would help with ABI stability plus better performance, isn't it enough?

Quoted 2 lines

Also I don't see any performance difference as crypto workload is heavier than Code cycles, so that wont matter.

Quoted 8 lines

It depends.
Suppose function call costs you ~30 cycles.
If you have burst of big packets (let say crypto for each will take ~2K cycles) that belong
to the same session, then yes you wouldn't notice these extra 30 cycles at all. If you have burst of small packets (let say crypto for each will take ~300 cycles) each
belongs to different session, then it will cost you ~10% extra.

Let us do some profiling on openssl with both the approaches and find out the difference.

Quoted 23 lines
Quoted 8 lines

So IMO, there is no advantage in your suggestion as well.

Disadvantages:
3) user has to carry on session_ops pointer explicitly
b) Your suggestion: add (*cpu_process) inside rte_cryptodev_ops and then:
rte_crypto_cpu_sym_process(uint8_t dev_id, rte_cryptodev_sym_session

*sess, /*data parameters*/) {...

Quoted 11 lines
                     rte_cryptodevs[dev_id].dev_ops->cpu_process(ses, ...);
                      /*and then inside PMD specifc process: */
                     pmd_private_session = sess->sess_data[this_pmd_driver_id].data;
                     /* and then most likely either */
                     pmd_private_session->process(pmd_private_session, ...);
                     /* or jump based on session/input data */
	Advantages:
	1) don't see any...
	Disadvantages:
	2) User has to carry on dev_id inside data-path
	3) Extra level of indirection (plus data dependency) - both for data and

instructions.

Quoted 2 lines

Possible slowdown compared to a) (not measured).

Quoted 7 lines
Quoted 1 line

Having said all this, if the disagreements cannot be resolved, you can go for a

pmd API specific

Quoted 1 line

to your PMDs,

I don't think it is good idea.
PMD specific API is sort of deprecated path, also there is no clean way to use it within the libraries.

I know that this is a deprecated path, we can use it until we are not allowed to break ABI/API

Quoted 7 lines
Quoted 2 lines

because as per my understanding the solution doesn't look scalable to other

PMDs.

Quoted 1 line

Your approach is aligned only to Intel , will not benefit others like openssl

which is used by all

Quoted 1 line

vendors.

Quoted 12 lines

I feel quite opposite, from my perspective majority of SW backed PMDs will benefit from it.
And I don't see anything Intel specific in my proposals above. About openssl PMD: I am not an expert here, but looking at the code, I think it will fit really well.
Look yourself at its internal functions:
process_openssl_auth_op/process_openssl_crypto_op,
I think they doing exactly the same - they use sync API underneath, and they are session based
(AFAIK you don't need any device/queue data, everything that needed for crypto/auth is stored inside session).

Quoted 4 lines

By vendor specific, I mean,
- no PMD would like to have 2 different variants of session Init APIs for doing the same stuff.
- stacks will become vendor specific while using 2 separate session create APIs. No stack would Like to support 2 variants of session create- one for HW PMDs and one for SW PMDs.

I think what you refer on has nothing to do with 'vendor specific'. I would name it 'extra overhead for PMD and stack writers'. Yes, for sure there is extra overhead (as always with new API) - for both producer (PMD writer) and consumer (stack writer):
New function(s) to support, probably more tests to create/run, etc. Though this API is optional - if PMD/stack maintainer doesn't see value in it, they are free not to support it.
From other side, re-using rte_cryptodev_sym_session_init() wouldn't help anyway - both data-path and control-path would differ from async mode anyway.
BTW, right now to support different HW flavors
we do have 4 different control and data-paths for both ipsec-secgw and librte_ipsec:
lkds-none/lksd-proto/inline-crypto/inline-proto.
And that is considered to be ok.
Honestly, I don't understand why SW backed implementations can't have their own path that would suite them most. Konstantin

#37 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Quoted 287 lines
Quoted 60 lines
Quoted 60 lines
Quoted 11 lines

Added my comments inline with your draft.
[snip]..

Quoted 8 lines

Ok, then my suggestion:
Let's at least write down all points about crypto-dev approach where we disagree and then probably try to resolve them one by one.... If we fail to make an agreement/progress in next week or so, (and no more reviews from the community)
will have bring that subject to TB meeting to decide.
Sounds fair to you?

Quoted 8 lines

Agreed

Quoted 5 lines

List is below.
Please add/correct me, if I missed something.

Konstantin

Before going into comparison, we should define the requirement as well.

Good point.

Quoted 2 lines

What I understood from the patchset,
"You need a synchronous API to perform crypto operations on raw data using

SW PMDs"

Quoted 7 lines

So,
- no crypto-ops,
- no separate enq-deq, only single process API for data path
- Do not need any value addition to the session parameters. (You would need some parameters from the crypto-op which Are constant per session and since you wont use crypto-op, You need some place to store that)

Yes, this is correct, I think.

Quoted 5 lines

Now as per your mail, the comparison
1. extra input parameters to create/init rte_(cpu)_sym_session.

Will leverage existing 6B gap inside rte_crypto_*_xform between 'algo' and

'key' fields.

Quoted 1 line

New fields will be optional and would be used by PMD only when cpu-crypto

session is requested.

Quoted 8 lines

For lksd-crypto session PMD is free to ignore these fields. No ABI breakage is required.

[Akhil] Agreed, no issues.

2. cpu-crypto create/init.
a) Our suggestion - introduce new API for that:
- rte_crypto_cpu_sym_init() that would init completely opaque

rte_crypto_cpu_sym_session.

Quoted 1 line

- struct rte_crypto_cpu_sym_session_ops {(*process)(...); (*clear);

/*whatever else we'll need *'};

Quoted 1 line

- rte_crypto_cpu_sym_get_ops(const struct rte_crypto_sym_xform

*xforms)

Quoted 1 line

that would return const struct rte_crypto_cpu_sym_session_ops *based

on input xforms.

Quoted 2 lines

Advantages:
1) totally opaque data structure (no ABI breakages in future), PMD

Quoted 8 lines
Quoted 6 lines

writer is totally free

Quoted 3 lines

with it format and contents.

[Akhil] It will have breakage at some point till we don't hit the union size.

Not sure, what union you are talking about?

Union of xforms in rte_security_session_conf

Hmm, how does it relates here?
I thought we discussing pure rte_cryptodev_sym_session, no?

Quoted 150 lines
Quoted 12 lines
Quoted 6 lines

Rather I don't suspect there will be more parameters added.
Or do we really care about the ABI breakage when the argument is about the correct place to add a piece of code or do we really agree to add code anywhere just to avoid that breakage.

I am talking about maintaining it in future.
if your struct is not seen externally, no chances to introduce ABI breakage.

Quoted 2 lines

2) each session entity is self-contained, user doesn't need to bring along

Quoted 8 lines

dev_id etc.

Quoted 1 line

dev_id is needed only at init stage, after that user will use session ops

to perform

Quoted 3 lines

all operations on that session (process(), clear(), etc.).

[Akhil] There is nothing called as session ops in current DPDK.

True, but it doesn't mean we can't/shouldn't have it.

We can have it if it is not adding complexity for the user. Creating 2 different code Paths for user is not desirable for the stack developers.

Quoted 20 lines
Quoted 3 lines

What you are proposing
is a new concept which doesn't have any extra benefit, rather it is adding

complexity

Quoted 4 lines

to have two different code paths for session create.

3) User can decide does he wants to store ops[] pointer on a per session

basis,

Quoted 3 lines

or on a per group of same sessions, or...

[Akhil] Will the user really care which process API should be called from the

PMD.

Quoted 1 line

Rather it should be driver's responsibility to store that in the session private

data

Quoted 1 line

which would be opaque to the user. As per my suggestion same process

function can

Quoted 1 line

be added to multiple sessions or a single session can be managed inside the

PMD.

Quoted 9 lines

In that case we either need to have a function per session (stored internally), or make decision (branches) at run-time.
But as I said in other mail - I am ok to add small shim structure here:
either rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops ops; }
or rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops *ops; }
And merge rte_crypto_cpu_sym_init() and rte_crypto_cpu_sym_get_ops() into one (init).

Again that will be a separate API call from the user perspective which is not good.

Quoted 82 lines
Quoted 4 lines

4) No mandatory mempools for private sessions. User can allocate

memory for cpu-crypto

Quoted 3 lines

session whenever he likes.

[Akhil] you mean session private data?

Yes.

Quoted 2 lines

You would need that memory anyways, user will be allocating that already. You do not need to manage that.

What I am saying - right now user has no choice but to allocate it via mempool. Which is probably not the best options for all cases.

Quoted 8 lines

Disadvantages:
5) Extra changes in control path
6) User has to store session_ops pointer explicitly.

[Akhil] More disadvantages:
- All supporting PMDs will need to maintain TWO types of session for the same crypto processing. Suppose a fix or a new feature(or algo) is added, PMD

owner

Quoted 1 line

will need to add code in both the session create APIs. Hence more

maintenance and

Quoted 1 line

error prone.

I think majority of code for both paths will be common, plus even we'll reuse current sym_session_init() -
changes in PMD session_init() code will be unavoidable.
But yes, it will be new entry in devops, that PMD will have to support. Ok to add it as 7) to the list.

Quoted 3 lines

- Stacks which will be using these new APIs also need to maintain two code path for the same processing while doing session initialization for sync and async

That's the same as #5 above, I think.

Quoted 3 lines

b) Your suggestion - reuse existing rte_cryptodev_sym_session_init() and

existing rte_cryptodev_sym_session

Quoted 6 lines

structure.
Advantages:
1) allows to reuse same struct and init/create/clear() functions. Probably less changes in control path.
Disadvantages:
2) rte_cryptodev_sym_session. sess_data[] is indexed by driver_id,

which means that

Quoted 1 line

we can't use the same rte_cryptodev_sym_session to hold private

sessions pointers

Quoted 5 lines

for both sync and async mode for the same device.
So the only option we have - make PMD devops-
sym_session_configure()
always create a session that can work in both cpu and lksd modes. For some implementations that would probably mean that under the

hood PMD would create

Quoted 1 line

2 different session structs (sync/async) and then use one or another

depending on from what API been called.

Quoted 3 lines

Seems doable, but ...:
- will contradict with statement from 1:
" New fields will be optional and would be used by PMD only when

cpu-crypto session is requested."

Quoted 1 line

Now it becomes mandatory for all apps to specify cpu-crypto

related parameters too,

Quoted 1 line

even if they don't plan to use that mode - i.e. behavior change,

existing app change.

Quoted 3 lines

- might cause extra space overhead.

[Akhil] It will not contradict with #1, you will only have few checks in the

session init PMD

Quoted 1 line

Which support this mode, find appropriate values and set the appropriate

process() in it.

Quoted 1 line

User should be able to call, legacy enq-deq as well as the new process()

without any issue.

Quoted 2 lines

User would be at runtime will be able to change the datapath.
So this is not a disadvantage, it would be additional flexibility for the user.

Quoted 11 lines

Ok, but that's what I am saying - if PMD would *always* have to create a session that can handle
both modes (sync/async), then user would *always* have to provide parameters for both modes too.
Otherwise if let say user didn't setup sync specific parameters at all, what PMD should do?
- return with error?
- init session that can be used with async path only?
My current assumption is #1.
If #2, then how user will be able to distinguish is that session valid for both modes, or only for one?

Quoted 3 lines

I would say a 3rd option, do nothing if sync params are not set. Probably have a debug print in the PMD(which support sync mode) to specify that session is not configured properly for sync mode.

So, just print warning and proceed with init session that can be used with async path only? Then it sounds the same as #2 above.
Which actually means that sync mode parameters for sym_session_init() becomes optional. Then we need an API to provide to the user information what modes (sync+async/async only) is supported by that session for given dev_id.
And user would have to query/retain this information at control-path,
and store it somewhere in user-space together with session pointer and dev_ids to use later at data-path (same as we do now for session type).
That definitely requires changes in control-path to start using it.
Plus the fact that this value can differ for different dev_ids for the same session - doesn't make things easier here.

Quoted 4 lines

Internally the PMD will not store the process() API in the session priv data And while calling the first packet, devops->process will give an assert that session Is not configured for sync mode. The session validation would be done in any case your suggestion or mine. So no extra overhead at runtime.

I believe that after session_init() user should get either an error or valid session handler that he can use at runtime.
Pushing session validation to runtime doesn't seem like a good idea.

Quoted 40 lines
Quoted 28 lines
Quoted 6 lines

3) not possible to store device (not driver) specific data within the

session, but I think it is not really needed right now.

Quoted 3 lines

So probably minor compared to 2.b.2.

[Akhil] So lets omit this for current discussion. And I hope we can find some

way to deal with it.

I don't think there is an easy way to fix that with existing API.

Quoted 6 lines

Actually #3 follows from #2, but decided to have them separated.

3. process() parameters/behavior
a) Our suggestion: user stores ptr to session ops (or to (*process) itself) and

just does:

Quoted 6 lines

session_ops->process(sess, ...); Advantages:
1) fastest possible execution path
2) no need to carry on dev_id for data-path

[Akhil] I don't see any overhead of carrying dev id, at least it would be inline

with the

Quoted 8 lines
Quoted 1 line

current DPDK methodology.

If we'll add process() into rte_cryptodev itself (same as we have enqueue_burst/dequeue_burst),
then it will be an ABI breakage.
Also there are discussions to get rid of that approach completely:
http://mails.dpdk.org/archives/dev/2019-September/144674.html
So I am not sure this is a recommended way these days.

We can either have it in rte_cryptodev or in rte_cryptodev_ops whichever is good for you.

Quoted 89 lines
Quoted 2 lines

Whether it is ABI breakage or not, as per your requirements, this is the correct approach. Do you agree with this or not?

I think it is possible approach, but not the best one:
it looks quite flakey to me (see all these uncertainty with sym_session_init above), plus introduces extra overhead at data-path.

Quoted 82 lines

Now handling the API/ABI breakage is a separate story. In 19.11 release we Are not much concerned about the ABI breakages, this was discussed in community. So adding a new dev_ops wouldn't have been an issue. Now since we are so close to RC1 deadline, we should come up with some other solution for next release. May be having a pmd API in 20.02 and converting it into formal one in 20.11

Quoted 9 lines
Quoted 2 lines

What you are suggesting is a new way to get the things done without much

benefit.

Would help with ABI stability plus better performance, isn't it enough?

Quoted 2 lines

Also I don't see any performance difference as crypto workload is heavier than Code cycles, so that wont matter.

Quoted 8 lines

It depends.
Suppose function call costs you ~30 cycles.
If you have burst of big packets (let say crypto for each will take ~2K cycles) that belong
to the same session, then yes you wouldn't notice these extra 30 cycles at all. If you have burst of small packets (let say crypto for each will take ~300 cycles) each
belongs to different session, then it will cost you ~10% extra.

Let us do some profiling on openssl with both the approaches and find out the difference.

Quoted 23 lines
Quoted 8 lines

So IMO, there is no advantage in your suggestion as well.

Disadvantages:
3) user has to carry on session_ops pointer explicitly
b) Your suggestion: add (*cpu_process) inside rte_cryptodev_ops and then:
rte_crypto_cpu_sym_process(uint8_t dev_id, rte_cryptodev_sym_session

*sess, /*data parameters*/) {...

Quoted 11 lines
                     rte_cryptodevs[dev_id].dev_ops->cpu_process(ses, ...);
                      /*and then inside PMD specifc process: */
                     pmd_private_session = sess->sess_data[this_pmd_driver_id].data;
                     /* and then most likely either */
                     pmd_private_session->process(pmd_private_session, ...);
                     /* or jump based on session/input data */
	Advantages:
	1) don't see any...
	Disadvantages:
	2) User has to carry on dev_id inside data-path
	3) Extra level of indirection (plus data dependency) - both for data and

instructions.

Quoted 2 lines

Possible slowdown compared to a) (not measured).

Quoted 7 lines
Quoted 1 line

Having said all this, if the disagreements cannot be resolved, you can go for a

pmd API specific

Quoted 1 line

to your PMDs,

I don't think it is good idea.
PMD specific API is sort of deprecated path, also there is no clean way to use it within the libraries.

I know that this is a deprecated path, we can use it until we are not allowed to break ABI/API

Quoted 18 lines
Quoted 2 lines

because as per my understanding the solution doesn't look scalable to other

PMDs.

Quoted 1 line

Your approach is aligned only to Intel , will not benefit others like openssl

which is used by all

Quoted 1 line

vendors.

I feel quite opposite, from my perspective majority of SW backed PMDs will benefit from it.
And I don't see anything Intel specific in my proposals above. About openssl PMD: I am not an expert here, but looking at the code, I think it will fit really well.
Look yourself at its internal functions:
process_openssl_auth_op/process_openssl_crypto_op,
I think they doing exactly the same - they use sync API underneath, and they are session based
(AFAIK you don't need any device/queue data, everything that needed for crypto/auth is stored inside session).

Looked at drivers/crypto/armv8 - same story here, I believe.

Quoted 29 lines
Quoted 5 lines
Quoted 1 line

By vendor specific, I mean,
- no PMD would like to have 2 different variants of session Init APIs for doing the same stuff.
- stacks will become vendor specific while using 2 separate session create APIs. No stack would Like to support 2 variants of session create- one for HW PMDs and one for SW PMDs.

I think what you refer on has nothing to do with 'vendor specific'. I would name it 'extra overhead for PMD and stack writers'. Yes, for sure there is extra overhead (as always with new API) - for both producer (PMD writer) and consumer (stack writer):
New function(s) to support, probably more tests to create/run, etc. Though this API is optional - if PMD/stack maintainer doesn't see value in it, they are free not to support it.
From other side, re-using rte_cryptodev_sym_session_init() wouldn't help anyway - both data-path and control-path would differ from async mode anyway.
BTW, right now to support different HW flavors
we do have 4 different control and data-paths for both ipsec-secgw and librte_ipsec:
lkds-none/lksd-proto/inline-crypto/inline-proto.
And that is considered to be ok.
Honestly, I don't understand why SW backed implementations can't have their own path that would suite them most. Konstantin

#38 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

In reply to Ananyev, Konstantin

Hi Konstantin,

Quoted 242 lines

Hi Akhil,

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

Added my comments inline with your draft.
[snip]..

Quoted 8 lines

Ok, then my suggestion:
Let's at least write down all points about crypto-dev approach where we disagree and then probably try to resolve them one by one.... If we fail to make an agreement/progress in next week or so, (and no more reviews from the community)
will have bring that subject to TB meeting to decide.
Sounds fair to you?

Agreed

Quoted 4 lines

List is below.
Please add/correct me, if I missed something.

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Konstantin

Before going into comparison, we should define the requirement as well.

Good point.

Quoted 2 lines

What I understood from the patchset,
"You need a synchronous API to perform crypto operations on raw data

using

Quoted 9 lines
Quoted 9 lines

SW PMDs"

Quoted 7 lines

So,
- no crypto-ops,
- no separate enq-deq, only single process API for data path
- Do not need any value addition to the session parameters. (You would need some parameters from the crypto-op which Are constant per session and since you wont use crypto-op, You need some place to store that)

Quoted 7 lines
Quoted 7 lines

Yes, this is correct, I think.

Quoted 5 lines

Now as per your mail, the comparison
1. extra input parameters to create/init rte_(cpu)_sym_session.

Will leverage existing 6B gap inside rte_crypto_*_xform between 'algo'

and

Quoted 2 lines
Quoted 2 lines

'key' fields.

Quoted 1 line

New fields will be optional and would be used by PMD only when cpu-

crypto

Quoted 9 lines
Quoted 9 lines

session is requested.

Quoted 8 lines

For lksd-crypto session PMD is free to ignore these fields. No ABI breakage is required.

[Akhil] Agreed, no issues.

2. cpu-crypto create/init.
a) Our suggestion - introduce new API for that:
- rte_crypto_cpu_sym_init() that would init completely opaque

Quoted 6 lines
Quoted 6 lines

rte_crypto_cpu_sym_session.

Quoted 1 line

- struct rte_crypto_cpu_sym_session_ops {(*process)(...); (*clear);

/*whatever else we'll need *'};

Quoted 1 line

- rte_crypto_cpu_sym_get_ops(const struct rte_crypto_sym_xform

*xforms)

Quoted 1 line

that would return const struct rte_crypto_cpu_sym_session_ops

*based

Quoted 11 lines
Quoted 9 lines

on input xforms.

Quoted 2 lines

Advantages:
1) totally opaque data structure (no ABI breakages in future), PMD

writer is totally free

Quoted 3 lines

with it format and contents.

[Akhil] It will have breakage at some point till we don't hit the union size.

Not sure, what union you are talking about?

Union of xforms in rte_security_session_conf

Hmm, how does it relates here?
I thought we discussing pure rte_cryptodev_sym_session, no?

Quoted 14 lines
Quoted 14 lines
Quoted 6 lines

Rather I don't suspect there will be more parameters added.
Or do we really care about the ABI breakage when the argument is about the correct place to add a piece of code or do we really agree to add code anywhere just to avoid that breakage.

I am talking about maintaining it in future.
if your struct is not seen externally, no chances to introduce ABI breakage.

Quoted 2 lines

2) each session entity is self-contained, user doesn't need to bring along

dev_id etc.

Quoted 1 line

dev_id is needed only at init stage, after that user will use session ops

Quoted 8 lines
Quoted 6 lines

to perform

Quoted 3 lines

all operations on that session (process(), clear(), etc.).

[Akhil] There is nothing called as session ops in current DPDK.

True, but it doesn't mean we can't/shouldn't have it.

We can have it if it is not adding complexity for the user. Creating 2 different

code

Quoted 30 lines

Paths for user is not desirable for the stack developers.

Quoted 28 lines
Quoted 3 lines

What you are proposing
is a new concept which doesn't have any extra benefit, rather it is adding

complexity

Quoted 4 lines

to have two different code paths for session create.

3) User can decide does he wants to store ops[] pointer on a per session

basis,

Quoted 3 lines

or on a per group of same sessions, or...

[Akhil] Will the user really care which process API should be called from the

PMD.

Quoted 1 line

Rather it should be driver's responsibility to store that in the session private

data

Quoted 1 line

which would be opaque to the user. As per my suggestion same process

function can

Quoted 1 line

be added to multiple sessions or a single session can be managed inside the

PMD.

In that case we either need to have a function per session (stored internally), or make decision (branches) at run-time.
But as I said in other mail - I am ok to add small shim structure here:
either rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops ops; }
or rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops *ops; }
And merge rte_crypto_cpu_sym_init() and rte_crypto_cpu_sym_get_ops()

into

Quoted 3 lines
Quoted 1 line

one (init).

Again that will be a separate API call from the user perspective which is not

good.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

4) No mandatory mempools for private sessions. User can allocate

memory for cpu-crypto

Quoted 2 lines

session whenever he likes.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

[Akhil] you mean session private data?

Yes.

Quoted 2 lines

You would need that memory anyways, user will be allocating that already. You do not need to manage that.

What I am saying - right now user has no choice but to allocate it via

mempool.

Quoted 10 lines
Quoted 10 lines

Which is probably not the best options for all cases.

Quoted 8 lines

Disadvantages:
5) Extra changes in control path
6) User has to store session_ops pointer explicitly.

[Akhil] More disadvantages:
- All supporting PMDs will need to maintain TWO types of session for the same crypto processing. Suppose a fix or a new feature(or algo) is added,

PMD

Quoted 52 lines
Quoted 52 lines

owner

Quoted 1 line

will need to add code in both the session create APIs. Hence more

maintenance and

Quoted 1 line

error prone.

I think majority of code for both paths will be common, plus even we'll reuse current sym_session_init() -
changes in PMD session_init() code will be unavoidable.
But yes, it will be new entry in devops, that PMD will have to support. Ok to add it as 7) to the list.

Quoted 3 lines

- Stacks which will be using these new APIs also need to maintain two code path for the same processing while doing session initialization for sync and async

That's the same as #5 above, I think.

Quoted 3 lines

b) Your suggestion - reuse existing rte_cryptodev_sym_session_init() and

existing rte_cryptodev_sym_session

Quoted 6 lines

structure.
Advantages:
1) allows to reuse same struct and init/create/clear() functions. Probably less changes in control path.
Disadvantages:
2) rte_cryptodev_sym_session. sess_data[] is indexed by driver_id,

which means that

Quoted 1 line

we can't use the same rte_cryptodev_sym_session to hold private

sessions pointers

Quoted 5 lines

for both sync and async mode for the same device.
So the only option we have - make PMD devops-
sym_session_configure()
always create a session that can work in both cpu and lksd modes. For some implementations that would probably mean that under the

hood PMD would create

Quoted 1 line

2 different session structs (sync/async) and then use one or another

depending on from what API been called.

Quoted 3 lines

Seems doable, but ...:
- will contradict with statement from 1:
" New fields will be optional and would be used by PMD only when

cpu-crypto session is requested."

Quoted 1 line

Now it becomes mandatory for all apps to specify cpu-crypto

related parameters too,

Quoted 1 line

even if they don't plan to use that mode - i.e. behavior change,

existing app change.

Quoted 3 lines

- might cause extra space overhead.

[Akhil] It will not contradict with #1, you will only have few checks in the

session init PMD

Quoted 1 line

Which support this mode, find appropriate values and set the appropriate

process() in it.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

User should be able to call, legacy enq-deq as well as the new process()

without any issue.

Quoted 2 lines

User would be at runtime will be able to change the datapath.
So this is not a disadvantage, it would be additional flexibility for the user.

Ok, but that's what I am saying - if PMD would *always* have to create a session that can handle
both modes (sync/async), then user would *always* have to provide

parameters

Quoted 2 lines
Quoted 2 lines

for both modes too.
Otherwise if let say user didn't setup sync specific parameters at all, what

PMD

Quoted 9 lines
Quoted 6 lines

should do?
- return with error?
- init session that can be used with async path only?
My current assumption is #1.
If #2, then how user will be able to distinguish is that session valid for both modes, or only for one?

I would say a 3rd option, do nothing if sync params are not set. Probably have a debug print in the PMD(which support sync mode) to specify

that

Quoted 1 line

session is not configured properly for sync mode.

Quoted 13 lines

So, just print warning and proceed with init session that can be used with async path only?
Then it sounds the same as #2 above.
Which actually means that sync mode parameters for sym_session_init() becomes optional.
Then we need an API to provide to the user information what modes (sync+async/async only) is supported by that session for given dev_id. And user would have to query/retain this information at control-path, and store it somewhere in user-space together with session pointer and dev_ids to use later at data-path (same as we do now for session type).
That definitely requires changes in control-path to start using it. Plus the fact that this value can differ for different dev_ids for the same session - doesn't make things easier here.

API wont be required to specify that. Feature flag will be sufficient, not a big change From the application perspective.

Here is some pseudo code just to elaborate my understanding. This will need some

From application,

If(dev_info->feature_flags & RTE_CRYPTODEV_FF_SYNC) {
	/* set additional params in crypto xform */
}

Now in the driver,

pmd_sym_session_configure(dev,xform,sess,mempool) {
	...
	If(dev_info->feature_flags & RTE_CRYPTODEV_FF_SYNC
		&& xform->/*sync params are set*/) {
		/*Assign process function pointer in sess->priv_data*/
	} /* It may return error if FF_SYNC is set and params are not correct.
	        It would be upto the driver whether it support both SYNC and ASYNC.*/
}

Now the new sync API

pmd_process(...) {
	If(dev_info->feature_flags & RTE_CRYPTODEV_FF_SYNC
			 && sess_priv->process != NULL)
		sess_priv->process(...);
	else
		ASSERT("sync mode not configured properly or not supported");
}

In the data path, there is no extra processing happening.
Even in case of your suggestion, you should have these type of error checks, You cannot blindly trust on the application that the pointers are correct.

Quoted 12 lines
Quoted 3 lines

Internally the PMD will not store the process() API in the session priv data And while calling the first packet, devops->process will give an assert that

session

Quoted 1 line

Is not configured for sync mode. The session validation would be done in any

case

Quoted 1 line

your suggestion or mine. So no extra overhead at runtime.

I believe that after session_init() user should get either an error or valid session handler that he can use at runtime.
Pushing session validation to runtime doesn't seem like a good idea.

It may get a warning from the PMD, that FF_SYNC is set but params are not Correct/available. See above.

Quoted 48 lines
Quoted 12 lines
Quoted 12 lines
Quoted 6 lines

3) not possible to store device (not driver) specific data within the

session, but I think it is not really needed right now.

Quoted 3 lines

So probably minor compared to 2.b.2.

[Akhil] So lets omit this for current discussion. And I hope we can find some

way to deal with it.

Quoted 8 lines
Quoted 8 lines

I don't think there is an easy way to fix that with existing API.

Quoted 6 lines

Actually #3 follows from #2, but decided to have them separated.

3. process() parameters/behavior
a) Our suggestion: user stores ptr to session ops (or to (*process) itself)

and

Quoted 7 lines
Quoted 7 lines

just does:

Quoted 6 lines

session_ops->process(sess, ...); Advantages:
1) fastest possible execution path
2) no need to carry on dev_id for data-path

[Akhil] I don't see any overhead of carrying dev id, at least it would be

inline

Quoted 8 lines
Quoted 8 lines

with the

Quoted 1 line

current DPDK methodology.

If we'll add process() into rte_cryptodev itself (same as we have enqueue_burst/dequeue_burst),
then it will be an ABI breakage.
Also there are discussions to get rid of that approach completely:

https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dpd
k.org%2Farchives%2Fdev%2F2019- September%2F144674.html&amp;data=02%7C01%7Cakhil.goyal%40nxp.com%7 C1859dc1d29cd45a51e9908d7571784bb%7C686ea1d3bc2b4c6fa92cd99c5c301 635%7C0%7C0%7C637073630835415165&amp;sdata=Bz9jgisyVzRJNt1BijtvSlurh JU1vXBbynNwlMDjaco%3D&amp;reserved=0

Quoted 5 lines
Quoted 1 line

So I am not sure this is a recommended way these days.

We can either have it in rte_cryptodev or in rte_cryptodev_ops whichever is good for you.

Quoted 7 lines
Quoted 2 lines

Whether it is ABI breakage or not, as per your requirements, this is the correct approach. Do you agree with this or not?

I think it is possible approach, but not the best one:
it looks quite flakey to me (see all these uncertainty with sym_session_init above),
plus introduces extra overhead at data-path.

Uncertainties can be handled appropriately using a feature flag And As per my understanding there is no extra overhead in data path.

Quoted 103 lines
Quoted 17 lines

Now handling the API/ABI breakage is a separate story. In 19.11 release we Are not much concerned about the ABI breakages, this was discussed in community. So adding a new dev_ops wouldn't have been an issue. Now since we are so close to RC1 deadline, we should come up with some other solution for next release. May be having a pmd API in 20.02 and converting it into formal one in 20.11

Quoted 7 lines
Quoted 2 lines

What you are suggesting is a new way to get the things done without much

benefit.

Would help with ABI stability plus better performance, isn't it enough?

Quoted 1 line

Also I don't see any performance difference as crypto workload is heavier

than

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

Code cycles, so that wont matter.

It depends.
Suppose function call costs you ~30 cycles.
If you have burst of big packets (let say crypto for each will take ~2K cycles)

that

Quoted 3 lines
Quoted 3 lines

belong
to the same session, then yes you wouldn't notice these extra 30 cycles at all. If you have burst of small packets (let say crypto for each will take ~300

cycles)

Quoted 13 lines
Quoted 2 lines

each
belongs to different session, then it will cost you ~10% extra.

Let us do some profiling on openssl with both the approaches and find out the difference.

Quoted 7 lines
Quoted 7 lines

So IMO, there is no advantage in your suggestion as well.

Disadvantages:
3) user has to carry on session_ops pointer explicitly
b) Your suggestion: add (*cpu_process) inside rte_cryptodev_ops and

then:

Quoted 1 line
Quoted 1 line
Quoted 1 line

rte_crypto_cpu_sym_process(uint8_t dev_id,

rte_cryptodev_sym_session

Quoted 17 lines
Quoted 4 lines

*sess, /*data parameters*/) {...

Quoted 3 lines
                     rte_cryptodevs[dev_id].dev_ops->cpu_process(ses, ...);
                      /*and then inside PMD specifc process: */
                     pmd_private_session = sess-

sess_data[this_pmd_driver_id].data;

Quoted 12 lines
Quoted 8 lines

/* and then most likely either */ pmd_private_session->process(pmd_private_session, ...); /* or jump based on session/input data */
Advantages:
1) don't see any...
Disadvantages:
2) User has to carry on dev_id inside data-path
3) Extra level of indirection (plus data dependency) - both for data and

instructions.

Quoted 3 lines

Possible slowdown compared to a) (not measured).

Having said all this, if the disagreements cannot be resolved, you can go

for a

Quoted 5 lines
Quoted 5 lines

pmd API specific

Quoted 1 line

to your PMDs,

I don't think it is good idea.
PMD specific API is sort of deprecated path, also there is no clean way to use

it

Quoted 7 lines
Quoted 1 line

within the libraries.

I know that this is a deprecated path, we can use it until we are not allowed to break ABI/API

Quoted 2 lines
Quoted 2 lines

because as per my understanding the solution doesn't look scalable to

other

Quoted 9 lines
Quoted 9 lines

PMDs.

Quoted 1 line

Your approach is aligned only to Intel , will not benefit others like openssl

which is used by all

Quoted 1 line

vendors.

I feel quite opposite, from my perspective majority of SW backed PMDs will benefit from it.
And I don't see anything Intel specific in my proposals above. About openssl PMD: I am not an expert here, but looking at the code, I think

it

Quoted 4 lines
Quoted 4 lines

will fit really well.
Look yourself at its internal functions:
process_openssl_auth_op/process_openssl_crypto_op,
I think they doing exactly the same - they use sync API underneath, and they

are

Quoted 6 lines
Quoted 4 lines

session based
(AFAIK you don't need any device/queue data, everything that needed for crypto/auth is stored inside session).

By vendor specific, I mean,
- no PMD would like to have 2 different variants of session Init APIs for doing

the same stuff.

Quoted 1 line

- stacks will become vendor specific while using 2 separate session create APIs.

No stack would

Quoted 1 line

Like to support 2 variants of session create- one for HW PMDs and one for SW

PMDs.

Quoted 15 lines

I think what you refer on has nothing to do with 'vendor specific'. I would name it 'extra overhead for PMD and stack writers'. Yes, for sure there is extra overhead (as always with new API) - for both producer (PMD writer) and consumer (stack writer):
New function(s) to support, probably more tests to create/run, etc. Though this API is optional - if PMD/stack maintainer doesn't see value in it, they are free not to support it.
From other side, re-using rte_cryptodev_sym_session_init() wouldn't help anyway - both data-path and control-path would differ from async mode anyway.
BTW, right now to support different HW flavors
we do have 4 different control and data-paths for both ipsec-secgw and librte_ipsec:
lkds-none/lksd-proto/inline-crypto/inline-proto.
And that is considered to be ok.

No that is not ok. We cannot add new paths for every other case. Those 4 are controlled using 2 set of APIs. We should try our best to Have minimum overhead to the application writer. This pain was also discussed In the one of DPDK conference as well.
DPDK is not a standalone entity, there are stacks running over it always. We should not add API for every other use case when we have an alternative Approach with the existing API set.

Now introducing another one would add to that pain and a lot of work for Both producer and consumer.
It would be interesting to see how much performance difference will be there in the Two approaches. As per my understanding it wont be much as compared to the Extra work that you will be inducing.

-Akhil

Quoted 8 lines

Honestly, I don't understand why SW backed implementations can't have their own path that would suite them most. Konstantin

#39 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

Hi Akhil,

Quoted 262 lines
Quoted 238 lines
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

Added my comments inline with your draft.
[snip]..

Quoted 8 lines

Ok, then my suggestion:
Let's at least write down all points about crypto-dev approach where we disagree and then probably try to resolve them one by one.... If we fail to make an agreement/progress in next week or so, (and no more reviews from the community)
will have bring that subject to TB meeting to decide.
Sounds fair to you?

Agreed

Quoted 4 lines

List is below.
Please add/correct me, if I missed something.

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Konstantin

Before going into comparison, we should define the requirement as well.

Good point.

Quoted 2 lines

What I understood from the patchset,
"You need a synchronous API to perform crypto operations on raw data

using

Quoted 9 lines
Quoted 9 lines

SW PMDs"

Quoted 7 lines

So,
- no crypto-ops,
- no separate enq-deq, only single process API for data path
- Do not need any value addition to the session parameters. (You would need some parameters from the crypto-op which Are constant per session and since you wont use crypto-op, You need some place to store that)

Quoted 7 lines
Quoted 7 lines

Yes, this is correct, I think.

Quoted 5 lines

Now as per your mail, the comparison
1. extra input parameters to create/init rte_(cpu)_sym_session.

Will leverage existing 6B gap inside rte_crypto_*_xform between 'algo'

and

Quoted 2 lines
Quoted 2 lines

'key' fields.

Quoted 1 line

New fields will be optional and would be used by PMD only when cpu-

crypto

Quoted 9 lines
Quoted 9 lines

session is requested.

Quoted 8 lines

For lksd-crypto session PMD is free to ignore these fields. No ABI breakage is required.

[Akhil] Agreed, no issues.

2. cpu-crypto create/init.
a) Our suggestion - introduce new API for that:
- rte_crypto_cpu_sym_init() that would init completely opaque

Quoted 6 lines
Quoted 6 lines

rte_crypto_cpu_sym_session.

Quoted 1 line

- struct rte_crypto_cpu_sym_session_ops {(*process)(...); (*clear);

/*whatever else we'll need *'};

Quoted 1 line

- rte_crypto_cpu_sym_get_ops(const struct rte_crypto_sym_xform

*xforms)

Quoted 1 line

that would return const struct rte_crypto_cpu_sym_session_ops

*based

Quoted 11 lines
Quoted 9 lines

on input xforms.

Quoted 2 lines

Advantages:
1) totally opaque data structure (no ABI breakages in future), PMD

writer is totally free

Quoted 3 lines

with it format and contents.

[Akhil] It will have breakage at some point till we don't hit the union size.

Not sure, what union you are talking about?

Union of xforms in rte_security_session_conf

Hmm, how does it relates here?
I thought we discussing pure rte_cryptodev_sym_session, no?

Quoted 14 lines
Quoted 14 lines
Quoted 6 lines

Rather I don't suspect there will be more parameters added.
Or do we really care about the ABI breakage when the argument is about the correct place to add a piece of code or do we really agree to add code anywhere just to avoid that breakage.

I am talking about maintaining it in future.
if your struct is not seen externally, no chances to introduce ABI breakage.

Quoted 2 lines

2) each session entity is self-contained, user doesn't need to bring along

dev_id etc.

Quoted 1 line

dev_id is needed only at init stage, after that user will use session ops

Quoted 8 lines
Quoted 6 lines

to perform

Quoted 3 lines

all operations on that session (process(), clear(), etc.).

[Akhil] There is nothing called as session ops in current DPDK.

True, but it doesn't mean we can't/shouldn't have it.

We can have it if it is not adding complexity for the user. Creating 2 different

code

Quoted 30 lines

Paths for user is not desirable for the stack developers.

Quoted 28 lines
Quoted 3 lines

What you are proposing
is a new concept which doesn't have any extra benefit, rather it is adding

complexity

Quoted 4 lines

to have two different code paths for session create.

3) User can decide does he wants to store ops[] pointer on a per session

basis,

Quoted 3 lines

or on a per group of same sessions, or...

[Akhil] Will the user really care which process API should be called from the

PMD.

Quoted 1 line

Rather it should be driver's responsibility to store that in the session private

data

Quoted 1 line

which would be opaque to the user. As per my suggestion same process

function can

Quoted 1 line

be added to multiple sessions or a single session can be managed inside the

PMD.

In that case we either need to have a function per session (stored internally), or make decision (branches) at run-time.
But as I said in other mail - I am ok to add small shim structure here:
either rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops ops; }
or rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops *ops; }
And merge rte_crypto_cpu_sym_init() and rte_crypto_cpu_sym_get_ops()

into

Quoted 3 lines
Quoted 1 line

one (init).

Again that will be a separate API call from the user perspective which is not

good.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

4) No mandatory mempools for private sessions. User can allocate

memory for cpu-crypto

Quoted 2 lines

session whenever he likes.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

[Akhil] you mean session private data?

Yes.

Quoted 2 lines

You would need that memory anyways, user will be allocating that already. You do not need to manage that.

What I am saying - right now user has no choice but to allocate it via

mempool.

Quoted 10 lines
Quoted 10 lines

Which is probably not the best options for all cases.

Quoted 8 lines

Disadvantages:
5) Extra changes in control path
6) User has to store session_ops pointer explicitly.

[Akhil] More disadvantages:
- All supporting PMDs will need to maintain TWO types of session for the same crypto processing. Suppose a fix or a new feature(or algo) is added,

PMD

Quoted 52 lines
Quoted 52 lines

owner

Quoted 1 line

will need to add code in both the session create APIs. Hence more

maintenance and

Quoted 1 line

error prone.

I think majority of code for both paths will be common, plus even we'll reuse current sym_session_init() -
changes in PMD session_init() code will be unavoidable.
But yes, it will be new entry in devops, that PMD will have to support. Ok to add it as 7) to the list.

Quoted 3 lines

- Stacks which will be using these new APIs also need to maintain two code path for the same processing while doing session initialization for sync and async

That's the same as #5 above, I think.

Quoted 3 lines

b) Your suggestion - reuse existing rte_cryptodev_sym_session_init() and

existing rte_cryptodev_sym_session

Quoted 6 lines

structure.
Advantages:
1) allows to reuse same struct and init/create/clear() functions. Probably less changes in control path.
Disadvantages:
2) rte_cryptodev_sym_session. sess_data[] is indexed by driver_id,

which means that

Quoted 1 line

we can't use the same rte_cryptodev_sym_session to hold private

sessions pointers

Quoted 5 lines

for both sync and async mode for the same device.
So the only option we have - make PMD devops-
sym_session_configure()
always create a session that can work in both cpu and lksd modes. For some implementations that would probably mean that under the

hood PMD would create

Quoted 1 line

2 different session structs (sync/async) and then use one or another

depending on from what API been called.

Quoted 3 lines

Seems doable, but ...:
- will contradict with statement from 1:
" New fields will be optional and would be used by PMD only when

cpu-crypto session is requested."

Quoted 1 line

Now it becomes mandatory for all apps to specify cpu-crypto

related parameters too,

Quoted 1 line

even if they don't plan to use that mode - i.e. behavior change,

existing app change.

Quoted 3 lines

- might cause extra space overhead.

[Akhil] It will not contradict with #1, you will only have few checks in the

session init PMD

Quoted 1 line

Which support this mode, find appropriate values and set the appropriate

process() in it.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

User should be able to call, legacy enq-deq as well as the new process()

without any issue.

Quoted 2 lines

User would be at runtime will be able to change the datapath.
So this is not a disadvantage, it would be additional flexibility for the user.

Ok, but that's what I am saying - if PMD would *always* have to create a session that can handle
both modes (sync/async), then user would *always* have to provide

parameters

Quoted 2 lines
Quoted 2 lines

for both modes too.
Otherwise if let say user didn't setup sync specific parameters at all, what

PMD

Quoted 9 lines
Quoted 6 lines

should do?
- return with error?
- init session that can be used with async path only?
My current assumption is #1.
If #2, then how user will be able to distinguish is that session valid for both modes, or only for one?

I would say a 3rd option, do nothing if sync params are not set. Probably have a debug print in the PMD(which support sync mode) to specify

that

Quoted 1 line

session is not configured properly for sync mode.

Quoted 13 lines

So, just print warning and proceed with init session that can be used with async path only?
Then it sounds the same as #2 above.
Which actually means that sync mode parameters for sym_session_init() becomes optional.
Then we need an API to provide to the user information what modes (sync+async/async only) is supported by that session for given dev_id. And user would have to query/retain this information at control-path, and store it somewhere in user-space together with session pointer and dev_ids to use later at data-path (same as we do now for session type).
That definitely requires changes in control-path to start using it. Plus the fact that this value can differ for different dev_ids for the same session - doesn't make things easier here.

API wont be required to specify that. Feature flag will be sufficient, not a big change From the application perspective.

Here is some pseudo code just to elaborate my understanding. This will need some

From application,

If(dev_info->feature_flags & RTE_CRYPTODEV_FF_SYNC) {
	/* set additional params in crypto xform */
}
Quoted 7 lines

Now in the driver,

pmd_sym_session_configure(dev,xform,sess,mempool) {
	...
	If(dev_info->feature_flags & RTE_CRYPTODEV_FF_SYNC
		&& xform->/*sync params are set*/) {
		/*Assign process function pointer in sess->priv_data*/
	} /* It may return error if FF_SYNC is set and params are not correct.

Then all apps will always *have to* setup sync parameters in xform. What you suggest is *mandatory* sync mode: user always has to setup sync mode params if PMD does support it (no matter does he plan to use sync mode or not). Which means behavior change in existing apps.

Quoted 23 lines

It would be upto the driver whether it support both SYNC and ASYNC.*/
}

Now the new sync API

pmd_process(...) {
	If(dev_info->feature_flags & RTE_CRYPTODEV_FF_SYNC
			 && sess_priv->process != NULL)
		sess_priv->process(...);
	else
		ASSERT("sync mode not configured properly or not supported");
}

In the data path, there is no extra processing happening.
Even in case of your suggestion, you should have these type of error checks, You cannot blindly trust on the application that the pointers are correct.

Quoted 6 lines
Quoted 3 lines

Internally the PMD will not store the process() API in the session priv data And while calling the first packet, devops->process will give an assert that

session

Quoted 1 line

Is not configured for sync mode. The session validation would be done in any

case

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

your suggestion or mine. So no extra overhead at runtime.

I believe that after session_init() user should get either an error or valid session handler that he can use at runtime.
Pushing session validation to runtime doesn't seem like a good idea.

It may get a warning from the PMD, that FF_SYNC is set but params are not Correct/available. See above.

I think warning is not enough.
There should be a clear way (API) for developer to realize is the created session can be used by sync API data-path or not.

Quoted 178 lines
Quoted 49 lines
Quoted 13 lines
Quoted 13 lines
Quoted 7 lines

3) not possible to store device (not driver) specific data within the

session, but I think it is not really needed right now.

Quoted 3 lines

So probably minor compared to 2.b.2.

[Akhil] So lets omit this for current discussion. And I hope we can find some

way to deal with it.

Quoted 8 lines
Quoted 8 lines

I don't think there is an easy way to fix that with existing API.

Quoted 6 lines

Actually #3 follows from #2, but decided to have them separated.

3. process() parameters/behavior
a) Our suggestion: user stores ptr to session ops (or to (*process) itself)

and

Quoted 7 lines
Quoted 7 lines

just does:

Quoted 6 lines

session_ops->process(sess, ...); Advantages:
1) fastest possible execution path
2) no need to carry on dev_id for data-path

[Akhil] I don't see any overhead of carrying dev id, at least it would be

inline

Quoted 8 lines
Quoted 8 lines

with the

Quoted 1 line

current DPDK methodology.

If we'll add process() into rte_cryptodev itself (same as we have enqueue_burst/dequeue_burst),
then it will be an ABI breakage.
Also there are discussions to get rid of that approach completely:

https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dpd
k.org%2Farchives%2Fdev%2F2019- September%2F144674.html&amp;data=02%7C01%7Cakhil.goyal%40nxp.com%7 C1859dc1d29cd45a51e9908d7571784bb%7C686ea1d3bc2b4c6fa92cd99c5c301 635%7C0%7C0%7C637073630835415165&amp;sdata=Bz9jgisyVzRJNt1BijtvSlurh JU1vXBbynNwlMDjaco%3D&amp;reserved=0

Quoted 5 lines
Quoted 1 line

So I am not sure this is a recommended way these days.

We can either have it in rte_cryptodev or in rte_cryptodev_ops whichever is good for you.

Quoted 7 lines
Quoted 2 lines

Whether it is ABI breakage or not, as per your requirements, this is the correct approach. Do you agree with this or not?

I think it is possible approach, but not the best one:
it looks quite flakey to me (see all these uncertainty with sym_session_init above),
plus introduces extra overhead at data-path.

Uncertainties can be handled appropriately using a feature flag

Quoted 103 lines
Quoted 17 lines

Now handling the API/ABI breakage is a separate story. In 19.11 release we Are not much concerned about the ABI breakages, this was discussed in community. So adding a new dev_ops wouldn't have been an issue. Now since we are so close to RC1 deadline, we should come up with some other solution for next release. May be having a pmd API in 20.02 and converting it into formal one in 20.11

Quoted 7 lines
Quoted 2 lines

What you are suggesting is a new way to get the things done without much

benefit.

Would help with ABI stability plus better performance, isn't it enough?

Quoted 1 line

Also I don't see any performance difference as crypto workload is heavier

than

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

Code cycles, so that wont matter.

It depends.
Suppose function call costs you ~30 cycles.
If you have burst of big packets (let say crypto for each will take ~2K cycles)

that

Quoted 3 lines
Quoted 3 lines

belong
to the same session, then yes you wouldn't notice these extra 30 cycles at all. If you have burst of small packets (let say crypto for each will take ~300

cycles)

Quoted 13 lines
Quoted 2 lines

each
belongs to different session, then it will cost you ~10% extra.

Let us do some profiling on openssl with both the approaches and find out the difference.

Quoted 7 lines
Quoted 7 lines

So IMO, there is no advantage in your suggestion as well.

Disadvantages:
3) user has to carry on session_ops pointer explicitly
b) Your suggestion: add (*cpu_process) inside rte_cryptodev_ops and

then:

Quoted 1 line
Quoted 1 line
Quoted 1 line

rte_crypto_cpu_sym_process(uint8_t dev_id,

rte_cryptodev_sym_session

Quoted 17 lines
Quoted 4 lines

*sess, /*data parameters*/) {...

Quoted 3 lines
                     rte_cryptodevs[dev_id].dev_ops->cpu_process(ses, ...);
                      /*and then inside PMD specifc process: */
                     pmd_private_session = sess-

sess_data[this_pmd_driver_id].data;

Quoted 12 lines
Quoted 8 lines

/* and then most likely either */ pmd_private_session->process(pmd_private_session, ...); /* or jump based on session/input data */
Advantages:
1) don't see any...
Disadvantages:
2) User has to carry on dev_id inside data-path
3) Extra level of indirection (plus data dependency) - both for data and

instructions.

Quoted 3 lines

Possible slowdown compared to a) (not measured).

Having said all this, if the disagreements cannot be resolved, you can go

for a

Quoted 5 lines
Quoted 5 lines

pmd API specific

Quoted 1 line

to your PMDs,

I don't think it is good idea.
PMD specific API is sort of deprecated path, also there is no clean way to use

it

Quoted 7 lines
Quoted 1 line

within the libraries.

I know that this is a deprecated path, we can use it until we are not allowed to break ABI/API

Quoted 2 lines
Quoted 2 lines

because as per my understanding the solution doesn't look scalable to

other

Quoted 9 lines
Quoted 9 lines

PMDs.

Quoted 1 line

Your approach is aligned only to Intel , will not benefit others like openssl

which is used by all

Quoted 1 line

vendors.

I feel quite opposite, from my perspective majority of SW backed PMDs will benefit from it.
And I don't see anything Intel specific in my proposals above. About openssl PMD: I am not an expert here, but looking at the code, I think

it

Quoted 4 lines
Quoted 4 lines

will fit really well.
Look yourself at its internal functions:
process_openssl_auth_op/process_openssl_crypto_op,
I think they doing exactly the same - they use sync API underneath, and they

are

Quoted 6 lines
Quoted 4 lines

session based
(AFAIK you don't need any device/queue data, everything that needed for crypto/auth is stored inside session).

By vendor specific, I mean,
- no PMD would like to have 2 different variants of session Init APIs for doing

the same stuff.

Quoted 1 line

- stacks will become vendor specific while using 2 separate session create APIs.

No stack would

Quoted 1 line

Like to support 2 variants of session create- one for HW PMDs and one for SW

PMDs.

Quoted 15 lines

I think what you refer on has nothing to do with 'vendor specific'. I would name it 'extra overhead for PMD and stack writers'. Yes, for sure there is extra overhead (as always with new API) - for both producer (PMD writer) and consumer (stack writer):
New function(s) to support, probably more tests to create/run, etc. Though this API is optional - if PMD/stack maintainer doesn't see value in it, they are free not to support it.
From other side, re-using rte_cryptodev_sym_session_init() wouldn't help anyway - both data-path and control-path would differ from async mode anyway.
BTW, right now to support different HW flavors
we do have 4 different control and data-paths for both ipsec-secgw and librte_ipsec:
lkds-none/lksd-proto/inline-crypto/inline-proto.
And that is considered to be ok.

Quoted 1 line

No that is not ok. We cannot add new paths for every other case.

What I am saying: if let-say lookaside-proto/inline-crypto/inline-proto deserves its own case in rte_security/rte_crypto API,
I don't understand why cpu-crypto doesn't.

Quoted 1 line

Those 4 are controlled using 2 set of APIs.

Yes there are 2 API sets (rte_cryptodev/rte_security),
but in fact if you look at ipsec-secgw and librte_ipsec we have 4 different code paths. For both create_session() and ipsec_enqueue() we have a big switch() with 4 different cases. Nearly the same for librte_ipsec - we have different prepare/process
function pointers for each security type.

Quoted 9 lines

We should try our best to
Have minimum overhead to the application writer. This pain was also discussed In the one of DPDK conference as well.
DPDK is not a standalone entity, there are stacks running over it always. We should not add API for every other use case when we have an alternative Approach with the existing API set.

Now introducing another one would add to that pain and a lot of work for Both producer and consumer.

If I would see a clean approach to implement desired functionality without introducing new API - I would definitely support it.
The problem is that from my perspective,
what you suggesting with existing API will bring more drawbacks then positives. BTW, our first approach (via rte_security) does reuse existing API, so if adding new API is the main concern - let's reconsider that path.

Quoted 14 lines

It would be interesting to see how much performance difference will be there in the Two approaches. As per my understanding it wont be much as compared to the Extra work that you will be inducing.

-Akhil

Quoted 8 lines

Honestly, I don't understand why SW backed implementations can't have their own path that would suite them most. Konstantin