cryptodev: sym session framework rework
Subsystems (5)
cover cryptodev: sym session framework rework
This patchset reworks the symmetric crypto session data structure to use a single virtual/physical contiguous buffer for symmetric crypto session and driver private data. In addition the session data structure is now private. The session is represented as an opaque pointer in the application.
With the change the session is no longer supported to be accessed by multiple device drivers. For the same reason rte_cryptodev_sym_session_init/clear APIs are deprecated as rte_cryptodev_sym_session_create/free will initialize and clear the driver specific data field.
Fan Zhang (3):
cryptodev: rework session framework
crypto/scheduler: use unified session
cryptodev: hide sym session structure
app/test-crypto-perf/cperf_ops.c | 21 +- app/test-crypto-perf/cperf_test_latency.c | 6 +- .../cperf_test_pmd_cyclecount.c | 5 +- app/test-crypto-perf/cperf_test_throughput.c | 6 +- app/test-crypto-perf/cperf_test_verify.c | 6 +- app/test-crypto-perf/main.c | 29 +- app/test-eventdev/test_perf_common.c | 35 +- app/test-eventdev/test_perf_common.h | 1 - app/test/test_cryptodev.c | 302 +++++------------- app/test/test_cryptodev_blockcipher.c | 16 +- app/test/test_event_crypto_adapter.c | 35 +- app/test/test_ipsec.c | 42 +-- app/test/test_ipsec_perf.c | 4 +- doc/guides/prog_guide/cryptodev_lib.rst | 16 +- doc/guides/rel_notes/deprecation.rst | 9 + doc/guides/rel_notes/release_22_11.rst | 7 + drivers/crypto/armv8/rte_armv8_pmd.c | 21 +- drivers/crypto/armv8/rte_armv8_pmd_ops.c | 32 +- drivers/crypto/bcmfs/bcmfs_sym_session.c | 38 +-- drivers/crypto/bcmfs/bcmfs_sym_session.h | 3 +- drivers/crypto/caam_jr/caam_jr.c | 28 +- drivers/crypto/ccp/ccp_crypto.c | 58 +--- drivers/crypto/ccp/ccp_pmd_ops.c | 32 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 - drivers/crypto/ccp/rte_ccp_pmd.c | 29 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 36 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 31 +- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 54 +--- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 14 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 31 +- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 3 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 37 +-- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 4 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 34 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 41 ++- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 48 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 26 +- drivers/crypto/ipsec_mb/pmd_chacha_poly.c | 4 - drivers/crypto/ipsec_mb/pmd_kasumi.c | 10 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 9 +- drivers/crypto/ipsec_mb/pmd_zuc.c | 4 - drivers/crypto/mlx5/mlx5_crypto.c | 26 +- drivers/crypto/mvsam/rte_mrvl_pmd.c | 8 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 21 +- drivers/crypto/nitrox/nitrox_sym.c | 39 +-- drivers/crypto/null/null_crypto_pmd.c | 19 +- drivers/crypto/null/null_crypto_pmd_ops.c | 33 +- drivers/crypto/null/null_crypto_pmd_private.h | 2 - .../crypto/octeontx/otx_cryptodev_hw_access.h | 1 - drivers/crypto/octeontx/otx_cryptodev_ops.c | 67 +--- drivers/crypto/openssl/openssl_pmd_private.h | 2 - drivers/crypto/openssl/rte_openssl_pmd.c | 24 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 29 +- drivers/crypto/qat/qat_sym.c | 10 +- drivers/crypto/qat/qat_sym.h | 4 +- drivers/crypto/qat/qat_sym_session.c | 40 +-- drivers/crypto/qat/qat_sym_session.h | 6 +- drivers/crypto/scheduler/scheduler_failover.c | 19 +- .../crypto/scheduler/scheduler_multicore.c | 17 + .../scheduler/scheduler_pkt_size_distr.c | 84 +++-- drivers/crypto/scheduler/scheduler_pmd_ops.c | 87 ++++- .../crypto/scheduler/scheduler_pmd_private.h | 120 ++++++- .../crypto/scheduler/scheduler_roundrobin.c | 11 +- drivers/crypto/virtio/virtio_cryptodev.c | 40 +-- drivers/crypto/virtio/virtio_rxtx.c | 3 +- examples/fips_validation/fips_dev_self_test.c | 30 +- examples/fips_validation/main.c | 35 +- examples/ipsec-secgw/ipsec-secgw.c | 10 +- examples/ipsec-secgw/ipsec.c | 7 +- examples/l2fwd-crypto/main.c | 54 +--- examples/vhost_crypto/main.c | 16 +- lib/cryptodev/cryptodev_pmd.h | 59 ++-- lib/cryptodev/cryptodev_trace_points.c | 6 - lib/cryptodev/rte_cryptodev.c | 280 ++++++---------- lib/cryptodev/rte_cryptodev.h | 153 +++------ lib/cryptodev/rte_cryptodev_trace.h | 35 +- lib/cryptodev/version.map | 6 - lib/ipsec/rte_ipsec_group.h | 5 +- lib/ipsec/ses.c | 3 +- lib/pipeline/rte_table_action.c | 10 +- lib/vhost/rte_vhost_crypto.h | 3 - lib/vhost/vhost_crypto.c | 28 +- 82 files changed, 910 insertions(+), 1711 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/3 cryptodev: rework session framework
As per current design, rte_cryptodev_sym_session_create() and rte_cryptodev_sym_session_init() use separate mempool objects for a single session.
And structure rte_cryptodev_sym_session is not directly used by the application, it may cause ABI breakage if the structure is modified in future.
To address these two issues, the rte_cryptodev_sym_session_create will take one mempool object that the session and session private data are virtually/physically contiguous, and initializes both fields. The API rte_cryptodev_sym_session_init is removed.
rte_cryptodev_sym_session_create will now return an opaque session pointer which will be used by the app and other APIs.
In data path, opaque session pointer is attached to rte_crypto_op and the PMD can call an internal library API to get the session private data pointer based on the driver id.
Note: currently single session may be used by different device drivers, given it is initialized by them. After the change the session created by one device driver cannot be used or reinitialized by another driver.
--- app/test-crypto-perf/cperf_ops.c | 21 +- app/test-crypto-perf/cperf_test_latency.c | 6 +- .../cperf_test_pmd_cyclecount.c | 5 +- app/test-crypto-perf/cperf_test_throughput.c | 6 +- app/test-crypto-perf/cperf_test_verify.c | 6 +- app/test-crypto-perf/main.c | 29 +- app/test-eventdev/test_perf_common.c | 35 +- app/test-eventdev/test_perf_common.h | 1 - app/test/test_cryptodev.c | 298 +++++------------- app/test/test_cryptodev_blockcipher.c | 16 +- app/test/test_event_crypto_adapter.c | 35 +- app/test/test_ipsec.c | 42 +-- drivers/crypto/armv8/rte_armv8_pmd.c | 21 +- drivers/crypto/armv8/rte_armv8_pmd_ops.c | 32 +- drivers/crypto/bcmfs/bcmfs_sym_session.c | 38 +-- drivers/crypto/bcmfs/bcmfs_sym_session.h | 3 +- drivers/crypto/caam_jr/caam_jr.c | 28 +- drivers/crypto/ccp/ccp_crypto.c | 58 +--- drivers/crypto/ccp/ccp_pmd_ops.c | 32 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 - drivers/crypto/ccp/rte_ccp_pmd.c | 29 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 36 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 31 +- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 54 +--- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 14 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 31 +- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 3 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 37 +-- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 4 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 34 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 41 ++- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 48 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 26 +- drivers/crypto/ipsec_mb/pmd_chacha_poly.c | 4 - drivers/crypto/ipsec_mb/pmd_kasumi.c | 10 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 9 +- drivers/crypto/ipsec_mb/pmd_zuc.c | 4 - drivers/crypto/mlx5/mlx5_crypto.c | 26 +- drivers/crypto/mvsam/rte_mrvl_pmd.c | 8 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 21 +- drivers/crypto/nitrox/nitrox_sym.c | 39 +-- drivers/crypto/null/null_crypto_pmd.c | 19 +- drivers/crypto/null/null_crypto_pmd_ops.c | 33 +- drivers/crypto/null/null_crypto_pmd_private.h | 2 - .../crypto/octeontx/otx_cryptodev_hw_access.h | 1 - drivers/crypto/octeontx/otx_cryptodev_ops.c | 67 +--- drivers/crypto/openssl/openssl_pmd_private.h | 2 - drivers/crypto/openssl/rte_openssl_pmd.c | 24 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 29 +- drivers/crypto/qat/qat_sym.c | 10 +- drivers/crypto/qat/qat_sym.h | 4 +- drivers/crypto/qat/qat_sym_session.c | 40 +-- drivers/crypto/qat/qat_sym_session.h | 6 +- drivers/crypto/scheduler/scheduler_pmd_ops.c | 38 +-- drivers/crypto/virtio/virtio_cryptodev.c | 40 +-- drivers/crypto/virtio/virtio_rxtx.c | 3 +- examples/fips_validation/fips_dev_self_test.c | 30 +- examples/fips_validation/main.c | 35 +- examples/ipsec-secgw/ipsec-secgw.c | 10 +- examples/ipsec-secgw/ipsec.c | 7 +- examples/l2fwd-crypto/main.c | 54 +--- examples/vhost_crypto/main.c | 16 +- lib/cryptodev/cryptodev_pmd.h | 28 +- lib/cryptodev/cryptodev_trace_points.c | 6 - lib/cryptodev/rte_cryptodev.c | 272 ++++++---------- lib/cryptodev/rte_cryptodev.h | 123 ++------ lib/cryptodev/rte_cryptodev_trace.h | 33 +- lib/cryptodev/version.map | 6 - lib/pipeline/rte_table_action.c | 10 +- lib/vhost/rte_vhost_crypto.h | 3 - lib/vhost/vhost_crypto.c | 28 +- 71 files changed, 545 insertions(+), 1657 deletions(-)
app/test-crypto-perf/cperf_ops.c +10 -11
@@ -912,7 +912,6 @@ cperf_create_session(struct rte_mempool *sess_mp, &sess_conf, sess_mp, priv_mp); }#endif - sess =rte_cryptodev_sym_session_create (sess_mp);/* * cipher only */
@@ -937,8 +936,8 @@ cperf_create_session(struct rte_mempool *sess_mp, cipher_xform.cipher.iv.length =0 ; }/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, sess, &cipher_xform, - priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, &cipher_xform, + sess_mp);/* * auth only */
@@ -965,8 +964,8 @@ cperf_create_session(struct rte_mempool *sess_mp, auth_xform.auth.iv.length =0 ; }/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, sess, &auth_xform, - priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, &auth_xform, + sess_mp);/* * cipher and auth */
@@ -1024,13 +1023,13 @@ cperf_create_session(struct rte_mempool *sess_mp,if (options->op_type ==CPERF_CIPHER_THEN_AUTH ) { cipher_xform.next = &auth_xform;/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, - sess, &cipher_xform, priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, + &cipher_xform, sess_mp); }else {/* auth then cipher */ auth_xform.next = &cipher_xform;/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, - sess, &auth_xform, priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, + &auth_xform, sess_mp); } }else {/* options->op_type == CPERF_AEAD */ aead_xform.type =RTE_CRYPTO_SYM_XFORM_AEAD ;
@@ -1050,8 +1049,8 @@ cperf_create_session(struct rte_mempool *sess_mp, options->aead_aad_sz;/* Create crypto session */ -rte_cryptodev_sym_session_init (dev_id, - sess, &aead_xform, priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, &aead_xform, + sess_mp); }return sess;
app/test-crypto-perf/cperf_test_latency.c +2 -4
@@ -44,10 +44,8 @@ static void cperf_latency_test_free(struct cperf_latency_ctx *ctx) {if (ctx) { -if (ctx->sess) { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } +if (ctx->sess) +rte_cryptodev_sym_session_free (ctx->dev_id, ctx->sess);rte_mempool_free (ctx->pool);
app/test-crypto-perf/cperf_test_pmd_cyclecount.c +1 -4
@@ -73,10 +73,7 @@ cperf_pmd_cyclecount_test_free(struct cperf_pmd_cyclecount_ctx *ctx) (struct rte_security_session *)ctx->sess); } else#endif - { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } +rte_cryptodev_sym_session_free (ctx->dev_id, ctx->sess); }rte_mempool_free (ctx->pool);
app/test-crypto-perf/cperf_test_throughput.c +2 -4
@@ -50,10 +50,8 @@ cperf_throughput_test_free(struct cperf_throughput_ctx *ctx) (struct rte_security_session *)ctx->sess); }#endif - else { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } + else + rte_cryptodev_sym_session_free(ctx->dev_id, ctx->sess); }rte_mempool_free (ctx->pool);
app/test-crypto-perf/cperf_test_verify.c +2 -4
@@ -37,10 +37,8 @@ static void cperf_verify_test_free(struct cperf_verify_ctx *ctx) {if (ctx) { -if (ctx->sess) { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } +if (ctx->sess) +rte_cryptodev_sym_session_free (ctx->dev_id, ctx->sess);rte_mempool_free (ctx->pool);
app/test-crypto-perf/main.c +3 -26
@@ -96,35 +96,14 @@ fill_session_pool_socket(int32_t socket_id, uint32_t session_priv_size,char mp_name[RTE_MEMPOOL_NAMESIZE ];struct rte_mempool *sess_mp; -if (session_pool_socket[socket_id].priv_mp ==NULL ) { -snprintf (mp_name,RTE_MEMPOOL_NAMESIZE , -"priv_sess_mp_%u" , socket_id); - - sess_mp =rte_mempool_create (mp_name, - nb_sessions, - session_priv_size, -0 ,0 ,NULL ,NULL ,NULL , -NULL , socket_id, -0 ); - -if (sess_mp ==NULL ) { -printf ("Cannot create pool \"%s\" on socket %d\n" , - mp_name, socket_id); -return -ENOMEM ; - } - -printf ("Allocated pool \"%s\" on socket %d\n" , - mp_name, socket_id); - session_pool_socket[socket_id].priv_mp = sess_mp; - } -if (session_pool_socket[socket_id].sess_mp ==NULL ) {snprintf (mp_name,RTE_MEMPOOL_NAMESIZE ,"sess_mp_%u" , socket_id); sess_mp =rte_cryptodev_sym_session_pool_create (mp_name, - nb_sessions,0 ,0 ,0 , socket_id); + nb_sessions, session_priv_size,0 ,0 , + socket_id);if (sess_mp ==NULL ) { printf("Cannot create pool \"%s\" on socket %d\n",
@@ -135,6 +114,7 @@ fill_session_pool_socket(int32_t socket_id, uint32_t session_priv_size,printf ("Allocated pool \"%s\" on socket %d\n" , mp_name, socket_id); session_pool_socket[socket_id].sess_mp = sess_mp; + session_pool_socket[socket_id].priv_mp = sess_mp; }return 0 ;
@@ -322,12 +302,9 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs)return ret; qp_conf.mp_session = session_pool_socket[socket_id].sess_mp; - qp_conf.mp_session_private = - session_pool_socket[socket_id].priv_mp;if (opts->op_type ==CPERF_ASYM_MODEX ) { qp_conf.mp_session =NULL ; - qp_conf.mp_session_private =NULL ; } ret =rte_cryptodev_configure (cdev_id, &conf);
app/test-eventdev/test_perf_common.c +8 -27
@@ -864,18 +864,13 @@ cryptodev_sym_sess_create(struct prod_data *p, struct test_perf *t) cipher_xform.cipher.op =RTE_CRYPTO_CIPHER_OP_ENCRYPT ; cipher_xform.next =NULL ; - sess =rte_cryptodev_sym_session_create (t->ca_sess_pool); + sess =rte_cryptodev_sym_session_create (p->ca.cdev_id, &cipher_xform, + t->ca_sess_pool);if (sess ==NULL ) {evt_err ("Failed to create sym session" );return NULL ; } -if (rte_cryptodev_sym_session_init (p->ca.cdev_id, sess, &cipher_xform, - t->ca_sess_priv_pool)) { -evt_err ("Failed to init session" ); -return NULL ; - } -return sess; }
@@ -1382,15 +1377,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) goto err; } - t->ca_sess_pool =rte_cryptodev_sym_session_pool_create ( -"ca_sess_pool" , nb_sessions,0 ,0 , -sizeof (union rte_event_crypto_metadata ),SOCKET_ID_ANY ); -if (t->ca_sess_pool ==NULL ) { -evt_err ("Failed to create sym session pool" ); - ret = -ENOMEM ; - goto err; - } - max_session_size =0 ;for (cdev_id =0 ; cdev_id < cdev_count; cdev_id++) {unsigned int session_size;
@@ -1401,12 +1387,11 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) max_session_size = session_size; } - max_session_size +=sizeof (union rte_event_crypto_metadata ); - t->ca_sess_priv_pool =rte_mempool_create ( -"ca_sess_priv_pool" , nb_sessions, max_session_size,0 ,0 ,NULL , -NULL ,NULL ,NULL ,SOCKET_ID_ANY ,0 ); -if (t->ca_sess_priv_pool ==NULL ) { -evt_err ("failed to create sym session private pool" ); + t->ca_sess_pool =rte_cryptodev_sym_session_pool_create ( +"ca_sess_pool" , nb_sessions, max_session_size,0 , +sizeof (union rte_event_crypto_metadata ),SOCKET_ID_ANY ); +if (t->ca_sess_pool ==NULL ) { +evt_err ("Failed to create sym session pool" ); ret = -ENOMEM ; goto err; }
@@ -1446,7 +1431,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) qp_conf.nb_descriptors =NB_CRYPTODEV_DESCRIPTORS ; qp_conf.mp_session = t->ca_sess_pool; - qp_conf.mp_session_private = t->ca_sess_priv_pool;for (qp_id =0 ; qp_id < conf.nb_queue_pairs; qp_id++) { ret = rte_cryptodev_queue_pair_setup(
@@ -1467,7 +1451,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt)rte_mempool_free (t->ca_op_pool);rte_mempool_free (t->ca_sess_pool); -rte_mempool_free (t->ca_sess_priv_pool);rte_mempool_free (t->ca_asym_sess_pool);return ret;
@@ -1492,8 +1475,7 @@ perf_cryptodev_destroy(struct evt_test *test, struct evt_options *opt)for (flow_id =0 ; flow_id < t->nb_flows; flow_id++) { sess = p->ca.crypto_sess[flow_id]; cdev_id = p->ca.cdev_id; -rte_cryptodev_sym_session_clear (cdev_id, sess); -rte_cryptodev_sym_session_free (sess); +rte_cryptodev_sym_session_free (cdev_id, sess); } rte_event_crypto_adapter_queue_pair_del(
@@ -1509,7 +1491,6 @@ perf_cryptodev_destroy(struct evt_test *test, struct evt_options *opt)rte_mempool_free (t->ca_op_pool);rte_mempool_free (t->ca_sess_pool); -rte_mempool_free (t->ca_sess_priv_pool);rte_mempool_free (t->ca_asym_sess_pool); }
app/test-eventdev/test_perf_common.h +0 -1
@@ -70,7 +70,6 @@ struct test_perf { RTE_EVENT_TIMER_ADAPTER_NUM_MAX] __rte_cache_aligned;struct rte_mempool *ca_op_pool;struct rte_mempool *ca_sess_pool; -struct rte_mempool *ca_sess_priv_pool;struct rte_mempool *ca_asym_sess_pool; } __rte_cache_aligned;
app/test/test_cryptodev.c +84 -214
651 lines not on this page · raw message
app/test/test_cryptodev_blockcipher.c +5 -11
@@ -68,7 +68,6 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t,struct rte_mempool *mbuf_pool, struct rte_mempool *op_mpool, struct rte_mempool *sess_mpool, - struct rte_mempool *sess_priv_mpool, uint8_t dev_id, char *test_msg) {
@@ -514,11 +513,9 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t,*/ if (!(t->feature_mask &BLOCKCIPHER_TEST_FEATURE_SESSIONLESS ) && nb_iterates ==0 ) { - sess =rte_cryptodev_sym_session_create (sess_mpool); - - status =rte_cryptodev_sym_session_init (dev_id, sess, - init_xform, sess_priv_mpool); -if (status == -ENOTSUP ) { + sess =rte_cryptodev_sym_session_create (dev_id, init_xform, + sess_mpool); +if (sess ==NULL ) {snprintf (test_msg,BLOCKCIPHER_TEST_MSG_LEN ,"UNSUPPORTED" ); status =TEST_SKIPPED ; goto error_exit;
@@ -801,10 +798,8 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, error_exit:if (!(t->feature_mask &BLOCKCIPHER_TEST_FEATURE_SESSIONLESS )) { -if (sess) { -rte_cryptodev_sym_session_clear (dev_id, sess); -rte_cryptodev_sym_session_free (sess); - } +if (sess) +rte_cryptodev_sym_session_free (dev_id, sess);rte_free (cipher_xform);rte_free (auth_xform); }
@@ -829,7 +824,6 @@ blockcipher_test_case_run(const void *data) p_testsuite_params->mbuf_pool, p_testsuite_params->op_mpool, p_testsuite_params->session_mpool, - p_testsuite_params->session_priv_mpool, p_testsuite_params->valid_devs[0 ], test_msg);return status;
app/test/test_event_crypto_adapter.c +5 -30
@@ -157,7 +157,6 @@ struct event_crypto_adapter_test_params {struct rte_mempool *op_mpool;struct rte_mempool *asym_op_mpool;struct rte_mempool *session_mpool; -struct rte_mempool *session_priv_mpool;struct rte_mempool *asym_sess_mpool;struct rte_cryptodev_config *config;uint8_t crypto_event_port_id;
@@ -307,15 +306,10 @@ test_op_forward_mode(uint8_t session_less) sym_op = op->sym;if (!session_less) { - sess =rte_cryptodev_sym_session_create ( - params.session_mpool); + sess =rte_cryptodev_sym_session_create (TEST_CDEV_ID , + &cipher_xform, params.session_mpool);TEST_ASSERT_NOT_NULL (sess,"Session creation failed\n" ); -/* Create Crypto session*/ - ret =rte_cryptodev_sym_session_init (TEST_CDEV_ID , sess, - &cipher_xform, params.session_priv_mpool); -TEST_ASSERT_SUCCESS (ret,"Failed to init session\n" ); - ret =rte_event_crypto_adapter_caps_get (evdev,TEST_CDEV_ID , &cap);TEST_ASSERT_SUCCESS (ret,"Failed to get adapter capabilities\n" );
@@ -683,8 +677,8 @@ test_op_new_mode(uint8_t session_less) sym_op = op->sym;if (!session_less) { - sess =rte_cryptodev_sym_session_create ( - params.session_mpool); + sess =rte_cryptodev_sym_session_create (TEST_CDEV_ID , + &cipher_xform, params.session_mpool);TEST_ASSERT_NOT_NULL (sess,"Session creation failed\n" ); ret = rte_event_crypto_adapter_caps_get(evdev, TEST_CDEV_ID,
@@ -699,9 +693,6 @@ test_op_new_mode(uint8_t session_less)RTE_CRYPTO_OP_WITH_SESSION , &m_data, sizeof(m_data)); } - ret =rte_cryptodev_sym_session_init (TEST_CDEV_ID , sess, - &cipher_xform, params.session_priv_mpool); -TEST_ASSERT_SUCCESS (ret,"Failed to init session\n" );rte_crypto_op_attach_sym_session (op, sess); }else {
@@ -994,22 +985,12 @@ configure_cryptodev(void) params.session_mpool =rte_cryptodev_sym_session_pool_create ("CRYPTO_ADAPTER_SESSION_MP" , -MAX_NB_SESSIONS ,0 ,0 , +MAX_NB_SESSIONS , session_size,0 ,sizeof (union rte_event_crypto_metadata ),SOCKET_ID_ANY );TEST_ASSERT_NOT_NULL (params.session_mpool,"session mempool allocation failed\n" ); - params.session_priv_mpool =rte_mempool_create ( -"CRYPTO_AD_SESS_MP_PRIV" , -MAX_NB_SESSIONS , - session_size, -0 ,0 ,NULL ,NULL ,NULL , -NULL ,SOCKET_ID_ANY , -0 ); -TEST_ASSERT_NOT_NULL (params.session_priv_mpool, -"session mempool allocation failed\n" ); -rte_cryptodev_info_get (TEST_CDEV_ID , &info); while ((capability = &info.capabilities[i++])->op !=
@@ -1048,7 +1029,6 @@ configure_cryptodev(void) qp_conf.nb_descriptors =DEFAULT_NUM_OPS_INFLIGHT ; qp_conf.mp_session = params.session_mpool; - qp_conf.mp_session_private = params.session_priv_mpool; TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( TEST_CDEV_ID, TEST_CDEV_QP_ID, &qp_conf,
@@ -1410,11 +1390,6 @@ crypto_teardown(void)rte_mempool_free (params.session_mpool); params.session_mpool =NULL ; } -if (params.session_priv_mpool !=NULL ) { -rte_mempool_avail_count (params.session_priv_mpool); -rte_mempool_free (params.session_priv_mpool); - params.session_priv_mpool =NULL ; - }/* Free asym session mempool */ if (params.asym_sess_mpool !=NULL ) {
app/test/test_ipsec.c +7 -35
79 lines not on this page · raw message
drivers/crypto/armv8/rte_armv8_pmd.c +3 -18
@@ -521,34 +521,23 @@ get_session(struct armv8_crypto_qp *qp, struct rte_crypto_op *op)if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {/* get existing session */ if (likely (op->sym->session !=NULL )) { - sess = (struct armv8_crypto_session *) -get_sym_session_private_data ( - op->sym->session, - cryptodev_driver_id); + sess = op->sym->session->data; } }else {/* provide internal session */ -void *_sess =NULL ; -void *_sess_private_data =NULL ; +struct rte_cryptodev_sym_session *_sess =NULL ;if (rte_mempool_get (qp->sess_mp, (void **)&_sess))return NULL ; -if (rte_mempool_get (qp->sess_mp_priv, - (void **)&_sess_private_data)) -return NULL ; - - sess = (struct armv8_crypto_session *)_sess_private_data; + sess = (struct armv8_crypto_session *)_sess->data;if (unlikely (armv8_crypto_set_session_parameters (sess, op->sym->xform) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); -rte_mempool_put (qp->sess_mp_priv, _sess_private_data); sess =NULL ; } op->sym->session = (struct rte_cryptodev_sym_session *)_sess; -set_sym_session_private_data (op->sym->session, - cryptodev_driver_id, _sess_private_data); }if (unlikely (sess ==NULL ))
@@ -674,10 +663,6 @@ process_op(struct armv8_crypto_qp *qp, struct rte_crypto_op *op,/* Free session if a session-less crypto op */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sess,0 ,sizeof (struct armv8_crypto_session )); -memset (op->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - op->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sess);rte_mempool_put (qp->sess_mp, op->sym->session); op->sym->session =NULL ; }
drivers/crypto/armv8/rte_armv8_pmd_ops.c +5 -27
@@ -267,8 +267,7 @@ armv8_crypto_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int armv8_crypto_pmd_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;
@@ -278,43 +277,22 @@ armv8_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev,return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -CDEV_LOG_ERR ( -"Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = sess->data; ret =armv8_crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {ARMV8_CRYPTO_LOG_ERR ("failed configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/** Clear the memory of session so it doesn't leave key material behind */ static void -armv8_crypto_pmd_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -/* Zero out the whole structure */ -if (sess_priv) { -memset (sess_priv,0 ,sizeof (struct armv8_crypto_session )); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +armv8_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}struct rte_cryptodev_ops armv8_crypto_pmd_ops = { .dev_configure = armv8_crypto_pmd_config,
drivers/crypto/bcmfs/bcmfs_sym_session.c +7 -31
@@ -211,8 +211,7 @@ bcmfs_sym_get_session(struct rte_crypto_op *op) }else if (likely (op->sym->session !=NULL )) {/* get existing session */ sess = (struct bcmfs_sym_session *) -get_sym_session_private_data (op->sym->session, - cryptodev_bcmfs_driver_id); + op->sym->session->driver_priv_data; }if (sess ==NULL )
@@ -222,10 +221,9 @@ bcmfs_sym_get_session(struct rte_crypto_op *op) }int -bcmfs_sym_session_configure (struct rte_cryptodev *dev, +bcmfs_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;
@@ -235,45 +233,23 @@ bcmfs_sym_session_configure(struct rte_cryptodev *dev,return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -BCMFS_DP_LOG (ERR , -"Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = (void *)sess->driver_priv_data; ret =crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {BCMFS_DP_LOG (ERR ,"Failed configure session parameters" ); -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/* Clear the memory of session so it doesn't leave key material behind */ void -bcmfs_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -if (sess_priv) { -struct rte_mempool *sess_mp; - -memset (sess_priv,0 ,sizeof (struct bcmfs_sym_session )); - sess_mp =rte_mempool_from_obj (sess_priv); - -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +bcmfs_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}unsigned int bcmfs_sym_session_get_private_size(struct rte_cryptodev *dev __rte_unused)
drivers/crypto/bcmfs/bcmfs_sym_session.h +1 -2
@@ -93,8 +93,7 @@ bcmfs_process_crypto_op(struct rte_crypto_op *op,int bcmfs_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool); +struct rte_cryptodev_sym_session *sess); void bcmfs_sym_session_clear(struct rte_cryptodev *dev,
drivers/crypto/caam_jr/caam_jr.c +6 -22
@@ -1357,8 +1357,7 @@ caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp)switch (op->sess_type) {case RTE_CRYPTO_OP_WITH_SESSION : ses = (struct caam_jr_session *) -get_sym_session_private_data (op->sym->session, - cryptodev_driver_id); + op->sym->session->driver_priv_data;break ;case RTE_CRYPTO_OP_SECURITY_SESSION : ses = (struct caam_jr_session *)
@@ -1692,54 +1691,39 @@ caam_jr_set_session_parameters(struct rte_cryptodev *dev, }static int -caam_jr_sym_session_configure (struct rte_cryptodev *dev, +caam_jr_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;PMD_INIT_FUNC_TRACE (); - -if (rte_mempool_get (mempool, &sess_private_data)) { -CAAM_JR_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } - + sess_private_data = (void *)sess->driver_priv_data;memset (sess_private_data,0 ,sizeof (struct caam_jr_session )); ret =caam_jr_set_session_parameters (dev, xform, sess_private_data);if (ret !=0 ) {CAAM_JR_ERR ("failed to configure session parameters" );/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, sess_private_data); -return 0 ; }/* Clear the memory of session so it doesn't leave key material behind */ static void -caam_jr_sym_session_clear (struct rte_cryptodev *dev, +caam_jr_sym_session_clear(struct rte_cryptodev *dev __rte_unused,struct rte_cryptodev_sym_session *sess) { -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = (void *)sess->driver_priv_data;struct caam_jr_session *s = (struct caam_jr_session *)sess_priv;PMD_INIT_FUNC_TRACE ();if (sess_priv) { -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -rte_free (s->cipher_key.data);rte_free (s->auth_key.data); -memset (s,0 ,sizeof (struct caam_jr_session )); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); } }
drivers/crypto/ccp/ccp_crypto.c +16 -42
142 lines not on this page · raw message
drivers/crypto/ccp/ccp_pmd_ops.c +6 -26
@@ -727,7 +727,6 @@ ccp_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, } qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private;/* mempool for batch info */ qp->batch_mp = rte_mempool_create(
@@ -757,8 +756,7 @@ ccp_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int ccp_pmd_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {int ret;void *sess_private_data;
@@ -769,40 +767,22 @@ ccp_pmd_sym_session_configure(struct rte_cryptodev *dev,return -ENOMEM ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -CCP_LOG_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = (void *)sess->driver_priv_data; + internals = (struct ccp_private *)dev->data->dev_private; ret =ccp_set_session_parameters (sess_private_data, xform, internals);if (ret !=0 ) {CCP_LOG_ERR ("failed configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data);return 0 ; }static void -ccp_pmd_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -if (sess_priv) { -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); - -rte_mempool_put (sess_mp, sess_priv); -memset (sess_priv,0 ,sizeof (struct ccp_session )); -set_sym_session_private_data (sess, index,NULL ); - } -} +ccp_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}struct rte_cryptodev_ops ccp_ops = { .dev_configure = ccp_pmd_config,
drivers/crypto/ccp/ccp_pmd_private.h +0 -2
@@ -78,8 +78,6 @@ struct ccp_qp {/**< Ring for placing process packets */ struct rte_mempool *sess_mp;/**< Session Mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session Private Data Mempool */ struct rte_mempool *batch_mp;/**< Session Mempool for batch info */ struct rte_cryptodev_stats qp_stats;
drivers/crypto/ccp/rte_ccp_pmd.c +8 -21
@@ -56,33 +56,23 @@ get_ccp_session(struct ccp_qp *qp, struct rte_crypto_op *op)if (unlikely (op->sym->session ==NULL ))return NULL ; - sess = (struct ccp_session *) -get_sym_session_private_data ( - op->sym->session, - ccp_cryptodev_driver_id); + sess = (void *)op->sym->session->driver_priv_data; }else if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -void *_sess; -void *_sess_private_data =NULL ; +struct rte_cryptodev_sym_session *_sess;struct ccp_private *internals; -if (rte_mempool_get (qp->sess_mp, &_sess)) -return NULL ; -if (rte_mempool_get (qp->sess_mp, (void **)&_sess_private_data)) +if (rte_mempool_get (qp->sess_mp, (void **)&_sess))return NULL ; - sess = (struct ccp_session *)_sess_private_data; + sess = (void *)_sess->driver_priv_data; internals = (struct ccp_private *)qp->dev->data->dev_private;if (unlikely (ccp_set_session_parameters (sess, op->sym->xform, internals) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); -rte_mempool_put (qp->sess_mp_priv, _sess_private_data); sess =NULL ; } - op->sym->session = (struct rte_cryptodev_sym_session *)_sess; -set_sym_session_private_data (op->sym->session, - ccp_cryptodev_driver_id, - _sess_private_data); + op->sym->session = _sess; }return sess;
@@ -161,13 +151,10 @@ ccp_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,for (i =0 ; i < nb_dequeued; i++)if (unlikely (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS )) { -struct ccp_session *sess = (struct ccp_session *) -get_sym_session_private_data ( - ops[i]->sym->session, - ccp_cryptodev_driver_id); +struct ccp_session *sess = + (void *)ops[i]->sym->session->driver_priv_data; -rte_mempool_put (qp->sess_mp_priv, - sess); +memset (sess,0 ,sizeof (*sess));rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ;
drivers/crypto/cnxk/cn10k_cryptodev_ops.c +14 -22
@@ -22,24 +22,23 @@static inline struct cnxk_se_sess *cn10k_cpt_sym_temp_sess_create (struct cnxk_cpt_qp *qp,struct rte_crypto_op *op) { -const int driver_id = cn10k_cryptodev_driver_id;struct rte_crypto_sym_op *sym_op = op->sym;struct rte_cryptodev_sym_session *sess;struct cnxk_se_sess *priv;int ret;/* Create temporary session */ - sess =rte_cryptodev_sym_session_create (qp->sess_mp); -if (sess ==NULL ) +if (rte_mempool_get (qp->sess_mp, (void **)&sess) <0 )return NULL ; - ret =sym_session_configure (qp->lf.roc_cpt, driver_id, sym_op->xform, - sess, qp->sess_mp_priv); -if (ret) + ret =sym_session_configure (qp->lf.roc_cpt, sym_op->xform, + sess); +if (ret) { +rte_mempool_put (qp->sess_mp, (void *)sess); goto sess_put; + } - priv =get_sym_session_private_data (sess, driver_id); - + priv = (void *)sess->driver_priv_data; sym_op->session = sess;return priv;
@@ -111,8 +110,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],return 0 ; w7 = sec_sess->sa.inst.w7; }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { - sess =get_sym_session_private_data ( - sym_op->session, cn10k_cryptodev_driver_id); + sess = (void *)sym_op->session->driver_priv_data; ret =cpt_sym_inst_fill (qp, op, sess, infl_req, &inst[0 ]);if (unlikely (ret))
@@ -128,8 +126,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], ret =cpt_sym_inst_fill (qp, op, sess, infl_req, &inst[0 ]);if (unlikely (ret)) { -sym_session_clear (cn10k_cryptodev_driver_id, - op->sym->session); +sym_session_clear (op->sym->session);rte_mempool_put (qp->sess_mp, op->sym->session);return 0 ; }
@@ -296,8 +293,9 @@ cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, }else if (sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - sess, cn10k_cryptodev_driver_id); + priv = (void *)( + ((struct rte_cryptodev_sym_session *)sess)-> + driver_priv_data); priv->qp = qp; priv->cpt_inst_w2 = w2; }else
@@ -334,8 +332,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - op->sym->session, cn10k_cryptodev_driver_id); + priv = (void *)op->sym->session->driver_priv_data; *qp = priv->qp; *w2 = priv->cpt_inst_w2; }else {
@@ -543,7 +540,6 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, {const uint8_t uc_compcode = res->uc_compcode;const uint8_t compcode = res->compcode; -unsigned int sz; cop->status =RTE_CRYPTO_OP_STATUS_SUCCESS ;
@@ -620,11 +616,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, temp_sess_free:if (unlikely (cop->sess_type ==RTE_CRYPTO_OP_SESSIONLESS )) {if (cop->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) { -sym_session_clear (cn10k_cryptodev_driver_id, - cop->sym->session); - sz =rte_cryptodev_sym_get_existing_header_session_size ( - cop->sym->session); -memset (cop->sym->session,0 , sz); +sym_session_clear (cop->sym->session);rte_mempool_put (qp->sess_mp, cop->sym->session); cop->sym->session =NULL ; }
drivers/crypto/cnxk/cn9k_cryptodev_ops.c +10 -21
@@ -56,23 +56,20 @@ cn9k_cpt_sec_inst_fill(struct rte_crypto_op *op,static inline struct cnxk_se_sess *cn9k_cpt_sym_temp_sess_create (struct cnxk_cpt_qp *qp,struct rte_crypto_op *op) { -const int driver_id = cn9k_cryptodev_driver_id;struct rte_crypto_sym_op *sym_op = op->sym;struct rte_cryptodev_sym_session *sess;struct cnxk_se_sess *priv;int ret;/* Create temporary session */ - sess =rte_cryptodev_sym_session_create (qp->sess_mp); -if (sess ==NULL ) +if (rte_mempool_get (qp->sess_mp, (void **)&sess) <0 )return NULL ; - ret =sym_session_configure (qp->lf.roc_cpt, driver_id, sym_op->xform, - sess, qp->sess_mp_priv); + ret =sym_session_configure (qp->lf.roc_cpt, sym_op->xform, sess);if (ret) goto sess_put; - priv =get_sym_session_private_data (sess, driver_id); + priv = (void *)sess->driver_priv_data; sym_op->session = sess;
@@ -95,8 +92,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { sym_op = op->sym; - sess =get_sym_session_private_data ( - sym_op->session, cn9k_cryptodev_driver_id); + sess = (void *)sym_op->session->driver_priv_data; ret =cpt_sym_inst_fill (qp, op, sess, infl_req, inst); inst->w7.u64 = sess->cpt_inst_w7; }else if (op->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION )
@@ -110,8 +106,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, ret =cpt_sym_inst_fill (qp, op, sess, infl_req, inst);if (unlikely (ret)) { -sym_session_clear (cn9k_cryptodev_driver_id, - op->sym->session); +sym_session_clear (op->sym->session);rte_mempool_put (qp->sess_mp, op->sym->session); } inst->w7.u64 = sess->cpt_inst_w7;
@@ -349,8 +344,9 @@ cn9k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, }else if (sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - sess, cn9k_cryptodev_driver_id); + priv = (void *)(( + (struct rte_cryptodev_sym_session *)sess)-> + driver_priv_data); priv->qp = qp; priv->cpt_inst_w2 = w2; }else
@@ -387,8 +383,7 @@ cn9k_ca_meta_info_extract(struct rte_crypto_op *op, }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - op->sym->session, cn9k_cryptodev_driver_id); + priv = (void *)op->sym->session->driver_priv_data; *qp = priv->qp; inst->w2.u64 = priv->cpt_inst_w2; }else {
@@ -583,8 +578,6 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop,struct cpt_inflight_req *infl_req, struct cpt_cn9k_res_s *res) { - unsigned int sz; - if (likely(res->compcode == CPT_COMP_GOOD)) { if (unlikely(res->uc_compcode)) { if (res->uc_compcode == ROC_SE_ERR_GC_ICV_MISCOMPARE)
@@ -645,11 +638,7 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, temp_sess_free:if (unlikely (cop->sess_type ==RTE_CRYPTO_OP_SESSIONLESS )) {if (cop->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) { -sym_session_clear (cn9k_cryptodev_driver_id, - cop->sym->session); - sz =rte_cryptodev_sym_get_existing_header_session_size ( - cop->sym->session); -memset (cop->sym->session,0 , sz); +sym_session_clear (cop->sym->session);rte_mempool_put (qp->sess_mp, cop->sym->session); cop->sym->session =NULL ; }
drivers/crypto/cnxk/cnxk_cryptodev_ops.c +9 -45
102 lines not on this page · raw message
drivers/crypto/cnxk/cnxk_cryptodev_ops.h +5 -9
@@ -78,8 +78,6 @@ struct cnxk_cpt_qp {/**< Crypto adapter related info */ struct rte_mempool *sess_mp;/**< Session mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session private data mempool */ }; int cnxk_cpt_dev_config(struct rte_cryptodev *dev,
@@ -104,18 +102,16 @@ unsigned int cnxk_cpt_sym_session_get_size(struct rte_cryptodev *dev);int cnxk_cpt_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *pool); +struct rte_cryptodev_sym_session *sess); -int sym_session_configure (struct roc_cpt *roc_cpt,int driver_id, +int sym_session_configure (struct roc_cpt *roc_cpt,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *pool); +struct rte_cryptodev_sym_session *sess);void cnxk_cpt_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess); +struct rte_cryptodev_sym_session *sess); -void sym_session_clear (int driver_id,struct rte_cryptodev_sym_session *sess); +void sym_session_clear (struct rte_cryptodev_sym_session *sess); unsigned int cnxk_ae_session_size_get(struct rte_cryptodev *dev __rte_unused);
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +7 -24
@@ -1379,8 +1379,7 @@ build_sec_fd(struct rte_crypto_op *op,dpaa2_sec_session *sess;if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa2_sec_session *)get_sym_session_private_data ( - op->sym->session, cryptodev_driver_id); + sess = (dpaa2_sec_session *)op->sym->session->driver_priv_data;#ifdef RTE_LIB_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa2_sec_session *)get_sec_session_private_data(
@@ -1678,8 +1677,7 @@ dpaa2_sec_dump(struct rte_crypto_op *op)struct rte_crypto_sym_op *sym_op;if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa2_sec_session *)get_sym_session_private_data ( - op->sym->session, cryptodev_driver_id); + sess = (dpaa2_sec_session *)op->sym->session->driver_priv_data;#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa2_sec_session *)get_sec_session_private_data(
@@ -3754,51 +3752,36 @@ dpaa2_sec_security_session_destroy(void *dev __rte_unused, }#endif static int -dpaa2_sec_sym_session_configure (struct rte_cryptodev *dev, +dpaa2_sec_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *sess_private_data; +void *sess_private_data = (void *)sess->driver_priv_data;int ret; -if (rte_mempool_get (mempool, &sess_private_data)) { -DPAA2_SEC_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } - ret =dpaa2_sec_set_session_parameters (xform, sess_private_data);if (ret !=0 ) {DPAA2_SEC_ERR ("Failed to configure session parameters" );/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/** Clear the memory of session so it doesn't leave key material behind */ static void -dpaa2_sec_sym_session_clear (struct rte_cryptodev *dev, +dpaa2_sec_sym_session_clear(struct rte_cryptodev *dev __rte_unused,struct rte_cryptodev_sym_session *sess) {PMD_INIT_FUNC_TRACE (); -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = (void *)sess->driver_priv_data;dpaa2_sec_session *s = (dpaa2_sec_session *)sess_priv;if (sess_priv) {rte_free (s->ctxt);rte_free (s->cipher_key.data);rte_free (s->auth_key.data); -memset (s,0 ,sizeof (dpaa2_sec_session)); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); } }
drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c +1 -2
@@ -1012,8 +1012,7 @@ dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess = (dpaa2_sec_session *)get_sec_session_private_data ( session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa2_sec_session *)get_sym_session_private_data ( - session_ctx.crypto_sess, cryptodev_driver_id); + sess = (void *)session_ctx.crypto_sess->driver_priv_data; else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa2_sec_raw_dequeue_burst;
drivers/crypto/dpaa_sec/dpaa_sec.c +7 -30
@@ -670,10 +670,7 @@ dpaa_sec_dump(struct dpaa_sec_op_ctx *ctx, struct dpaa_sec_qp *qp)struct qm_sg_entry sg[2 ];if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa_sec_session *) -get_sym_session_private_data ( - op->sym->session, - dpaa_cryptodev_driver_id); + sess = (dpaa_sec_session *)op->sym->session->driver_priv_data;#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa_sec_session *)
@@ -1927,10 +1924,8 @@ dpaa_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops,switch (op->sess_type) {case RTE_CRYPTO_OP_WITH_SESSION : - ses = (dpaa_sec_session *) -get_sym_session_private_data ( - op->sym->session, - dpaa_cryptodev_driver_id); + ses = (void *) + op->sym->session->driver_priv_data;break ;#ifdef RTE_LIB_SECURITY case RTE_CRYPTO_OP_SECURITY_SESSION :
@@ -2676,31 +2671,19 @@ dpaa_sec_set_session_parameters(struct rte_cryptodev *dev,static int dpaa_sec_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *sess_private_data; +void *sess_private_data = (void *)sess->driver_priv_data;int ret;PMD_INIT_FUNC_TRACE (); -if (rte_mempool_get (mempool, &sess_private_data)) { -DPAA_SEC_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } - ret =dpaa_sec_set_session_parameters (dev, xform, sess_private_data);if (ret !=0 ) {DPAA_SEC_ERR ("failed to configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); - ret =dpaa_sec_prep_cdb (sess_private_data);if (ret) {DPAA_SEC_ERR ("Unable to prepare sec cdb" );
@@ -2714,7 +2697,6 @@ static inline void free_session_memory(struct rte_cryptodev *dev, dpaa_sec_session *s) { struct dpaa_sec_dev_private *qi = dev->data->dev_private; -struct rte_mempool *sess_mp =rte_mempool_from_obj ((void *)s);uint8_t i;for (i =0 ; i <MAX_DPAA_CORES ; i++) {
@@ -2724,7 +2706,6 @@ free_session_memory(struct rte_cryptodev *dev, dpaa_sec_session *s) s->qp[i] =NULL ; }free_session_data (s); -rte_mempool_put (sess_mp, (void *)s); }/** Clear the memory of session so it doesn't leave key material behind */
@@ -2733,14 +2714,10 @@ dpaa_sec_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) {PMD_INIT_FUNC_TRACE (); -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = (void *)sess->driver_priv_data;dpaa_sec_session *s = (dpaa_sec_session *)sess_priv; -if (sess_priv) { -free_session_memory (dev, s); -set_sym_session_private_data (sess, index,NULL ); - } +free_session_memory (dev, s); }#ifdef RTE_LIB_SECURITY
drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c +2 -2
@@ -1017,8 +1017,8 @@ dpaa_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess = (dpaa_sec_session *)get_sec_session_private_data ( session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa_sec_session *)get_sym_session_private_data ( - session_ctx.crypto_sess, dpaa_cryptodev_driver_id); + sess = (dpaa_sec_session *) + session_ctx.crypto_sess->driver_priv_data; else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa_sec_raw_dequeue_burst;
drivers/crypto/ipsec_mb/ipsec_mb_ops.c +6 -28
@@ -264,7 +264,6 @@ ipsec_mb_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, qp->pmd_type = internals->pmd_type; qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private; qp->ingress_queue =ipsec_mb_qp_create_processed_ops_ring (qp, qp_conf->nb_descriptors, socket_id);
@@ -312,9 +311,8 @@ ipsec_mb_sym_session_get_size(struct rte_cryptodev *dev)int ipsec_mb_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess,struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *sess_private_data;struct ipsec_mb_dev_private *internals = dev->data->dev_private;struct ipsec_mb_internals *pmd_data = &ipsec_mb_pmds[internals->pmd_type];
@@ -330,42 +328,22 @@ ipsec_mb_sym_session_configure(return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -IPSEC_MB_LOG (ERR ,"Couldn't get object from session mempool" ); -free_mb_mgr (mb_mgr); -return -ENOMEM ; - } - - ret = (*pmd_data->session_configure)(mb_mgr, sess_private_data, xform); + ret = (*pmd_data->session_configure)(mb_mgr, + (void *)sess->driver_priv_data, xform);if (ret !=0 ) {IPSEC_MB_LOG (ERR ,"failed configure session parameters" );/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);free_mb_mgr (mb_mgr);return ret; } -set_sym_session_private_data (sess, dev->driver_id, sess_private_data); -free_mb_mgr (mb_mgr);return 0 ; }/** Clear the session memory */ void -ipsec_mb_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -/* Zero out the whole structure */ -if (sess_priv) { -memset (sess_priv,0 ,ipsec_mb_sym_session_get_size (dev)); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); - -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +ipsec_mb_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}
drivers/crypto/ipsec_mb/ipsec_mb_private.h +19 -22
85 lines not on this page · raw message
drivers/crypto/ipsec_mb/pmd_aesni_gcm.c +14 -34
90 lines not on this page · raw message
drivers/crypto/ipsec_mb/pmd_aesni_mb.c +3 -23
@@ -1710,8 +1710,6 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) {struct rte_crypto_op *op = (struct rte_crypto_op *)job->user_data;struct aesni_mb_session *sess =NULL ; -uint32_t driver_id =ipsec_mb_get_driver_id ( -IPSEC_MB_PMD_TYPE_AESNI_MB );#ifdef AESNI_MB_DOCSIS_SEC_ENABLED uint8_t is_docsis_sec =0 ;
@@ -1725,15 +1723,7 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) sess =get_sec_session_private_data (op->sym->sec_session); } else#endif - { - sess =get_sym_session_private_data (op->sym->session, - driver_id); - } - -if (unlikely (sess ==NULL )) { - op->status =RTE_CRYPTO_OP_STATUS_INVALID_SESSION ; -return op; - } + sess = (void *)op->sym->session->driver_priv_data;if (likely (op->status ==RTE_CRYPTO_OP_STATUS_NOT_PROCESSED )) {switch (job->status) {
@@ -1771,10 +1761,6 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job)/* Free session if a session-less crypto op */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sess,0 ,sizeof (struct aesni_mb_session )); -memset (op->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - op->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sess);rte_mempool_put (qp->sess_mp, op->sym->session); op->sym->session =NULL ; }
@@ -2028,7 +2014,7 @@ verify_sync_dgst(struct rte_crypto_sym_vec *vec, }static uint32_t -aesni_mb_process_bulk (struct rte_cryptodev *dev, +aesni_mb_process_bulk(struct rte_cryptodev *dev __rte_unused,struct rte_cryptodev_sym_session *sess,union rte_crypto_sym_ofs sofs,struct rte_crypto_sym_vec *vec) {
@@ -2037,15 +2023,9 @@ aesni_mb_process_bulk(struct rte_cryptodev *dev,void *buf;IMB_JOB *job;IMB_MGR *mb_mgr; -struct aesni_mb_session *s; +struct aesni_mb_session *s = (void *)sess->driver_priv_data;uint8_t tmp_dgst[vec->num][DIGEST_LENGTH_MAX ]; - s =get_sym_session_private_data (sess, dev->driver_id); -if (s ==NULL ) { -ipsec_mb_fill_error_code (vec,EINVAL ); -return 0 ; - } -/* get per-thread MB MGR, create one if needed */ mb_mgr =get_per_thread_mb_mgr ();if (unlikely (mb_mgr ==NULL ))
drivers/crypto/ipsec_mb/pmd_chacha_poly.c +0 -4
@@ -290,10 +290,6 @@ handle_completed_chacha20_poly1305_crypto_op(struct ipsec_mb_qp *qp,/* Free session if a session-less crypto op */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sess,0 ,sizeof (struct chacha20_poly1305_session )); -memset (op->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - op->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sess);rte_mempool_put (qp->sess_mp, op->sym->session); op->sym->session =NULL ; }
drivers/crypto/ipsec_mb/pmd_kasumi.c +3 -7
@@ -231,11 +231,6 @@ process_ops(struct rte_crypto_op **ops, struct kasumi_session *session,/* Free session if a session-less crypto op. */ if (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (session,0 ,sizeof (struct kasumi_session )); -memset ( - ops[i]->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - ops[i]->sym->session)); -rte_mempool_put (qp->sess_mp_priv, session);rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ; }
@@ -287,8 +282,9 @@ process_op_bit(struct rte_crypto_op *op, struct kasumi_session *session,/* Free session if a session-less crypto op. */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -memset (op->sym->session,0 ,sizeof (struct kasumi_session )); -rte_cryptodev_sym_session_free (op->sym->session); +memset (op->sym->session->driver_priv_data,0 , +sizeof (struct kasumi_session )); +rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ; }return processed_op;
drivers/crypto/ipsec_mb/pmd_snow3g.c +3 -6
@@ -362,10 +362,6 @@ process_ops(struct rte_crypto_op **ops, struct snow3g_session *session,/* Free session if a session-less crypto op. */ if (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (session,0 ,sizeof (struct snow3g_session )); -memset (ops[i]->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - ops[i]->sym->session)); -rte_mempool_put (qp->sess_mp_priv, session);rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ; }
@@ -417,8 +413,9 @@ process_op_bit(struct rte_crypto_op *op, struct snow3g_session *session,/* Free session if a session-less crypto op. */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -memset (op->sym->session,0 ,sizeof (struct snow3g_session )); -rte_cryptodev_sym_session_free (op->sym->session); +memset (op->sym->session->driver_priv_data,0 , +sizeof (struct snow3g_session )); +rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ; }
drivers/crypto/ipsec_mb/pmd_zuc.c +0 -4
@@ -239,10 +239,6 @@ process_ops(struct rte_crypto_op **ops, enum ipsec_mb_operation op_type,/* Free session if a session-less crypto op. */ if (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sessions[i],0 ,sizeof (struct zuc_session )); -memset (ops[i]->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - ops[i]->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sessions[i]);rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ; }
drivers/crypto/mlx5/mlx5_crypto.c +6 -20
@@ -171,14 +171,13 @@ mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int mlx5_crypto_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *session, -struct rte_mempool *mp) +struct rte_cryptodev_sym_session *session) {struct mlx5_crypto_priv *priv = dev->data->dev_private; -struct mlx5_crypto_session *sess_private_data; +struct mlx5_crypto_session *sess_private_data = + (void *)session->driver_priv_data;struct rte_crypto_cipher_xform *cipher;uint8_t encryption_order; -int ret;if (unlikely (xform->next !=NULL )) {DRV_LOG (ERR ,"Xform next is not supported." );
@@ -189,17 +188,9 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev,DRV_LOG (ERR ,"Only AES-XTS algorithm is supported." );return -ENOTSUP ; } - ret =rte_mempool_get (mp, (void *)&sess_private_data); -if (ret !=0 ) { -DRV_LOG (ERR , -"Failed to get session %p private data from mempool." , - sess_private_data); -return -ENOMEM ; - } cipher = &xform->cipher; sess_private_data->dek =mlx5_crypto_dek_prepare (priv, cipher);if (sess_private_data->dek ==NULL ) { -rte_mempool_put (mp, sess_private_data);DRV_LOG (ERR ,"Failed to prepare dek." );return -ENOMEM ; }
@@ -239,8 +230,6 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev, sess_private_data->dek_id =rte_cpu_to_be_32 (sess_private_data->dek->obj->id &0xffffff ); -set_sym_session_private_data (session, dev->driver_id, - sess_private_data);DRV_LOG (DEBUG ,"Session %p was configured." , sess_private_data);return 0 ; }
@@ -250,16 +239,13 @@ mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) {struct mlx5_crypto_priv *priv = dev->data->dev_private; -struct mlx5_crypto_session *spriv =get_sym_session_private_data (sess, - dev->driver_id); +struct mlx5_crypto_session *spriv = (void *)sess->driver_priv_data;if (unlikely (spriv ==NULL )) {DRV_LOG (ERR ,"Failed to get session %p private data." , spriv);return ; }mlx5_crypto_dek_destroy (priv, spriv->dek); -set_sym_session_private_data (sess, dev->driver_id,NULL ); -rte_mempool_put (rte_mempool_from_obj (spriv), spriv);DRV_LOG (DEBUG ,"Session %p was cleared." , spriv); }
@@ -369,8 +355,8 @@ mlx5_crypto_wqe_set(struct mlx5_crypto_priv *priv,struct rte_crypto_op *op, struct mlx5_umr_wqe *umr) { - struct mlx5_crypto_session *sess = get_sym_session_private_data - (op->sym->session, mlx5_crypto_driver_id); + struct mlx5_crypto_session *sess = + (void *)op->sym->session->driver_priv_data;struct mlx5_wqe_cseg *cseg = &umr->ctr;struct mlx5_wqe_mkey_cseg *mkc = &umr->mkc;struct mlx5_wqe_dseg *klms = &umr->kseg[0 ];
drivers/crypto/mvsam/rte_mrvl_pmd.c +1 -7
@@ -597,13 +597,7 @@ mrvl_request_prepare_crp(struct sam_cio_op_params *request,return -EINVAL ; } - sess = (struct mrvl_crypto_session *)get_sym_session_private_data ( - op->sym->session, - cryptodev_driver_id); -if (unlikely (sess ==NULL )) { -MRVL_LOG (ERR ,"Session was not created for this device!" ); -return -EINVAL ; - } + sess = (struct mrvl_crypto_session *)op->sym->session->data; request->sa = sess->sam_sess; request->cookie = op;
drivers/crypto/mvsam/rte_mrvl_pmd_ops.c +3 -18
@@ -736,8 +736,7 @@ mrvl_crypto_pmd_sym_session_get_size(__rte_unused struct rte_cryptodev *dev)static int mrvl_crypto_pmd_sym_session_configure(__rte_unused struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mp) +struct rte_cryptodev_sym_session *sess) {struct mrvl_crypto_session *mrvl_sess;void *sess_private_data;
@@ -748,23 +747,15 @@ mrvl_crypto_pmd_sym_session_configure(__rte_unused struct rte_cryptodev *dev,return -EINVAL ; } -if (rte_mempool_get (mp, &sess_private_data)) { -CDEV_LOG_ERR ("Couldn't get object from session mempool." ); -return -ENOMEM ; - } - + sess_private_data = sess->data;memset (sess_private_data,0 ,sizeof (struct mrvl_crypto_session )); ret =mrvl_crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {MRVL_LOG (ERR ,"Failed to configure session parameters!" ); - -/* Return session to mempool */ -rte_mempool_put (mp, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, sess_private_data); mrvl_sess = (struct mrvl_crypto_session *)sess_private_data; if (sam_session_create(&mrvl_sess->sam_sess_params,
@@ -791,8 +782,7 @@ mrvl_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) { -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = sess->data;/* Zero out the whole structure */ if (sess_priv) {
@@ -803,11 +793,6 @@ mrvl_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev, sam_session_destroy(mrvl_sess->sam_sess) < 0) {MRVL_LOG (ERR ,"Error while destroying session!" ); } - -memset (mrvl_sess,0 ,sizeof (struct mrvl_crypto_session )); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); } }
drivers/crypto/nitrox/nitrox_sym.c +8 -31
@@ -530,24 +530,16 @@ configure_aead_ctx(struct rte_crypto_aead_xform *xform, }static int -nitrox_sym_dev_sess_configure (struct rte_cryptodev *cdev, +nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *mp_obj; -struct nitrox_crypto_ctx *ctx; +struct nitrox_crypto_ctx *ctx = (void *)sess->driver_priv_data;struct rte_crypto_cipher_xform *cipher_xform =NULL ;struct rte_crypto_auth_xform *auth_xform =NULL ;struct rte_crypto_aead_xform *aead_xform =NULL ;int ret = -EINVAL ; -if (rte_mempool_get (mempool, &mp_obj)) { -NITROX_LOG (ERR ,"Couldn't allocate context\n" ); -return -ENOMEM ; - } - - ctx = mp_obj; ctx->nitrox_chain =get_crypto_chain_order (xform);switch (ctx->nitrox_chain) {case NITROX_CHAIN_CIPHER_ONLY :
@@ -585,38 +577,23 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev, goto err; } - ctx->iova =rte_mempool_virt2iova (ctx); -set_sym_session_private_data (sess, cdev->driver_id, ctx); + ctx->iova = sess->driver_priv_data_iova;return 0 ; err: -rte_mempool_put (mempool, mp_obj);return ret; }static void -nitrox_sym_dev_sess_clear (struct rte_cryptodev *cdev, -struct rte_cryptodev_sym_session *sess) -{ -struct nitrox_crypto_ctx *ctx =get_sym_session_private_data (sess, - cdev->driver_id); -struct rte_mempool *sess_mp; - -if (!ctx) -return ; - -memset (ctx,0 ,sizeof (*ctx)); - sess_mp =rte_mempool_from_obj (ctx); -set_sym_session_private_data (sess, cdev->driver_id,NULL ); -rte_mempool_put (sess_mp, ctx); -} +nitrox_sym_dev_sess_clear(struct rte_cryptodev *cdev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}static struct nitrox_crypto_ctx *get_crypto_ctx (struct rte_crypto_op *op) {if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {if (likely (op->sym->session)) -return get_sym_session_private_data (op->sym->session, - nitrox_sym_drv_id); +return (void *)op->sym->session->driver_priv_data; }return NULL ;
drivers/crypto/null/null_crypto_pmd.c +5 -14
@@ -58,7 +58,7 @@ process_op(const struct null_crypto_qp *qp, struct rte_crypto_op *op,if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (op->sym->session,0 ,sizeof (struct null_crypto_session )); -rte_cryptodev_sym_session_free (op->sym->session); +rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ; }
@@ -78,30 +78,21 @@ get_session(struct null_crypto_qp *qp, struct rte_crypto_op *op)if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {if (likely (sym_op->session !=NULL )) sess = (struct null_crypto_session *) -get_sym_session_private_data ( - sym_op->session, cryptodev_driver_id); + sym_op->session->driver_priv_data; }else { -void *_sess =NULL ; -void *_sess_private_data =NULL ; +struct rte_cryptodev_sym_session *_sess =NULL ;if (rte_mempool_get (qp->sess_mp, (void **)&_sess))return NULL ; -if (rte_mempool_get (qp->sess_mp_priv, - (void **)&_sess_private_data)) -return NULL ; - - sess = (struct null_crypto_session *)_sess_private_data; + sess = (struct null_crypto_session *)_sess->driver_priv_data;if (unlikely (null_crypto_set_session_parameters (sess, sym_op->xform) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); -rte_mempool_put (qp->sess_mp_priv, _sess_private_data); sess =NULL ; } - sym_op->session = (struct rte_cryptodev_sym_session *)_sess; -set_sym_session_private_data (op->sym->session, - cryptodev_driver_id, _sess_private_data); + sym_op->session = _sess; }return sess;
drivers/crypto/null/null_crypto_pmd_ops.c +5 -28
@@ -233,7 +233,6 @@ null_crypto_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, } qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private;memset (&qp->qp_stats,0 ,sizeof (qp->qp_stats));
@@ -256,8 +255,7 @@ null_crypto_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int null_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mp) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;
@@ -267,43 +265,22 @@ null_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused,return -EINVAL ; } -if (rte_mempool_get (mp, &sess_private_data)) { -NULL_LOG (ERR , -"Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = (void *)sess->driver_priv_data; ret =null_crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {NULL_LOG (ERR ,"failed configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mp, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/** Clear the memory of session so it doesn't leave key material behind */ static void -null_crypto_pmd_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -/* Zero out the whole structure */ -if (sess_priv) { -memset (sess_priv,0 ,sizeof (struct null_crypto_session )); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +null_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}static struct rte_cryptodev_ops pmd_ops = { .dev_configure = null_crypto_pmd_config,
drivers/crypto/null/null_crypto_pmd_private.h +0 -2
@@ -31,8 +31,6 @@ struct null_crypto_qp {/**< Ring for placing process packets */ struct rte_mempool *sess_mp;/**< Session Mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session Mempool */ struct rte_cryptodev_stats qp_stats;/**< Queue pair statistics */ } __rte_cache_aligned;
drivers/crypto/octeontx/otx_cryptodev_hw_access.h +0 -1
@@ -49,7 +49,6 @@ struct cpt_instance {uint32_t queue_id;uintptr_t rsvd;struct rte_mempool *sess_mp; -struct rte_mempool *sess_mp_priv;struct cpt_qp_meta_info meta_info;uint8_t ca_enabled; };
drivers/crypto/octeontx/otx_cryptodev_ops.c +17 -50
166 lines not on this page · raw message
drivers/crypto/openssl/openssl_pmd_private.h +0 -2
@@ -70,8 +70,6 @@ struct openssl_qp {/**< Ring for placing process packets */ struct rte_mempool *sess_mp;/**< Session Mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session Private Data Mempool */ struct rte_cryptodev_stats stats;/**< Queue pair statistics */ uint8_t temp_digest[DIGEST_LENGTH_MAX ];
drivers/crypto/openssl/rte_openssl_pmd.c +6 -18
@@ -887,10 +887,8 @@ get_session(struct openssl_qp *qp, struct rte_crypto_op *op)if (op->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) {/* get existing session */ if (likely (op->sym->session !=NULL )) - sess = (struct openssl_session *) -get_sym_session_private_data ( - op->sym->session, - cryptodev_driver_id); + sess = (void *) + op->sym->session->driver_priv_data; }else {if (likely (op->asym->session !=NULL )) asym_sess = (struct openssl_asym_session *)
@@ -901,32 +899,26 @@ get_session(struct openssl_qp *qp, struct rte_crypto_op *op)return asym_sess; } }else { +struct rte_cryptodev_sym_session *_sess;/* sessionless asymmetric not supported */ if (op->type ==RTE_CRYPTO_OP_TYPE_ASYMMETRIC )return NULL ;/* provide internal session */ -void *_sess =rte_cryptodev_sym_session_create (qp->sess_mp); -void *_sess_private_data =NULL ; +rte_mempool_get (qp->sess_mp, (void **)&_sess);if (_sess ==NULL )return NULL ; -if (rte_mempool_get (qp->sess_mp_priv, - (void **)&_sess_private_data)) -return NULL ; - - sess = (struct openssl_session *)_sess_private_data; + sess = (struct openssl_session *)_sess->driver_priv_data;if (unlikely (openssl_set_session_parameters (sess, op->sym->xform) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); -rte_mempool_put (qp->sess_mp_priv, _sess_private_data); sess =NULL ; }
Lines of this file are not shown. View the raw message.
drivers/crypto/openssl/rte_openssl_pmd_ops.c +5 -24
Lines of this file are not shown. View the raw message.
drivers/crypto/scheduler/scheduler_pmd_ops.c +6 -32
Lines of this file are not shown. View the raw message.
drivers/crypto/virtio/virtio_cryptodev.c +7 -33
Lines of this file are not shown. View the raw message.
examples/fips_validation/fips_dev_self_test.c +5 -25
Lines of this file are not shown. View the raw message.
examples/l2fwd-crypto/main.c +6 -48
54 lines not on this page · raw message
lib/cryptodev/rte_cryptodev.c +95 -177
272 lines not on this page · raw message
lib/cryptodev/rte_cryptodev.h +33 -90
123 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.34.1
Part of the diff is not shown here. View the raw message.
RE: [EXT] [PATCH 1/3] cryptodev: rework session framework
Quoted 19 lines
As per current design,
rte_cryptodev_sym_session_create()andrte_cryptodev_sym_session_init()use separate mempool objects for a single session.
And structurerte_cryptodev_sym_sessionis not directly used by the application, it may cause ABI breakage if the structure is modified in future.To address these two issues, the
rte_cryptodev_sym_session_createwill take one mempool object that the session and session private data are virtually/physically contiguous, and initializes both fields. The APIrte_cryptodev_sym_session_initis removed.
rte_cryptodev_sym_session_createwill now return an opaque session pointer which will be used by the app and other APIs.In data path, opaque session pointer is attached to
rte_crypto_opand the PMD can call an internal library API to get the session private data pointer based on the driver id.
Quoted 7 lines
Note: currently single session may be used by different device drivers, given it is initialized by them. After the change the session created by one device driver cannot be used or reinitialized by another driver.
Signed-off-by Fan Zhang <[email protected]>Signed-off-by Akhil Goyal <[email protected]>
Please fix compilation for unused function ipsec_mb_fill_error_code
RE: [EXT] [PATCH 1/3] cryptodev: rework session framework
++Konstantin
Requesting everyone in cc to review this patchset from last year. This patch was not merged in 21.11 due to lack of review. Please help getting this merged in RC1 for 22.11.
Regards,
Akhil
Quoted 38 lines
-----Original Message-----
From: Akhil Goyal
Sent: Thursday, September 15, 2022 12:38 PM
To: Fan Zhang <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Anoob Joseph <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [EXT] [PATCH 1/3] cryptodev: rework session frameworkQuoted 19 lines
As per current design,
rte_cryptodev_sym_session_create()andrte_cryptodev_sym_session_init()use separate mempool objects for a single session.
And structurerte_cryptodev_sym_sessionis not directly used by the application, it may cause ABI breakage if the structure is modified in future.To address these two issues, the
rte_cryptodev_sym_session_createwill take one mempool object that the session and session private data are virtually/physically contiguous, and initializes both fields. The APIrte_cryptodev_sym_session_initis removed.
rte_cryptodev_sym_session_createwill now return an opaque session pointer which will be used by the app and other APIs.In data path, opaque session pointer is attached to
rte_crypto_opand the PMD can call an internal library API to get the session private data pointer based on the driver id.Quoted 7 lines
Note: currently single session may be used by different device drivers, given it is initialized by them. After the change the session created by one device driver cannot be used or reinitialized by another driver.
Signed-off-by Fan Zhang <[email protected]>Signed-off-by Akhil Goyal <[email protected]>Please fix compilation for unused function
ipsec_mb_fill_error_code
PATCH 2/3 crypto/scheduler: use unified session
In reply to Fan ZhangThis patch updates the scheduler PMD to use unified session data structure. Previously thanks to the private session array in cryptodev sym session there are no necessary change needed for scheduler PMD other than the way ops are enqueued/dequeued. The patch inherits the same design in the original session data structure to the scheduler PMD so the cryptodev sym session can be as a linear buffer for both session header and driver private data.
With the change there are inevitable extra cost on both memory (64 bytes per session per driver type) and cycle count (set the correct session for each cop based on the worker before enqueue, and retrieve the original session after dequeue).
--- app/test/test_cryptodev.c | 4 +- drivers/crypto/scheduler/scheduler_failover.c | 19 ++- .../crypto/scheduler/scheduler_multicore.c | 17 +++ .../scheduler/scheduler_pkt_size_distr.c | 84 +++++++++--- drivers/crypto/scheduler/scheduler_pmd_ops.c | 101 ++++++++++++++- .../crypto/scheduler/scheduler_pmd_private.h | 120 +++++++++++++++++- .../crypto/scheduler/scheduler_roundrobin.c | 11 +- 7 files changed, 314 insertions(+), 42 deletions(-)
app/test/test_cryptodev.c +2 -2
@@ -14986,8 +14986,8 @@ test_scheduler_attach_worker_op(void) ts_params->session_mpool =rte_cryptodev_sym_session_pool_create ("test_sess_mp" , -MAX_NB_SESSIONS ,0 ,0 ,0 , -SOCKET_ID_ANY ); +MAX_NB_SESSIONS , session_size, +0 ,0 ,SOCKET_ID_ANY );TEST_ASSERT_NOT_NULL (ts_params->session_mpool,"session mempool allocation failed" ); }
drivers/crypto/scheduler/scheduler_failover.c +12 -7
@@ -16,18 +16,19 @@struct fo_scheduler_qp_ctx {struct scheduler_worker primary_worker;struct scheduler_worker secondary_worker; +uint8_t primary_worker_index; +uint8_t secondary_worker_index;uint8_t deq_idx; }; static __rte_always_inline uint16_tfailover_worker_enqueue (struct scheduler_worker *worker, -struct rte_crypto_op **ops,uint16_t nb_ops) +struct rte_crypto_op **ops,uint16_t nb_ops,uint8_t index) { -uint16_t i, processed_ops; +uint16_t processed_ops; -for (i =0 ; i < nb_ops && i <4 ; i++) -rte_prefetch0 (ops[i]->sym->session); +scheduler_set_worker_session (ops, nb_ops, index); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, ops, nb_ops);
@@ -47,13 +48,14 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops)return 0 ; enqueued_ops =failover_worker_enqueue (&qp_ctx->primary_worker, - ops, nb_ops); + ops, nb_ops,PRIMARY_WORKER_IDX );if (enqueued_ops < nb_ops) enqueued_ops +=failover_worker_enqueue ( &qp_ctx->secondary_worker, &ops[enqueued_ops], - nb_ops - enqueued_ops); + nb_ops - enqueued_ops, +SECONDARY_WORKER_IDX );return enqueued_ops; }
@@ -94,7 +96,7 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) qp_ctx->deq_idx = (~qp_ctx->deq_idx) &WORKER_SWITCH_MASK ;if (nb_deq_ops == nb_ops) -return nb_deq_ops; + goto retrieve_session; worker = workers[qp_ctx->deq_idx];
@@ -104,6 +106,9 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) worker->nb_inflight_cops -= nb_deq_ops2; } +retrieve_session: +scheduler_retrieve_session (ops, nb_deq_ops + nb_deq_ops2); +return nb_deq_ops + nb_deq_ops2; }
drivers/crypto/scheduler/scheduler_multicore.c +17 -0
@@ -183,11 +183,19 @@ mc_scheduler_worker(struct rte_cryptodev *dev)while (!mc_ctx->stop_signal) {if (pending_enq_ops) { +scheduler_set_worker_session ( + &enq_ops[pending_enq_ops_idx], pending_enq_ops, + worker_idx); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, &enq_ops[pending_enq_ops_idx], pending_enq_ops); +if (processed_ops < pending_deq_ops) +scheduler_retrieve_session ( + &enq_ops[pending_enq_ops_idx + + processed_ops], + pending_deq_ops - processed_ops); pending_enq_ops -= processed_ops; pending_enq_ops_idx += processed_ops; inflight_ops += processed_ops;
@@ -195,9 +203,16 @@ mc_scheduler_worker(struct rte_cryptodev *dev) processed_ops =rte_ring_dequeue_burst (enq_ring, (void *)enq_ops,MC_SCHED_BUFFER_SIZE ,NULL );if (processed_ops) { +scheduler_set_worker_session (enq_ops, + processed_ops, worker_idx); pending_enq_ops_idx =rte_cryptodev_enqueue_burst ( worker->dev_id, worker->qp_id, enq_ops, processed_ops); +if (pending_enq_ops_idx < processed_ops) +scheduler_retrieve_session ( + enq_ops + pending_enq_ops_idx, + processed_ops - + pending_enq_ops_idx); pending_enq_ops = processed_ops - pending_enq_ops_idx; inflight_ops += pending_enq_ops_idx; }
@@ -214,6 +229,8 @@ mc_scheduler_worker(struct rte_cryptodev *dev) worker->dev_id, worker->qp_id, deq_ops, MC_SCHED_BUFFER_SIZE);if (processed_ops) { +scheduler_retrieve_session (deq_ops, + processed_ops); inflight_ops -= processed_ops;if (reordering_enabled) {uint16_t j;
drivers/crypto/scheduler/scheduler_pkt_size_distr.c +63 -21
149 lines not on this page · raw message
drivers/crypto/scheduler/scheduler_pmd_ops.c +95 -6
120 lines not on this page · raw message
drivers/crypto/scheduler/scheduler_pmd_private.h +119 -1
144 lines not on this page · raw message
drivers/crypto/scheduler/scheduler_roundrobin.c +6 -5
@@ -23,16 +23,17 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) ((struct scheduler_qp_ctx *)qp)->private_qp_ctx;uint32_t worker_idx = rr_qp_ctx->last_enq_worker_idx;struct scheduler_worker *worker = &rr_qp_ctx->workers[worker_idx]; -uint16_t i, processed_ops; +uint16_t processed_ops;if (unlikely (nb_ops ==0 ))return 0 ; -for (i =0 ; i < nb_ops && i <4 ; i++) -rte_prefetch0 (ops[i]->sym->session); - +scheduler_set_worker_session (ops, nb_ops, worker_idx); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, ops, nb_ops); +if (processed_ops < nb_ops) +scheduler_retrieve_session (ops + processed_ops, + nb_ops - processed_ops); worker->nb_inflight_cops += processed_ops;
@@ -86,7 +87,7 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) nb_deq_ops =rte_cryptodev_dequeue_burst (worker->dev_id, worker->qp_id, ops, nb_ops); - +scheduler_retrieve_session (ops, nb_deq_ops); last_worker_idx +=1 ; last_worker_idx %= rr_qp_ctx->nb_workers;
Signature · 1 line
2.34.1
RE: [EXT] [PATCH 2/3] crypto/scheduler: use unified session
Hi Fan,
Are these patches getting compiled at your end? Are these a working patchset on Intel platform? For me it is failing for each of the patch
'drivers/a715181@@tmp_rte_crypto_scheduler@sta/crypto_scheduler_scheduler_pmd_ops.c.o' -MF 'drivers/a715181@@tmp_rte_crypto_scheduler@sta/crypto_scheduler_scheduler_pmd_ops.c.o.d' -o 'drivers/a715181@@tmp_rte_crypto_scheduler@sta/crypto_scheduler_scheduler_pmd_ops.c.o' -c ../drivers/crypto/scheduler/scheduler_pmd_ops.c ../drivers/crypto/scheduler/scheduler_pmd_ops.c: In function 'scheduler_pmd_sym_session_configure':../drivers/crypto/scheduler/scheduler_pmd_ops.c:525:1: error: label at end of compound statement
525 | next_worker:
| ^~~~~~~~~~~
../drivers/crypto/scheduler/scheduler_pmd_ops.c: In function 'scheduler_pmd_sym_session_clear':
../drivers/crypto/scheduler/scheduler_pmd_ops.c:569:1: error: label at end of compound statement
569 | next_worker:
| ^~~~~~~~~~~Quoted 667 lines
This patch updates the scheduler PMD to use unified session data structure. Previously thanks to the private session array in cryptodev sym session there are no necessary change needed for scheduler PMD other than the way ops are enqueued/dequeued. The patch inherits the same design in the original session data structure to the scheduler PMD so the cryptodev sym session can be as a linear buffer for both session header and driver private data.
With the change there are inevitable extra cost on both memory (64 bytes per session per driver type) and cycle count (set the correct session for each cop based on the worker before enqueue, and retrieve the original session after dequeue).
Signed-off-by Fan Zhang <[email protected]>--- app/test/test_cryptodev.c | 4 +- drivers/crypto/scheduler/scheduler_failover.c | 19 ++- .../crypto/scheduler/scheduler_multicore.c | 17 +++ .../scheduler/scheduler_pkt_size_distr.c | 84 +++++++++--- drivers/crypto/scheduler/scheduler_pmd_ops.c | 101 ++++++++++++++- .../crypto/scheduler/scheduler_pmd_private.h | 120 +++++++++++++++++- .../crypto/scheduler/scheduler_roundrobin.c | 11 +- 7 files changed, 314 insertions(+), 42 deletions(-)app/test/test_cryptodev.c +2
-2@@ -14986,8 +14986,8 @@ test_scheduler_attach_worker_op(void) ts_params->session_mpool =rte_cryptodev_sym_session_pool_create ("test_sess_mp" , -MAX_NB_SESSIONS ,0 ,0 ,0 , -SOCKET_ID_ANY ); +MAX_NB_SESSIONS , session_size, +0 ,0 ,SOCKET_ID_ANY );TEST_ASSERT_NOT_NULL (ts_params->session_mpool,"session mempool allocation failed" );app/test/test_cryptodev.c +0
-0}drivers/crypto/scheduler/scheduler_failover.c +8
-6@@ -16,18 +16,19 @@struct fo_scheduler_qp_ctx {struct scheduler_worker primary_worker;struct scheduler_worker secondary_worker; +uint8_t primary_worker_index; +uint8_t secondary_worker_index;uint8_t deq_idx; }; static __rte_always_inline uint16_tfailover_worker_enqueue (struct scheduler_worker *worker, -struct rte_crypto_op **ops,uint16_t nb_ops) +struct rte_crypto_op **ops,uint16_t nb_ops,uint8_t index) { -uint16_t i, processed_ops; +uint16_t processed_ops; -for (i =0 ; i < nb_ops && i <4 ; i++) -rte_prefetch0 (ops[i]->sym->session); +scheduler_set_worker_session (ops, nb_ops, index); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, ops, nb_ops);@@ -47,13 +48,14 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops)return 0 ; enqueued_ops =failover_worker_enqueue (&qp_ctx->primary_worker, - ops, nb_ops); + ops, nb_ops,PRIMARY_WORKER_IDX );if (enqueued_ops < nb_ops) enqueued_ops +=failover_worker_enqueue ( &qp_ctx->secondary_worker, &ops[enqueued_ops], - nb_ops - enqueued_ops); + nb_ops - enqueued_ops, +SECONDARY_WORKER_IDX );return enqueued_ops;}
drivers/crypto/scheduler/scheduler_failover.c +4
-1@@ -94,7 +96,7 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) qp_ctx->deq_idx = (~qp_ctx->deq_idx) & WORKER_SWITCH_MASK;if (nb_deq_ops == nb_ops) -return nb_deq_ops; + goto retrieve_session; worker = workers[qp_ctx->deq_idx];@@ -104,6 +106,9 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) worker->nb_inflight_cops -= nb_deq_ops2; } +retrieve_session: +scheduler_retrieve_session (ops, nb_deq_ops + nb_deq_ops2); +return nb_deq_ops + nb_deq_ops2; }drivers/crypto/scheduler/scheduler_multicore.c +8
-0@@ -183,11 +183,19 @@ mc_scheduler_worker(struct rte_cryptodev *dev)while (!mc_ctx->stop_signal) {if (pending_enq_ops) { +scheduler_set_worker_session ( + &enq_ops[pending_enq_ops_idx], pending_enq_ops, + worker_idx); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, &enq_ops[pending_enq_ops_idx], pending_enq_ops); +if (processed_ops < pending_deq_ops) +scheduler_retrieve_session ( + &enq_ops[pending_enq_ops_idx + + processed_ops], + pending_deq_ops - processed_ops); pending_enq_ops -= processed_ops; pending_enq_ops_idx += processed_ops;drivers/crypto/scheduler/scheduler_multicore.c +2
-0inflight_ops += processed_ops;@@ -195,9 +203,16 @@ mc_scheduler_worker(struct rte_cryptodev *dev) processed_ops =rte_ring_dequeue_burst (enq_ring, (void *)enq_ops,MC_SCHED_BUFFER_SIZE ,NULL );if (processed_ops) { +scheduler_set_worker_session (enq_ops, + processed_ops, worker_idx); pending_enq_ops_idx =rte_cryptodev_enqueue_burst(
worker->dev_id, worker-Quoted 1 line
qp_id,drivers/crypto/scheduler/scheduler_multicore.c +3
-0enq_ops, processed_ops); +if (pending_enq_ops_idx < processed_ops) + scheduler_retrieve_session( + enq_ops +
pending_enq_ops_idx,drivers/crypto/scheduler/scheduler_multicore.c +2
-0+ processed_ops - + pending_enq_ops_idx); pending_enq_ops = processed_ops -
pending_enq_ops_idx;drivers/crypto/scheduler/scheduler_multicore.c +2
-0inflight_ops += pending_enq_ops_idx; }@@ -214,6 +229,8 @@ mc_scheduler_worker(struct rte_cryptodev *dev) worker->dev_id, worker->qp_id, deq_ops, MC_SCHED_BUFFER_SIZE);if (processed_ops) { +scheduler_retrieve_session (deq_ops, + processed_ops); inflight_ops -= processed_ops;if (reordering_enabled) {drivers/crypto/scheduler/scheduler_multicore.c +0
-0uint16_t j;drivers/crypto/scheduler/scheduler_pkt_size_distr.c +29
-9@@ -48,34 +48,54 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) };struct psd_schedule_op *p_enq_op;uint16_t i, processed_ops_pri =0 , processed_ops_sec =0 ; -uint32_t job_len;if (unlikely (nb_ops ==0 ))return 0 ;for (i =0 ; i < nb_ops && i <4 ; i++) {rte_prefetch0 (ops[i]->sym); -rte_prefetch0 (ops[i]->sym->session); +rte_prefetch0 ((uint8_t *)ops[i]->sym->session + +sizeof (struct rte_cryptodev_sym_session )); }for (i =0 ; (i < (nb_ops -8 )) && (nb_ops >8 ); i +=4 ) { +struct scheduler_session_ctx *sess_ctx[4 ]; +uint8_t target[4 ]; +uint32_t job_len[4 ]; +rte_prefetch0 (ops[i +4 ]->sym); -rte_prefetch0 (ops[i +4 ]->sym->session); +rte_prefetch0 ((uint8_t *)ops[i +4 ]->sym->session + +sizeof (struct rte_cryptodev_sym_session ));rte_prefetch0 (ops[i +5 ]->sym); -rte_prefetch0 (ops[i +5 ]->sym->session); +rte_prefetch0 ((uint8_t *)ops[i +5 ]->sym->session + +sizeof (struct rte_cryptodev_sym_session ));rte_prefetch0 (ops[i +6 ]->sym); -rte_prefetch0 (ops[i +6 ]->sym->session); +rte_prefetch0 ((uint8_t *)ops[i +6 ]->sym->session + +sizeof (struct rte_cryptodev_sym_session ));rte_prefetch0 (ops[i +7 ]->sym); -rte_prefetch0 (ops[i +7 ]->sym->session); +rte_prefetch0 ((uint8_t *)ops[i +7 ]->sym->session + +sizeof (struct rte_cryptodev_sym_session )); + + sess_ctx[0 ] = (void *)ops[i]->sym->session->driver_priv_data; + sess_ctx[1 ] = + (void *)ops[i +1 ]->sym->session->driver_priv_data; + sess_ctx[2 ] = + (void *)ops[i +2 ]->sym->session->driver_priv_data; + sess_ctx[3 ] = + (void *)ops[i +3 ]->sym->session->driver_priv_data;/* job_len is initialized as cipher data length, once * it is 0, equals to auth data length */ - job_len = ops[i]->sym->cipher.data.length; - job_len += (ops[i]->sym->cipher.data.length ==0 ) * + job_len[0 ] = ops[i]->sym->cipher.data.length; + job_len[0 ] += (ops[i]->sym->cipher.data.length ==0 ) * ops[i]->sym->auth.data.length;/* decide the target op based on the job length */ - p_enq_op = &enq_ops[!(job_len & psd_qp_ctx->threshold)]; + target[0 ] = !(job_len[0 ] & psd_qp_ctx->threshold); +if (ops[i]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) + ops[i]->sym->session = + sess_ctx[0 ]->worker_sess[target[0 ]]; + p_enq_op = &enq_ops[target[0 ]];/* stop schedule cops before the queue is full, this shall drivers/crypto/scheduler/scheduler_pkt_size_distr.c +7
-3* prevent the failed enqueue @@ -89,10 +109,14 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) sched_ops[p_enq_op->worker_idx][p_enq_op->pos] = ops[i]; p_enq_op->pos++; - job_len = ops[i+1 ]->sym->cipher.data.length; - job_len += (ops[i+1 ]->sym->cipher.data.length ==0 ) * + job_len[1 ] = ops[i +1 ]->sym->cipher.data.length; + job_len[1 ] += (ops[i +1 ]->sym->cipher.data.length ==0 ) * ops[i+1 ]->sym->auth.data.length; - p_enq_op = &enq_ops[!(job_len & psd_qp_ctx->threshold)]; + target[1 ] = !(job_len[1 ] & psd_qp_ctx->threshold); +if (ops[i +1 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) + ops[i +1 ]->sym->session = + sess_ctx[1 ]->worker_sess[target[1 ]]; + p_enq_op = &enq_ops[target[1 ]]; if (p_enq_op->pos + in_flight_ops[p_enq_op->worker_idx] ==drivers/crypto/scheduler/scheduler_pkt_size_distr.c +8
-4qp_ctx->max_nb_objs) {@@ -103,10 +127,14 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) sched_ops[p_enq_op->worker_idx][p_enq_op->pos] = ops[i+1 ]; p_enq_op->pos++; - job_len = ops[i+2 ]->sym->cipher.data.length; - job_len += (ops[i+2 ]->sym->cipher.data.length ==0 ) * - ops[i+2 ]->sym->auth.data.length; - p_enq_op = &enq_ops[!(job_len & psd_qp_ctx->threshold)]; + job_len[2 ] = ops[i +2 ]->sym->cipher.data.length; + job_len[2 ] += (ops[i +2 ]->sym->cipher.data.length ==0 ) * + ops[i +2 ]->sym->auth.data.length; + target[2 ] = !(job_len[2 ] & psd_qp_ctx->threshold); +if (ops[i +2 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) + ops[i +2 ]->sym->session = + sess_ctx[2 ]->worker_sess[target[2 ]]; + p_enq_op = &enq_ops[target[2 ]]; if (p_enq_op->pos + in_flight_ops[p_enq_op->worker_idx] ==drivers/crypto/scheduler/scheduler_pkt_size_distr.c +8
-4qp_ctx->max_nb_objs) {@@ -117,10 +145,14 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) sched_ops[p_enq_op->worker_idx][p_enq_op->pos] = ops[i+2 ]; p_enq_op->pos++; - job_len = ops[i+3 ]->sym->cipher.data.length; - job_len += (ops[i+3 ]->sym->cipher.data.length ==0 ) * - ops[i+3 ]->sym->auth.data.length; - p_enq_op = &enq_ops[!(job_len & psd_qp_ctx->threshold)]; + job_len[3 ] = ops[i +3 ]->sym->cipher.data.length; + job_len[3 ] += (ops[i +3 ]->sym->cipher.data.length ==0 ) * + ops[i +3 ]->sym->auth.data.length; + target[3 ] = !(job_len[3 ] & psd_qp_ctx->threshold); +if (ops[i +3 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) + ops[i +3 ]->sym->session = + sess_ctx[1 ]->worker_sess[target[3 ]]; + p_enq_op = &enq_ops[target[3 ]]; if (p_enq_op->pos + in_flight_ops[p_enq_op->worker_idx] ==drivers/crypto/scheduler/scheduler_pkt_size_distr.c +9
-1qp_ctx->max_nb_objs) {@@ -133,10 +165,18 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) }for (; i < nb_ops; i++) { +struct scheduler_session_ctx *sess_ctx = + (void *)ops[i]->sym->session->driver_priv_data; +uint32_t job_len; +uint8_t target; + job_len = ops[i]->sym->cipher.data.length; job_len += (ops[i]->sym->cipher.data.length ==0 ) * ops[i]->sym->auth.data.length; - p_enq_op = &enq_ops[!(job_len & psd_qp_ctx->threshold)]; + target = !(job_len & psd_qp_ctx->threshold); +if (ops[i]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) + ops[i]->sym->session = sess_ctx->worker_sess[target]; + p_enq_op = &enq_ops[target]; if (p_enq_op->pos + in_flight_ops[p_enq_op->worker_idx] ==drivers/crypto/scheduler/scheduler_pkt_size_distr.c +1
-0qp_ctx->max_nb_objs) {@@ -199,6 +239,7 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops)if (worker->nb_inflight_cops) { nb_deq_ops_pri =rte_cryptodev_dequeue_burst (worker- >dev_id, worker->qp_id, ops, nb_ops); +scheduler_retrieve_session (ops, nb_deq_ops_pri); worker->nb_inflight_cops -= nb_deq_ops_pri;drivers/crypto/scheduler/scheduler_pkt_size_distr.c +1
-0}@@ -213,6 +254,7 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) nb_deq_ops_sec =rte_cryptodev_dequeue_burst (worker- >dev_id, worker->qp_id, &ops[nb_deq_ops_pri], nb_ops - nb_deq_ops_pri); +scheduler_retrieve_session (ops, nb_deq_ops_sec); worker->nb_inflight_cops -= nb_deq_ops_sec;drivers/crypto/scheduler/scheduler_pkt_size_distr.c +0
-0if (!worker->nb_inflight_cops)drivers/crypto/scheduler/scheduler_pmd_ops.c +69
-6@@ -9,6 +9,7 @@#include <rte_cryptodev.h> #include <cryptodev_pmd.h> #include <rte_reorder.h> +#include <rte_errno.h> #include "scheduler_pmd_private.h" @@ -467,19 +468,107 @@ scheduler_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)return max_priv_sess_size; } +struct scheduler_configured_sess_info { +uint8_t dev_id; +uint8_t driver_id; +struct rte_cryptodev_sym_session *sess; +}; +static int -scheduler_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused, - struct rte_crypto_sym_xform *xform __rte_unused, - struct rte_cryptodev_sym_session *sess __rte_unused) +scheduler_pmd_sym_session_configure (struct rte_cryptodev *dev, +struct rte_crypto_sym_xform *xform, +struct rte_cryptodev_sym_session *sess) { + struct scheduler_ctx *sched_ctx = dev->data->dev_private; +struct rte_mempool *mp =rte_mempool_from_obj (sess); +struct scheduler_session_ctx *sess_ctx = (void *)sess->driver_priv_data; +struct scheduler_configured_sess_info configured_sess[ +RTE_CRYPTODEV_SCHEDULER_MAX_NB_WORKERS ] = {0 }; +uint32_t i, j, n_configured_sess =0 ; +int ret =0 ; + +if (mp ==NULL ) +return -EINVAL ; + +for (i =0 ; i < sched_ctx->nb_workers; i++) { +struct scheduler_worker *worker = &sched_ctx->workers[i]; +struct rte_cryptodev_sym_session *worker_sess; + +for (j =0 ; j < n_configured_sess; j++) { +if (configured_sess[j].driver_id == + worker->driver_id) { + sess_ctx->worker_sess[i] = + configured_sess[j].sess; + goto next_worker; + } + } + +if (rte_mempool_avail_count (mp) ==0 ) { + ret = -ENOMEM ; + goto error_exit; + } + + worker_sess =rte_cryptodev_sym_session_create (worker- >dev_id, + xform, mp); +if (worker_sess ==NULL ) { + ret = -rte_errno; + goto error_exit; + } + + worker_sess->opaque_data = (uint64_t )sess; + sess_ctx->worker_sess[i] = worker_sess; + configured_sess[n_configured_sess].driver_id = + worker->driver_id; + configured_sess[n_configured_sess].dev_id = worker->dev_id; + configured_sess[n_configured_sess].sess = worker_sess; + n_configured_sess++; +next_worker: + } +return 0 ; +error_exit: + sess_ctx->ref_cnt = sched_ctx->ref_cnt; +for (i =0 ; i < n_configured_sess; i++) +rte_cryptodev_sym_session_free (configured_sess[i].dev_id, + configured_sess[i].sess); +return ret; }/** Clear the memory of session so it doesn't leave key material behind */ static void -scheduler_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, - struct rte_cryptodev_sym_session *sess __rte_unused) -{} +scheduler_pmd_sym_session_clear (struct rte_cryptodev *dev, +struct rte_cryptodev_sym_session *sess) +{ +struct scheduler_ctx *sched_ctx = dev->data->dev_private; +struct scheduler_session_ctx *sess_ctx = (void *)sess->driver_priv_data; +struct scheduler_configured_sess_info deleted_sess[ + RTE_CRYPTODEV_SCHEDULER_MAX_NB_WORKERS] ={ 0 };
drivers/crypto/scheduler/scheduler_pmd_ops.c +26
-0+uint32_t i, j, n_deleted_sess =0 ; + +if (sched_ctx->ref_cnt != sess_ctx->ref_cnt) { +CR_SCHED_LOG (WARNING , +"Worker updated between session creation/deletion. " +"The session may not be freed fully." ); + } + +for (i =0 ; i < sched_ctx->nb_workers; i++) { +struct scheduler_worker *worker = &sched_ctx->workers[i]; + +for (j =0 ; j < n_deleted_sess; j++) { +if (deleted_sess[j].driver_id == worker->driver_id) { + sess_ctx->worker_sess[i] =NULL ; + goto next_worker; + } + } + +rte_cryptodev_sym_session_free (worker->dev_id, + sess_ctx->worker_sess[i]); + + deleted_sess[n_deleted_sess++].driver_id = worker->driver_id; + sess_ctx->worker_sess[i] =NULL ; +next_worker: + } +}static struct rte_cryptodev_ops scheduler_pmd_ops = { .dev_configure = scheduler_pmd_config,drivers/crypto/scheduler/scheduler_pmd_private.h +119
-1@@ -22,7 +22,6 @@ struct scheduler_worker {uint8_t dev_id;uint16_t qp_id;uint32_t nb_inflight_cops; -uint8_t driver_id; };@@ -37,6 +36,8 @@ struct scheduler_ctx {struct scheduler_worker workers[RTE_CRYPTODEV_SCHEDULER_MAX_NB_WORKERS ];uint32_t nb_workers; +/* reference count when the workers are incremented/decremented */ +uint32_t ref_cnt;enum rte_cryptodev_scheduler_mode mode;@@ -61,6 +62,11 @@ struct scheduler_qp_ctx {struct rte_ring *order_ring; } __rte_cache_aligned; +struct scheduler_session_ctx { +uint32_t ref_cnt; +struct rte_cryptodev_sym_session *worker_sess[ +RTE_CRYPTODEV_SCHEDULER_MAX_NB_WORKERS ]; +};extern uint8_t cryptodev_scheduler_driver_id;@@ -101,6 +107,118 @@ scheduler_order_drain(struct rte_ring *order_ring,return nb_ops_to_deq; } +static __rte_always_inline void +scheduler_set_worker_session (struct rte_crypto_op **ops,uint16_t nb_ops, +uint8_t worker_index) +{ +struct rte_crypto_op **op = ops; +uint16_t n = nb_ops; + +if (n >=4 ) { +rte_prefetch0 (op[0 ]->sym->session); +rte_prefetch0 (op[1 ]->sym->session); +rte_prefetch0 (op[2 ]->sym->session); +rte_prefetch0 (op[3 ]->sym->session); + } + +while (n >=4 ) { +if (n >=8 ) { +rte_prefetch0 (op[4 ]->sym->session); +rte_prefetch0 (op[5 ]->sym->session); +rte_prefetch0 (op[6 ]->sym->session); +rte_prefetch0 (op[7 ]->sym->session); + } + +if (op[0 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { +struct scheduler_session_ctx *sess_ctx = + (void *)op[0 ]->sym->session->driver_priv_data; + op[0 ]->sym->session = + sess_ctx->worker_sess[worker_index]; + } + +if (op[1 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { +struct scheduler_session_ctx *sess_ctx = + (void *)op[1 ]->sym->session->driver_priv_data; + op[1 ]->sym->session = + sess_ctx->worker_sess[worker_index]; + } + +if (op[2 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { +struct scheduler_session_ctx *sess_ctx = + (void *)op[2 ]->sym->session->driver_priv_data; + op[2 ]->sym->session = + sess_ctx->worker_sess[worker_index]; + } + +if (op[3 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { +struct scheduler_session_ctx *sess_ctx = + (void *)op[3 ]->sym->session->driver_priv_data; + op[3 ]->sym->session = + sess_ctx->worker_sess[worker_index]; + } + + op +=4 ; + n -=4 ; + } + +while (n--) { +if (op[0 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { +struct scheduler_session_ctx *sess_ctx = + (void *)op[0 ]->sym->session->driver_priv_data; + + op[0 ]->sym->session = + sess_ctx->worker_sess[worker_index]; + op++; + } + } +} + +static __rte_always_inline void +scheduler_retrieve_session (struct rte_crypto_op **ops,uint16_t nb_ops) +{ +uint16_t n = nb_ops; +struct rte_crypto_op **op = ops; + +if (n >=4 ) { +rte_prefetch0 (op[0 ]->sym->session); +rte_prefetch0 (op[1 ]->sym->session); +rte_prefetch0 (op[2 ]->sym->session); +rte_prefetch0 (op[3 ]->sym->session); + } + +while (n >=4 ) { +if (n >=8 ) { +rte_prefetch0 (op[4 ]->sym->session); +rte_prefetch0 (op[5 ]->sym->session); +rte_prefetch0 (op[6 ]->sym->session); +rte_prefetch0 (op[7 ]->sym->session); + } + +if (op[0 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) + op[0 ]->sym->session = + (void *)op[0 ]->sym->session->opaque_data; +if (op[1 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) + op[1 ]->sym->session = + (void *)op[1 ]->sym->session->opaque_data; +if (op[2 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) + op[2 ]->sym->session = + (void *)op[2 ]->sym->session->opaque_data; +if (op[3 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) + op[3 ]->sym->session = + (void *)op[3 ]->sym->session->opaque_data; + + op +=4 ; + n -=4 ; + } + +while (n--) { +if (op[0 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) + op[0 ]->sym->session = + (void *)op[0 ]->sym->session->opaque_data; + op++; + } +} +/** device specific operations function pointer structure */ extern struct rte_cryptodev_ops *rte_crypto_scheduler_pmd_ops;drivers/crypto/scheduler/scheduler_roundrobin.c +6
-6@@ -23,16 +23,17 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) ((struct scheduler_qp_ctx *)qp)->private_qp_ctx;uint32_t worker_idx = rr_qp_ctx->last_enq_worker_idx;struct scheduler_worker *worker = &rr_qp_ctx->workers[worker_idx]; -uint16_t i, processed_ops; +uint16_t processed_ops;if (unlikely (nb_ops ==0 ))return 0 ; -for (i =0 ; i < nb_ops && i <4 ; i++) -rte_prefetch0 (ops[i]->sym->session); - +scheduler_set_worker_session (ops, nb_ops, worker_idx); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, ops, nb_ops); +if (processed_ops < nb_ops) +scheduler_retrieve_session (ops + processed_ops, + nb_ops - processed_ops); worker->nb_inflight_cops += processed_ops;@@ -86,7 +87,7 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) nb_deq_ops =rte_cryptodev_dequeue_burst (worker->dev_id, worker->qp_id, ops, nb_ops); - +scheduler_retrieve_session (ops, nb_deq_ops); last_worker_idx +=1 ; last_worker_idx %= rr_qp_ctx->nb_workers; --2.34.1
RE: [EXT] [PATCH 2/3] crypto/scheduler: use unified session
Quoted 17 lines
Hi Fan,
Are these patches getting compiled at your end? Are these a working patchset on Intel platform?
For me it is failing for each of the patch'
drivers/a715181@@tmp_rte_crypto_scheduler@sta/crypto_scheduler_scheduler_pmd_ops.c.o' -MF 'drivers/a715181@@tmp_rte_crypto_scheduler@sta/crypto_scheduler_scheduler_pmd_ops.c.o.d' -o 'drivers/a715181@@tmp_rte_crypto_scheduler@sta/crypto_scheduler_scheduler_pmd_ops.c.o' -c../drivers/crypto/scheduler/scheduler_pmd_ops.c../drivers/crypto/scheduler/scheduler_pmd_ops.c: In function 'scheduler_pmd_sym_session_configure':../drivers/crypto/scheduler/scheduler_pmd_ops.c:525:1: error: label at end of compound statement
525 |next_worker:
Quoted 7 lines
| ^~~~~~~~~~~
../drivers/crypto/scheduler/scheduler_pmd_ops.c: In function 'scheduler_pmd_sym_session_clear':../drivers/crypto/scheduler/scheduler_pmd_ops.c:569:1: error: label at end of compound statement
569 |next_worker:| ^~~~~~~~~~~
You can try this fix
drivers/crypto/scheduler/scheduler_pmd_ops.c +10 -4
@@ -493,15 +493,19 @@ scheduler_pmd_sym_session_configure(struct rte_cryptodev *dev,for (i =0 ; i < sched_ctx->nb_workers; i++) {struct scheduler_worker *worker = &sched_ctx->workers[i];struct rte_cryptodev_sym_session *worker_sess; +uint8_t next_worker =0 ;for (j =0 ; j < n_configured_sess; j++) {if (configured_sess[j].driver_id == worker->driver_id) { sess_ctx->worker_sess[i] = configured_sess[j].sess; - goto next_worker; + next_worker =1 ; +break ; } } +if (next_worker) +continue ;if (rte_mempool_avail_count (mp) ==0 ) { ret = -ENOMEM ;
@@ -522,7 +526,6 @@ scheduler_pmd_sym_session_configure(struct rte_cryptodev *dev, configured_sess[n_configured_sess].dev_id = worker->dev_id; configured_sess[n_configured_sess].sess = worker_sess; n_configured_sess++; -next_worker: }return 0 ;
@@ -553,20 +556,23 @@ scheduler_pmd_sym_session_clear(struct rte_cryptodev *dev,for (i =0 ; i < sched_ctx->nb_workers; i++) {struct scheduler_worker *worker = &sched_ctx->workers[i]; +uint8_t next_worker =0 ;for (j =0 ; j < n_deleted_sess; j++) {if (deleted_sess[j].driver_id == worker->driver_id) { sess_ctx->worker_sess[i] =NULL ; - goto next_worker; + next_worker =1 ; +break ; } } +if (next_worker) +continue ;rte_cryptodev_sym_session_free (worker->dev_id, sess_ctx->worker_sess[i]); deleted_sess[n_deleted_sess++].driver_id = worker->driver_id; sess_ctx->worker_sess[i] =NULL ; -next_worker: } }
RE: [EXT] [PATCH 2/3] crypto/scheduler: use unified session
In reply to Fan ZhangQuoted 15 lines
app/test/test_cryptodev.c +2
-2@@ -14986,8 +14986,8 @@ test_scheduler_attach_worker_op(void) ts_params->session_mpool =rte_cryptodev_sym_session_pool_create ("test_sess_mp" , -MAX_NB_SESSIONS ,0 ,0 ,0 , -SOCKET_ID_ANY ); +MAX_NB_SESSIONS , session_size, +0 ,0 ,SOCKET_ID_ANY );TEST_ASSERT_NOT_NULL (ts_params->session_mpool,"session mempool allocation failed" );
Quoted 1 line
}
This change should not be part of this patch. Please move it to other patch.
PATCH 3/3 cryptodev: hide sym session structure
In reply to Fan ZhangStructure rte_cryptodev_sym_session is moved to internal headers which are not visible to applications.
The only field which should be used by app is opaque_data. This field can now be accessed via set/get APIs added in this patch.
Subsequent changes in app and lib are made to compile the code.
--- app/test/test_ipsec_perf.c | 4 +- doc/guides/prog_guide/cryptodev_lib.rst | 16 ++---- doc/guides/rel_notes/deprecation.rst | 9 ++++ doc/guides/rel_notes/release_22_11.rst | 7 +++ drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 2 +- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 2 +- drivers/crypto/qat/qat_sym.c | 2 +- lib/cryptodev/cryptodev_pmd.h | 31 ++++++++++++ lib/cryptodev/rte_cryptodev.c | 16 +++--- lib/cryptodev/rte_cryptodev.h | 56 ++++++++++----------- lib/cryptodev/rte_cryptodev_trace.h | 8 +-- lib/ipsec/rte_ipsec_group.h | 5 +- lib/ipsec/ses.c | 3 +- 13 files changed, 100 insertions(+), 61 deletions(-)
app/test/test_ipsec_perf.c +2 -2
@@ -227,7 +227,7 @@ static int create_sa(enum rte_security_session_action_type action_type, struct ipsec_sa *sa) { - static struct rte_cryptodev_sym_session dummy_ses; + void *dummy_ses = NULL;size_t sz;int rc;
@@ -247,7 +247,7 @@ create_sa(enum rte_security_session_action_type action_type, "failed to allocate memory for rte_ipsec_sa\n"); sa->ss[0 ].type = action_type; - sa->ss[0 ].crypto.ses = &dummy_ses; + sa->ss[0 ].crypto.ses = dummy_ses; rc =rte_ipsec_sa_init (sa->ss[0 ].sa, &sa->sa_prm, sz); rc = (rc >0 && (uint32_t )rc <= sz) ?0 : -EINVAL ;
doc/guides/prog_guide/cryptodev_lib.rst +5 -11
@@ -125,13 +125,11 @@ Each queue pairs resources may be allocated on a specified socket. uint32_t nb_descriptors; /**< Number of descriptors per queue pair */ struct rte_mempool *mp_session; /**< The mempool for creating session in sessionless mode */ - struct rte_mempool *mp_session_private; - /**< The mempool for creating sess private data in sessionless mode */ }; -The fields ``mp_session`` and ``mp_session_private`` are used for creating -temporary session to process the crypto operations in the session-less mode. +The field ``mp_session`` is used for creating temporary session to process +the crypto operations in the session-less mode. They can be the same other different mempools. Please note not all Cryptodev PMDs supports session-less mode.
@@ -595,7 +593,7 @@ chain. struct rte_mbuf *m_dst; union { - struct rte_cryptodev_sym_session *session; + void *session; /**< Handle for the initialised session context */ struct rte_crypto_sym_xform *xform; /**< Session-less API Crypto operation parameters */
@@ -943,15 +941,11 @@ using one of the crypto PMDs available in DPDK. /* Create crypto session and initialize it for the crypto device. */ struct rte_cryptodev_sym_session *session; - session = rte_cryptodev_sym_session_create(session_pool); + session = rte_cryptodev_sym_session_create(cdev_id, &cipher_xform, + session_pool); if (session == NULL) rte_exit(EXIT_FAILURE, "Session could not be created\n"); - if (rte_cryptodev_sym_session_init(cdev_id, session, - &cipher_xform, session_priv_pool) < 0) - rte_exit(EXIT_FAILURE, "Session could not be initialized " - "for the crypto device\n"); - /* Get a burst of crypto operations. */ struct rte_crypto_op *crypto_ops[BURST_SIZE]; if (rte_crypto_op_bulk_alloc(crypto_op_pool,
doc/guides/rel_notes/deprecation.rst +9 -0
@@ -237,3 +237,12 @@ Deprecation Notices applications should be updated to use the ``dmadev`` library instead, with the underlying HW-functionality being provided by the ``ioat`` or ``idxd`` dma drivers + +* cryptodev: Hide structure ``rte_cryptodev_sym_session`` to remove unnecessary + indirection between session and the private data of session. An opaque pointer + can be exposed directly to application which can be attached to the + ``rte_crypto_op``. + +* cryptodev: The functions ``rte_cryptodev_sym_session_init`` and + ``rte_cryptodev_sym_session_clear`` are deprecated. The sym crypto session + opaque pointer can no longer be used by different crypto device drivers.
doc/guides/rel_notes/release_22_11.rst +7 -0
@@ -89,6 +89,13 @@ API Changes Also, make sure to start the actual text at the margin. ======================================================= +* cryptodev: The structure ``rte_cryptodev_sym_session`` was moved to + cryptodev_pmd.h and was hidden from the application. The APIs to create/init and + destroy sym crypto session were updated to take a single mempool with element size + enough to hold session data and session private data. Inline APIs was created to + get and set the session data. All sample applications were updated to attach an + opaque pointer for the session to the ``rte_crypto_op`` while enqueuing. + ABI Changes -----------
drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c +1 -1
@@ -1012,7 +1012,7 @@ dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess = (dpaa2_sec_session *)get_sec_session_private_data ( session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (void *)session_ctx.crypto_sess->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (session_ctx.crypto_sess); else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa2_sec_raw_dequeue_burst;
drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c +1 -1
@@ -1018,7 +1018,7 @@ dpaa_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) sess = (dpaa_sec_session *) - session_ctx.crypto_sess->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (session_ctx.crypto_sess); else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa_sec_raw_dequeue_burst;
drivers/crypto/qat/qat_sym.c +1 -1
@@ -386,7 +386,7 @@ qat_sym_configure_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id,if (sess_type !=RTE_CRYPTO_OP_WITH_SESSION )return -EINVAL ; - ctx = (void *)session_ctx.crypto_sess->driver_priv_data; + ctx =CRYPTODEV_GET_SYM_SESS_PRIV (session_ctx.crypto_sess); dp_ctx->session = ctx;
lib/cryptodev/cryptodev_pmd.h +31 -0
@@ -132,6 +132,37 @@ struct cryptodev_driver {uint8_t id; }; +/** Cryptodev symmetric crypto session +* Each session is derived from a fixed xform chain. Therefore each session +* has a fixed algo, key, op-type, digest_len etc. +*/ +struct rte_cryptodev_sym_session { +RTE_MARKER cacheline0; +uint8_t driver_id; +uint64_t opaque_data; +/**< Can be used for external metadata */ +uint32_t sess_data_sz; +/**< Pointer to the user data stored after sess data */ +uint16_t user_data_sz; +/**< session user data will be placed after sess data */ +rte_iova_t driver_priv_data_iova; +/**< session driver data IOVA address */ + + RTE_MARKER cacheline1 __rte_cache_min_aligned; +/**< second cache line - start of the driver session data */ +uint8_t driver_priv_data[0 ]; +/**< Driver specific session data, variable size */ +}; + +/** +* Helper macro to get driver private data +*/ +#define CRYPTODEV_GET_SYM_SESS_PRIV (s) \ + (void *)(((struct rte_cryptodev_sym_session *)s)->driver_priv_data) +#define CRYPTODEV_GET_SYM_SESS_PRIV_IOVA (s) \ + (((struct rte_cryptodev_sym_session *)s)->driver_priv_data_iova) + +/** * Get the rte_cryptodev structure device pointer for the device. Assumes a * valid device index.
lib/cryptodev/rte_cryptodev.c +10 -6
@@ -1941,11 +1941,11 @@ rte_cryptodev_asym_session_create(uint8_t dev_id, }int -rte_cryptodev_sym_session_free (uint8_t dev_id, -struct rte_cryptodev_sym_session *sess) +rte_cryptodev_sym_session_free (uint8_t dev_id,void *_sess) {struct rte_cryptodev *dev;struct rte_mempool *sess_mp; +struct rte_cryptodev_sym_session *sess = _sess;struct rte_cryptodev_sym_session_pool_private_data *pool_priv;if (sess ==NULL )
@@ -2060,10 +2060,11 @@ rte_cryptodev_asym_get_private_session_size(uint8_t dev_id) }int -rte_cryptodev_sym_session_set_user_data ( -struct rte_cryptodev_sym_session *sess,void *data, +rte_cryptodev_sym_session_set_user_data (void *_sess,void *data,uint16_t size) { +struct rte_cryptodev_sym_session *sess = _sess; +if (sess ==NULL )return -EINVAL ;
@@ -2075,8 +2076,10 @@ rte_cryptodev_sym_session_set_user_data( }void * -rte_cryptodev_sym_session_get_user_data (struct rte_cryptodev_sym_session *sess) +rte_cryptodev_sym_session_get_user_data (void *_sess) { +struct rte_cryptodev_sym_session *sess = _sess; +if (sess ==NULL || sess->user_data_sz ==0 )return NULL ;
@@ -2120,10 +2123,11 @@ sym_crypto_fill_status(struct rte_crypto_sym_vec *vec, int32_t errnum)uint32_t rte_cryptodev_sym_cpu_crypto_process (uint8_t dev_id, -struct rte_cryptodev_sym_session *sess,union rte_crypto_sym_ofs ofs, +void *_sess,union rte_crypto_sym_ofs ofs,struct rte_crypto_sym_vec *vec) {struct rte_cryptodev *dev; +struct rte_cryptodev_sym_session *sess = _sess;if (!rte_cryptodev_is_valid_dev (dev_id)) {sym_crypto_fill_status (vec,EINVAL );
lib/cryptodev/rte_cryptodev.h +26 -30
95 lines not on this page · raw message
lib/cryptodev/rte_cryptodev_trace.h +2 -6
@@ -73,14 +73,10 @@ RTE_TRACE_POINT(RTE_TRACE_POINT ( rte_cryptodev_trace_sym_session_create, - RTE_TRACE_POINT_ARGS(uint8_t dev_id, - struct rte_cryptodev_sym_session *sess, void *xforms, + RTE_TRACE_POINT_ARGS(uint8_t dev_id, void *sess, void *xforms, void *mempool), rte_trace_point_emit_u8(dev_id);rte_trace_point_emit_ptr (sess); -rte_trace_point_emit_u64 (sess->opaque_data); -rte_trace_point_emit_u8 (sess->driver_id); -rte_trace_point_emit_u16 (sess->user_data_sz);rte_trace_point_emit_ptr (xforms);rte_trace_point_emit_ptr (mempool); )
@@ -108,7 +104,7 @@ RTE_TRACE_POINT(RTE_TRACE_POINT ( rte_cryptodev_trace_sym_session_free, - RTE_TRACE_POINT_ARGS(struct rte_cryptodev_sym_session *sess), +RTE_TRACE_POINT_ARGS (void *sess), rte_trace_point_emit_ptr(sess); )
lib/ipsec/rte_ipsec_group.h +3 -2
@@ -45,14 +45,15 @@ static inline struct rte_ipsec_session * rte_ipsec_ses_from_crypto(const struct rte_crypto_op *cop) {const struct rte_security_session *ss; -const struct rte_cryptodev_sym_session *cs; +void *cs;if (cop->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION ) { ss = cop->sym[0 ].sec_session;return (struct rte_ipsec_session *)(uintptr_t )ss->opaque_data; }else if (cop->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { cs = cop->sym[0 ].session; -return (struct rte_ipsec_session *)(uintptr_t )cs->opaque_data; +return (struct rte_ipsec_session *)(uintptr_t ) +rte_cryptodev_sym_session_opaque_data_get (cs); }return NULL ; }
lib/ipsec/ses.c +2 -1
@@ -45,7 +45,8 @@ rte_ipsec_session_prepare(struct rte_ipsec_session *ss) ss->pkt_func = fp;if (ss->type ==RTE_SECURITY_ACTION_TYPE_NONE ) - ss->crypto.ses->opaque_data = (uintptr_t )ss; +rte_cryptodev_sym_session_opaque_data_set (ss->crypto.ses, + (uintptr_t )ss);else ss->security.ses->opaque_data = (uintptr_t )ss;
Signature · 1 line
2.34.1
RE: [EXT] [PATCH 3/3] cryptodev: hide sym session structure
Quoted 1 line
+#define
CRYPTO_SESS_OPAQUE_DATA_OFF0
CRYPTO_SESS_OPAQUE_DATA_OFF cannot be 0 as you have added a driver_id at start of struct.
Quoted 19 lines
lib/cryptodev/rte_cryptodev.h +19
-0+/** +* Get opaque data from session handle +*/ +static inline uint64_t +rte_cryptodev_sym_session_opaque_data_get (void *sess) +{ +return *((uint64_t *)sess -CRYPTO_SESS_OPAQUE_DATA_OFF ); +} + +/** +* Set opaque data in session handle +*/ +static inline void +rte_cryptodev_sym_session_opaque_data_set (void *sess,uint64_t opaque) +{ +uint64_t *data; + data = (((uint64_t *)sess) -CRYPTO_SESS_OPAQUE_DATA_OFF ); + *data = opaque; +}
v2 cover crypto/security session framework rework
In reply to Fan ZhangThis patchset reworks the symmetric crypto and security session data structure to use a single virtual/physical contiguous buffer for symmetric crypto/security session and driver private data. In addition the session data structure is now private. The session is represented as an opaque pointer in the application.
With the change the session is no longer supported to be accessed by multiple device drivers. For the same reason rte_cryptodev_sym_session_init/clear APIs are deprecated as rte_cryptodev_sym_session_create/free will initialize and clear the driver specific data field.
The change was also submitted last year during DPDK 21.11 timeframe also[1], but was not applied due to lack of feedback from community. Please help in getting this cleanup merged in this cycle.
Now the similar work was already done for asymmetric crypto. This patchset is rebased over current tree and fixes all the issues reported so far.
This patchset is a v2 for the patch that was sent by Fan Zhang(Intel) with a few changes
- Added security session rework also.
- fixed issues in [2] reported on mailing list.
- few other fixes.
Please review and provide feedback as soon as possible as this is intended to be merged in DPDK 22.11 RC1.
Currently the cnxk platform is tested with this change. Request everyone to review and test on their platform.
Special note to ixgbe and txgbe maintainers.
There is a wrong implementation for flow creation. Please check. A hack is added to bypass it. Please fix it separately.
[1] https://patches.dpdk.org/project/dpdk/cover/[email protected]/
[2] https://patches.dpdk.org/project/dpdk/cover/[email protected]/
Akhil Goyal (5):
cryptodev: rework session framework
cryptodev: hide sym session structure
security: remove priv mempool usage drivers/crypto: support security session get size op
security: hide session structure
Fan Zhang (1):
crypto/scheduler: use unified session
app/test-crypto-perf/cperf.h | 1 - app/test-crypto-perf/cperf_ops.c | 40 +-- app/test-crypto-perf/cperf_ops.h | 2 +- app/test-crypto-perf/cperf_test_latency.c | 9 +- app/test-crypto-perf/cperf_test_latency.h | 1 - .../cperf_test_pmd_cyclecount.c | 10 +- .../cperf_test_pmd_cyclecount.h | 1 - app/test-crypto-perf/cperf_test_throughput.c | 11 +- app/test-crypto-perf/cperf_test_throughput.h | 1 - app/test-crypto-perf/cperf_test_verify.c | 9 +- app/test-crypto-perf/cperf_test_verify.h | 1 - app/test-crypto-perf/main.c | 30 +- app/test-eventdev/test_perf_common.c | 35 +- app/test-eventdev/test_perf_common.h | 1 - app/test/test_cryptodev.c | 336 +++++------------- app/test/test_cryptodev_blockcipher.c | 16 +- app/test/test_cryptodev_security_ipsec.c | 2 +- app/test/test_cryptodev_security_ipsec.h | 2 +- app/test/test_event_crypto_adapter.c | 35 +- app/test/test_ipsec.c | 47 +-- app/test/test_ipsec_perf.c | 4 +- app/test/test_security.c | 178 ++-------- app/test/test_security_inline_proto.c | 26 +- doc/guides/prog_guide/cryptodev_lib.rst | 16 +- doc/guides/rel_notes/deprecation.rst | 9 + doc/guides/rel_notes/release_22_11.rst | 7 + drivers/common/cnxk/roc_cpt.c | 4 +- drivers/crypto/armv8/rte_armv8_pmd.c | 21 +- drivers/crypto/armv8/rte_armv8_pmd_ops.c | 32 +- drivers/crypto/bcmfs/bcmfs_sym_session.c | 39 +- drivers/crypto/bcmfs/bcmfs_sym_session.h | 3 +- drivers/crypto/caam_jr/caam_jr.c | 69 +--- drivers/crypto/ccp/ccp_crypto.c | 56 +-- drivers/crypto/ccp/ccp_pmd_ops.c | 32 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 - drivers/crypto/ccp/rte_ccp_pmd.c | 29 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 41 +-- drivers/crypto/cnxk/cn10k_ipsec.c | 45 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 38 +- drivers/crypto/cnxk/cn9k_ipsec.c | 54 +-- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 55 +-- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 16 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 70 ++-- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 6 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 69 +--- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 7 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 34 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 41 +-- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 48 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 62 +--- drivers/crypto/ipsec_mb/pmd_chacha_poly.c | 4 - drivers/crypto/ipsec_mb/pmd_kasumi.c | 10 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 9 +- drivers/crypto/ipsec_mb/pmd_zuc.c | 4 - drivers/crypto/mlx5/mlx5_crypto.c | 25 +- drivers/crypto/mvsam/rte_mrvl_pmd.c | 11 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 49 +-- drivers/crypto/nitrox/nitrox_sym.c | 39 +- drivers/crypto/null/null_crypto_pmd.c | 20 +- drivers/crypto/null/null_crypto_pmd_ops.c | 33 +- drivers/crypto/null/null_crypto_pmd_private.h | 2 - .../crypto/octeontx/otx_cryptodev_hw_access.h | 1 - drivers/crypto/octeontx/otx_cryptodev_ops.c | 67 +--- drivers/crypto/openssl/openssl_pmd_private.h | 2 - drivers/crypto/openssl/rte_openssl_pmd.c | 24 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 29 +- drivers/crypto/qat/dev/qat_sym_pmd_gen1.c | 1 + drivers/crypto/qat/qat_sym.c | 15 +- drivers/crypto/qat/qat_sym.h | 16 +- drivers/crypto/qat/qat_sym_session.c | 73 ++-- drivers/crypto/qat/qat_sym_session.h | 10 +- drivers/crypto/scheduler/scheduler_failover.c | 19 +- .../crypto/scheduler/scheduler_multicore.c | 17 + .../scheduler/scheduler_pkt_size_distr.c | 81 +++-- drivers/crypto/scheduler/scheduler_pmd_ops.c | 93 ++++- .../crypto/scheduler/scheduler_pmd_private.h | 120 ++++++- .../crypto/scheduler/scheduler_roundrobin.c | 11 +- drivers/crypto/virtio/virtio_cryptodev.c | 40 +-- drivers/crypto/virtio/virtio_rxtx.c | 3 +- drivers/net/cnxk/cn10k_ethdev_sec.c | 38 +- drivers/net/cnxk/cn9k_ethdev_sec.c | 32 +- drivers/net/iavf/iavf_ipsec_crypto.c | 23 +- drivers/net/iavf/iavf_ipsec_crypto.h | 2 +- drivers/net/ixgbe/ixgbe_ipsec.c | 31 +- drivers/net/txgbe/txgbe_ipsec.c | 32 +- examples/fips_validation/fips_dev_self_test.c | 40 +-- examples/fips_validation/main.c | 35 +- examples/ipsec-secgw/ipsec-secgw.c | 44 +-- examples/ipsec-secgw/ipsec.c | 16 +- examples/ipsec-secgw/ipsec.h | 1 - examples/ipsec-secgw/ipsec_worker.c | 2 +- examples/l2fwd-crypto/main.c | 54 +-- examples/vhost_crypto/main.c | 16 +- lib/cryptodev/cryptodev_pmd.h | 60 ++-- lib/cryptodev/cryptodev_trace_points.c | 6 - lib/cryptodev/rte_crypto.h | 3 +- lib/cryptodev/rte_crypto_sym.h | 11 +- lib/cryptodev/rte_cryptodev.c | 284 ++++++--------- lib/cryptodev/rte_cryptodev.h | 153 +++----- lib/cryptodev/rte_cryptodev_trace.h | 35 +- lib/cryptodev/version.map | 6 - lib/ipsec/rte_ipsec_group.h | 13 +- lib/ipsec/ses.c | 5 +- lib/pipeline/rte_table_action.c | 10 +- lib/security/rte_security.c | 33 +- lib/security/rte_security.h | 90 +++-- lib/security/rte_security_driver.h | 31 +- lib/vhost/rte_vhost_crypto.h | 3 - lib/vhost/vhost_crypto.c | 28 +- 109 files changed, 1255 insertions(+), 2413 deletions(-)
Signature · 1 line
2.25.1
PATCH v2 1/6 cryptodev: rework session framework
As per current design, rte_cryptodev_sym_session_create() and rte_cryptodev_sym_session_init() use separate mempool objects for a single session.
And structure rte_cryptodev_sym_session is not directly used by the application, it may cause ABI breakage if the structure is modified in future.
To address these two issues, the rte_cryptodev_sym_session_create will take one mempool object that the session and session private data are virtually/physically contiguous, and initializes both fields. The API rte_cryptodev_sym_session_init is removed.
rte_cryptodev_sym_session_create will now return an opaque session pointer which will be used by the app and other APIs.
In data path, opaque session pointer is attached to rte_crypto_op and the PMD can call an internal library API to get the session private data pointer based on the driver id.
Note: currently single session may be used by different device drivers, given it is initialized by them. After the change the session created by one device driver cannot be used or reinitialized by another driver.
--- app/test-crypto-perf/cperf_ops.c | 21 +- app/test-crypto-perf/cperf_test_latency.c | 6 +- .../cperf_test_pmd_cyclecount.c | 5 +- app/test-crypto-perf/cperf_test_throughput.c | 6 +- app/test-crypto-perf/cperf_test_verify.c | 6 +- app/test-crypto-perf/main.c | 29 +- app/test-eventdev/test_perf_common.c | 35 +- app/test-eventdev/test_perf_common.h | 1 - app/test/test_cryptodev.c | 302 +++++------------- app/test/test_cryptodev_blockcipher.c | 16 +- app/test/test_event_crypto_adapter.c | 35 +- app/test/test_ipsec.c | 42 +-- drivers/crypto/armv8/rte_armv8_pmd.c | 21 +- drivers/crypto/armv8/rte_armv8_pmd_ops.c | 32 +- drivers/crypto/bcmfs/bcmfs_sym_session.c | 38 +-- drivers/crypto/bcmfs/bcmfs_sym_session.h | 3 +- drivers/crypto/caam_jr/caam_jr.c | 28 +- drivers/crypto/ccp/ccp_crypto.c | 58 +--- drivers/crypto/ccp/ccp_pmd_ops.c | 32 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 - drivers/crypto/ccp/rte_ccp_pmd.c | 29 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 36 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 31 +- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 55 +--- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 14 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 31 +- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 3 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 37 +-- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 4 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 34 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 41 ++- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 48 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 29 +- drivers/crypto/ipsec_mb/pmd_chacha_poly.c | 4 - drivers/crypto/ipsec_mb/pmd_kasumi.c | 10 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 9 +- drivers/crypto/ipsec_mb/pmd_zuc.c | 4 - drivers/crypto/mlx5/mlx5_crypto.c | 26 +- drivers/crypto/mvsam/rte_mrvl_pmd.c | 8 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 21 +- drivers/crypto/nitrox/nitrox_sym.c | 39 +-- drivers/crypto/null/null_crypto_pmd.c | 19 +- drivers/crypto/null/null_crypto_pmd_ops.c | 33 +- drivers/crypto/null/null_crypto_pmd_private.h | 2 - .../crypto/octeontx/otx_cryptodev_hw_access.h | 1 - drivers/crypto/octeontx/otx_cryptodev_ops.c | 67 +--- drivers/crypto/openssl/openssl_pmd_private.h | 2 - drivers/crypto/openssl/rte_openssl_pmd.c | 24 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 29 +- drivers/crypto/qat/qat_sym.c | 10 +- drivers/crypto/qat/qat_sym.h | 4 +- drivers/crypto/qat/qat_sym_session.c | 40 +-- drivers/crypto/qat/qat_sym_session.h | 6 +- drivers/crypto/scheduler/scheduler_pmd_ops.c | 38 +-- drivers/crypto/virtio/virtio_cryptodev.c | 40 +-- drivers/crypto/virtio/virtio_rxtx.c | 3 +- examples/fips_validation/fips_dev_self_test.c | 30 +- examples/fips_validation/main.c | 35 +- examples/ipsec-secgw/ipsec-secgw.c | 10 +- examples/ipsec-secgw/ipsec.c | 7 +- examples/l2fwd-crypto/main.c | 54 +--- examples/vhost_crypto/main.c | 16 +- lib/cryptodev/cryptodev_pmd.h | 28 +- lib/cryptodev/cryptodev_trace_points.c | 6 - lib/cryptodev/rte_cryptodev.c | 276 ++++++---------- lib/cryptodev/rte_cryptodev.h | 123 ++----- lib/cryptodev/rte_cryptodev_trace.h | 33 +- lib/cryptodev/version.map | 6 - lib/pipeline/rte_table_action.c | 10 +- lib/vhost/rte_vhost_crypto.h | 3 - lib/vhost/vhost_crypto.c | 28 +- 71 files changed, 553 insertions(+), 1661 deletions(-)
app/test-crypto-perf/cperf_ops.c +10 -11
@@ -912,7 +912,6 @@ cperf_create_session(struct rte_mempool *sess_mp, &sess_conf, sess_mp, priv_mp); }#endif - sess =rte_cryptodev_sym_session_create (sess_mp);/* * cipher only */
@@ -937,8 +936,8 @@ cperf_create_session(struct rte_mempool *sess_mp, cipher_xform.cipher.iv.length =0 ; }/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, sess, &cipher_xform, - priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, &cipher_xform, + sess_mp);/* * auth only */
@@ -965,8 +964,8 @@ cperf_create_session(struct rte_mempool *sess_mp, auth_xform.auth.iv.length =0 ; }/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, sess, &auth_xform, - priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, &auth_xform, + sess_mp);/* * cipher and auth */
@@ -1024,13 +1023,13 @@ cperf_create_session(struct rte_mempool *sess_mp,if (options->op_type ==CPERF_CIPHER_THEN_AUTH ) { cipher_xform.next = &auth_xform;/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, - sess, &cipher_xform, priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, + &cipher_xform, sess_mp); }else {/* auth then cipher */ auth_xform.next = &cipher_xform;/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, - sess, &auth_xform, priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, + &auth_xform, sess_mp); } }else {/* options->op_type == CPERF_AEAD */ aead_xform.type =RTE_CRYPTO_SYM_XFORM_AEAD ;
@@ -1050,8 +1049,8 @@ cperf_create_session(struct rte_mempool *sess_mp, options->aead_aad_sz;/* Create crypto session */ -rte_cryptodev_sym_session_init (dev_id, - sess, &aead_xform, priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, &aead_xform, + sess_mp); }return sess;
app/test-crypto-perf/cperf_test_latency.c +2 -4
@@ -44,10 +44,8 @@ static void cperf_latency_test_free(struct cperf_latency_ctx *ctx) {if (ctx) { -if (ctx->sess) { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } +if (ctx->sess) +rte_cryptodev_sym_session_free (ctx->dev_id, ctx->sess);rte_mempool_free (ctx->pool);
app/test-crypto-perf/cperf_test_pmd_cyclecount.c +1 -4
@@ -73,10 +73,7 @@ cperf_pmd_cyclecount_test_free(struct cperf_pmd_cyclecount_ctx *ctx) (struct rte_security_session *)ctx->sess); } else#endif - { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } +rte_cryptodev_sym_session_free (ctx->dev_id, ctx->sess); }rte_mempool_free (ctx->pool);
app/test-crypto-perf/cperf_test_throughput.c +2 -4
@@ -50,10 +50,8 @@ cperf_throughput_test_free(struct cperf_throughput_ctx *ctx) (struct rte_security_session *)ctx->sess); }#endif - else { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } + else + rte_cryptodev_sym_session_free(ctx->dev_id, ctx->sess); }rte_mempool_free (ctx->pool);
app/test-crypto-perf/cperf_test_verify.c +2 -4
@@ -37,10 +37,8 @@ static void cperf_verify_test_free(struct cperf_verify_ctx *ctx) {if (ctx) { -if (ctx->sess) { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } +if (ctx->sess) +rte_cryptodev_sym_session_free (ctx->dev_id, ctx->sess);rte_mempool_free (ctx->pool);
app/test-crypto-perf/main.c +3 -26
@@ -96,35 +96,14 @@ fill_session_pool_socket(int32_t socket_id, uint32_t session_priv_size,char mp_name[RTE_MEMPOOL_NAMESIZE ];struct rte_mempool *sess_mp; -if (session_pool_socket[socket_id].priv_mp ==NULL ) { -snprintf (mp_name,RTE_MEMPOOL_NAMESIZE , -"priv_sess_mp_%u" , socket_id); - - sess_mp =rte_mempool_create (mp_name, - nb_sessions, - session_priv_size, -0 ,0 ,NULL ,NULL ,NULL , -NULL , socket_id, -0 ); - -if (sess_mp ==NULL ) { -printf ("Cannot create pool \"%s\" on socket %d\n" , - mp_name, socket_id); -return -ENOMEM ; - } - -printf ("Allocated pool \"%s\" on socket %d\n" , - mp_name, socket_id); - session_pool_socket[socket_id].priv_mp = sess_mp; - } -if (session_pool_socket[socket_id].sess_mp ==NULL ) {snprintf (mp_name,RTE_MEMPOOL_NAMESIZE ,"sess_mp_%u" , socket_id); sess_mp =rte_cryptodev_sym_session_pool_create (mp_name, - nb_sessions,0 ,0 ,0 , socket_id); + nb_sessions, session_priv_size,0 ,0 , + socket_id);if (sess_mp ==NULL ) { printf("Cannot create pool \"%s\" on socket %d\n",
@@ -135,6 +114,7 @@ fill_session_pool_socket(int32_t socket_id, uint32_t session_priv_size,printf ("Allocated pool \"%s\" on socket %d\n" , mp_name, socket_id); session_pool_socket[socket_id].sess_mp = sess_mp; + session_pool_socket[socket_id].priv_mp = sess_mp; }return 0 ;
@@ -322,12 +302,9 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs)return ret; qp_conf.mp_session = session_pool_socket[socket_id].sess_mp; - qp_conf.mp_session_private = - session_pool_socket[socket_id].priv_mp;if (opts->op_type ==CPERF_ASYM_MODEX ) { qp_conf.mp_session =NULL ; - qp_conf.mp_session_private =NULL ; } ret =rte_cryptodev_configure (cdev_id, &conf);
app/test-eventdev/test_perf_common.c +8 -27
@@ -864,18 +864,13 @@ cryptodev_sym_sess_create(struct prod_data *p, struct test_perf *t) cipher_xform.cipher.op =RTE_CRYPTO_CIPHER_OP_ENCRYPT ; cipher_xform.next =NULL ; - sess =rte_cryptodev_sym_session_create (t->ca_sess_pool); + sess =rte_cryptodev_sym_session_create (p->ca.cdev_id, &cipher_xform, + t->ca_sess_pool);if (sess ==NULL ) {evt_err ("Failed to create sym session" );return NULL ; } -if (rte_cryptodev_sym_session_init (p->ca.cdev_id, sess, &cipher_xform, - t->ca_sess_priv_pool)) { -evt_err ("Failed to init session" ); -return NULL ; - } -return sess; }
@@ -1382,15 +1377,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) goto err; } - t->ca_sess_pool =rte_cryptodev_sym_session_pool_create ( -"ca_sess_pool" , nb_sessions,0 ,0 , -sizeof (union rte_event_crypto_metadata ),SOCKET_ID_ANY ); -if (t->ca_sess_pool ==NULL ) { -evt_err ("Failed to create sym session pool" ); - ret = -ENOMEM ; - goto err; - } - max_session_size =0 ;for (cdev_id =0 ; cdev_id < cdev_count; cdev_id++) {unsigned int session_size;
@@ -1401,12 +1387,11 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) max_session_size = session_size; } - max_session_size +=sizeof (union rte_event_crypto_metadata ); - t->ca_sess_priv_pool =rte_mempool_create ( -"ca_sess_priv_pool" , nb_sessions, max_session_size,0 ,0 ,NULL , -NULL ,NULL ,NULL ,SOCKET_ID_ANY ,0 ); -if (t->ca_sess_priv_pool ==NULL ) { -evt_err ("failed to create sym session private pool" ); + t->ca_sess_pool =rte_cryptodev_sym_session_pool_create ( +"ca_sess_pool" , nb_sessions, max_session_size,0 , +sizeof (union rte_event_crypto_metadata ),SOCKET_ID_ANY ); +if (t->ca_sess_pool ==NULL ) { +evt_err ("Failed to create sym session pool" ); ret = -ENOMEM ; goto err; }
@@ -1446,7 +1431,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) qp_conf.nb_descriptors =NB_CRYPTODEV_DESCRIPTORS ; qp_conf.mp_session = t->ca_sess_pool; - qp_conf.mp_session_private = t->ca_sess_priv_pool;for (qp_id =0 ; qp_id < conf.nb_queue_pairs; qp_id++) { ret = rte_cryptodev_queue_pair_setup(
@@ -1467,7 +1451,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt)rte_mempool_free (t->ca_op_pool);rte_mempool_free (t->ca_sess_pool); -rte_mempool_free (t->ca_sess_priv_pool);rte_mempool_free (t->ca_asym_sess_pool);return ret;
@@ -1492,8 +1475,7 @@ perf_cryptodev_destroy(struct evt_test *test, struct evt_options *opt)for (flow_id =0 ; flow_id < t->nb_flows; flow_id++) { sess = p->ca.crypto_sess[flow_id]; cdev_id = p->ca.cdev_id; -rte_cryptodev_sym_session_clear (cdev_id, sess); -rte_cryptodev_sym_session_free (sess); +rte_cryptodev_sym_session_free (cdev_id, sess); } rte_event_crypto_adapter_queue_pair_del(
@@ -1509,7 +1491,6 @@ perf_cryptodev_destroy(struct evt_test *test, struct evt_options *opt)rte_mempool_free (t->ca_op_pool);rte_mempool_free (t->ca_sess_pool); -rte_mempool_free (t->ca_sess_priv_pool);rte_mempool_free (t->ca_asym_sess_pool); }
app/test-eventdev/test_perf_common.h +0 -1
@@ -70,7 +70,6 @@ struct test_perf { RTE_EVENT_TIMER_ADAPTER_NUM_MAX] __rte_cache_aligned;struct rte_mempool *ca_op_pool;struct rte_mempool *ca_sess_pool; -struct rte_mempool *ca_sess_priv_pool;struct rte_mempool *ca_asym_sess_pool; } __rte_cache_aligned;
app/test/test_cryptodev.c +86 -216
662 lines not on this page · raw message
app/test/test_cryptodev_blockcipher.c +5 -11
@@ -68,7 +68,6 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t,struct rte_mempool *mbuf_pool, struct rte_mempool *op_mpool, struct rte_mempool *sess_mpool, - struct rte_mempool *sess_priv_mpool, uint8_t dev_id, char *test_msg) {
@@ -514,11 +513,9 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t,*/ if (!(t->feature_mask &BLOCKCIPHER_TEST_FEATURE_SESSIONLESS ) && nb_iterates ==0 ) { - sess =rte_cryptodev_sym_session_create (sess_mpool); - - status =rte_cryptodev_sym_session_init (dev_id, sess, - init_xform, sess_priv_mpool); -if (status == -ENOTSUP ) { + sess =rte_cryptodev_sym_session_create (dev_id, init_xform, + sess_mpool); +if (sess ==NULL ) {snprintf (test_msg,BLOCKCIPHER_TEST_MSG_LEN ,"UNSUPPORTED" ); status =TEST_SKIPPED ; goto error_exit;
@@ -801,10 +798,8 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, error_exit:if (!(t->feature_mask &BLOCKCIPHER_TEST_FEATURE_SESSIONLESS )) { -if (sess) { -rte_cryptodev_sym_session_clear (dev_id, sess); -rte_cryptodev_sym_session_free (sess); - } +if (sess) +rte_cryptodev_sym_session_free (dev_id, sess);rte_free (cipher_xform);rte_free (auth_xform); }
@@ -829,7 +824,6 @@ blockcipher_test_case_run(const void *data) p_testsuite_params->mbuf_pool, p_testsuite_params->op_mpool, p_testsuite_params->session_mpool, - p_testsuite_params->session_priv_mpool, p_testsuite_params->valid_devs[0 ], test_msg);return status;
app/test/test_event_crypto_adapter.c +5 -30
@@ -157,7 +157,6 @@ struct event_crypto_adapter_test_params {struct rte_mempool *op_mpool;struct rte_mempool *asym_op_mpool;struct rte_mempool *session_mpool; -struct rte_mempool *session_priv_mpool;struct rte_mempool *asym_sess_mpool;struct rte_cryptodev_config *config;uint8_t crypto_event_port_id;
@@ -307,15 +306,10 @@ test_op_forward_mode(uint8_t session_less) sym_op = op->sym;if (!session_less) { - sess =rte_cryptodev_sym_session_create ( - params.session_mpool); + sess =rte_cryptodev_sym_session_create (TEST_CDEV_ID , + &cipher_xform, params.session_mpool);TEST_ASSERT_NOT_NULL (sess,"Session creation failed\n" ); -/* Create Crypto session*/ - ret =rte_cryptodev_sym_session_init (TEST_CDEV_ID , sess, - &cipher_xform, params.session_priv_mpool); -TEST_ASSERT_SUCCESS (ret,"Failed to init session\n" ); - ret =rte_event_crypto_adapter_caps_get (evdev,TEST_CDEV_ID , &cap);TEST_ASSERT_SUCCESS (ret,"Failed to get adapter capabilities\n" );
@@ -683,8 +677,8 @@ test_op_new_mode(uint8_t session_less) sym_op = op->sym;if (!session_less) { - sess =rte_cryptodev_sym_session_create ( - params.session_mpool); + sess =rte_cryptodev_sym_session_create (TEST_CDEV_ID , + &cipher_xform, params.session_mpool);TEST_ASSERT_NOT_NULL (sess,"Session creation failed\n" ); ret = rte_event_crypto_adapter_caps_get(evdev, TEST_CDEV_ID,
@@ -699,9 +693,6 @@ test_op_new_mode(uint8_t session_less)RTE_CRYPTO_OP_WITH_SESSION , &m_data, sizeof(m_data)); } - ret =rte_cryptodev_sym_session_init (TEST_CDEV_ID , sess, - &cipher_xform, params.session_priv_mpool); -TEST_ASSERT_SUCCESS (ret,"Failed to init session\n" );rte_crypto_op_attach_sym_session (op, sess); }else {
@@ -994,22 +985,12 @@ configure_cryptodev(void) params.session_mpool =rte_cryptodev_sym_session_pool_create ("CRYPTO_ADAPTER_SESSION_MP" , -MAX_NB_SESSIONS ,0 ,0 , +MAX_NB_SESSIONS , session_size,0 ,sizeof (union rte_event_crypto_metadata ),SOCKET_ID_ANY );TEST_ASSERT_NOT_NULL (params.session_mpool,"session mempool allocation failed\n" ); - params.session_priv_mpool =rte_mempool_create ( -"CRYPTO_AD_SESS_MP_PRIV" , -MAX_NB_SESSIONS , - session_size, -0 ,0 ,NULL ,NULL ,NULL , -NULL ,SOCKET_ID_ANY , -0 ); -TEST_ASSERT_NOT_NULL (params.session_priv_mpool, -"session mempool allocation failed\n" ); -rte_cryptodev_info_get (TEST_CDEV_ID , &info); while ((capability = &info.capabilities[i++])->op !=
@@ -1048,7 +1029,6 @@ configure_cryptodev(void) qp_conf.nb_descriptors =DEFAULT_NUM_OPS_INFLIGHT ; qp_conf.mp_session = params.session_mpool; - qp_conf.mp_session_private = params.session_priv_mpool; TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( TEST_CDEV_ID, TEST_CDEV_QP_ID, &qp_conf,
@@ -1410,11 +1390,6 @@ crypto_teardown(void)rte_mempool_free (params.session_mpool); params.session_mpool =NULL ; } -if (params.session_priv_mpool !=NULL ) { -rte_mempool_avail_count (params.session_priv_mpool); -rte_mempool_free (params.session_priv_mpool); - params.session_priv_mpool =NULL ; - }/* Free asym session mempool */ if (params.asym_sess_mpool !=NULL ) {
app/test/test_ipsec.c +7 -35
79 lines not on this page · raw message
drivers/crypto/armv8/rte_armv8_pmd.c +3 -18
@@ -521,34 +521,23 @@ get_session(struct armv8_crypto_qp *qp, struct rte_crypto_op *op)if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {/* get existing session */ if (likely (op->sym->session !=NULL )) { - sess = (struct armv8_crypto_session *) -get_sym_session_private_data ( - op->sym->session, - cryptodev_driver_id); + sess = op->sym->session->data; } }else {/* provide internal session */ -void *_sess =NULL ; -void *_sess_private_data =NULL ; +struct rte_cryptodev_sym_session *_sess =NULL ;if (rte_mempool_get (qp->sess_mp, (void **)&_sess))return NULL ; -if (rte_mempool_get (qp->sess_mp_priv, - (void **)&_sess_private_data)) -return NULL ; - - sess = (struct armv8_crypto_session *)_sess_private_data; + sess = (struct armv8_crypto_session *)_sess->data;if (unlikely (armv8_crypto_set_session_parameters (sess, op->sym->xform) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); -rte_mempool_put (qp->sess_mp_priv, _sess_private_data); sess =NULL ; } op->sym->session = (struct rte_cryptodev_sym_session *)_sess; -set_sym_session_private_data (op->sym->session, - cryptodev_driver_id, _sess_private_data); }if (unlikely (sess ==NULL ))
@@ -674,10 +663,6 @@ process_op(struct armv8_crypto_qp *qp, struct rte_crypto_op *op,/* Free session if a session-less crypto op */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sess,0 ,sizeof (struct armv8_crypto_session )); -memset (op->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - op->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sess);rte_mempool_put (qp->sess_mp, op->sym->session); op->sym->session =NULL ; }
drivers/crypto/armv8/rte_armv8_pmd_ops.c +5 -27
@@ -267,8 +267,7 @@ armv8_crypto_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int armv8_crypto_pmd_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;
@@ -278,43 +277,22 @@ armv8_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev,return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -CDEV_LOG_ERR ( -"Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = sess->data; ret =armv8_crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {ARMV8_CRYPTO_LOG_ERR ("failed configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/** Clear the memory of session so it doesn't leave key material behind */ static void -armv8_crypto_pmd_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -/* Zero out the whole structure */ -if (sess_priv) { -memset (sess_priv,0 ,sizeof (struct armv8_crypto_session )); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +armv8_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}struct rte_cryptodev_ops armv8_crypto_pmd_ops = { .dev_configure = armv8_crypto_pmd_config,
drivers/crypto/bcmfs/bcmfs_sym_session.c +7 -31
@@ -211,8 +211,7 @@ bcmfs_sym_get_session(struct rte_crypto_op *op) }else if (likely (op->sym->session !=NULL )) {/* get existing session */ sess = (struct bcmfs_sym_session *) -get_sym_session_private_data (op->sym->session, - cryptodev_bcmfs_driver_id); + op->sym->session->driver_priv_data; }if (sess ==NULL )
@@ -222,10 +221,9 @@ bcmfs_sym_get_session(struct rte_crypto_op *op) }int -bcmfs_sym_session_configure (struct rte_cryptodev *dev, +bcmfs_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;
@@ -235,45 +233,23 @@ bcmfs_sym_session_configure(struct rte_cryptodev *dev,return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -BCMFS_DP_LOG (ERR , -"Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = (void *)sess->driver_priv_data; ret =crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {BCMFS_DP_LOG (ERR ,"Failed configure session parameters" ); -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/* Clear the memory of session so it doesn't leave key material behind */ void -bcmfs_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -if (sess_priv) { -struct rte_mempool *sess_mp; - -memset (sess_priv,0 ,sizeof (struct bcmfs_sym_session )); - sess_mp =rte_mempool_from_obj (sess_priv); - -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +bcmfs_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}unsigned int bcmfs_sym_session_get_private_size(struct rte_cryptodev *dev __rte_unused)
drivers/crypto/bcmfs/bcmfs_sym_session.h +1 -2
@@ -93,8 +93,7 @@ bcmfs_process_crypto_op(struct rte_crypto_op *op,int bcmfs_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool); +struct rte_cryptodev_sym_session *sess); void bcmfs_sym_session_clear(struct rte_cryptodev *dev,
drivers/crypto/caam_jr/caam_jr.c +6 -22
@@ -1357,8 +1357,7 @@ caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp)switch (op->sess_type) {case RTE_CRYPTO_OP_WITH_SESSION : ses = (struct caam_jr_session *) -get_sym_session_private_data (op->sym->session, - cryptodev_driver_id); + op->sym->session->driver_priv_data;break ;case RTE_CRYPTO_OP_SECURITY_SESSION : ses = (struct caam_jr_session *)
@@ -1692,54 +1691,39 @@ caam_jr_set_session_parameters(struct rte_cryptodev *dev, }static int -caam_jr_sym_session_configure (struct rte_cryptodev *dev, +caam_jr_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;PMD_INIT_FUNC_TRACE (); - -if (rte_mempool_get (mempool, &sess_private_data)) { -CAAM_JR_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } - + sess_private_data = (void *)sess->driver_priv_data;memset (sess_private_data,0 ,sizeof (struct caam_jr_session )); ret =caam_jr_set_session_parameters (dev, xform, sess_private_data);if (ret !=0 ) {CAAM_JR_ERR ("failed to configure session parameters" );/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, sess_private_data); -return 0 ; }/* Clear the memory of session so it doesn't leave key material behind */ static void -caam_jr_sym_session_clear (struct rte_cryptodev *dev, +caam_jr_sym_session_clear(struct rte_cryptodev *dev __rte_unused,struct rte_cryptodev_sym_session *sess) { -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = (void *)sess->driver_priv_data;struct caam_jr_session *s = (struct caam_jr_session *)sess_priv;PMD_INIT_FUNC_TRACE ();if (sess_priv) { -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -rte_free (s->cipher_key.data);rte_free (s->auth_key.data); -memset (s,0 ,sizeof (struct caam_jr_session )); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); } }
drivers/crypto/ccp/ccp_crypto.c +16 -42
142 lines not on this page · raw message
drivers/crypto/ccp/ccp_pmd_ops.c +6 -26
@@ -727,7 +727,6 @@ ccp_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, } qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private;/* mempool for batch info */ qp->batch_mp = rte_mempool_create(
@@ -757,8 +756,7 @@ ccp_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int ccp_pmd_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {int ret;void *sess_private_data;
@@ -769,40 +767,22 @@ ccp_pmd_sym_session_configure(struct rte_cryptodev *dev,return -ENOMEM ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -CCP_LOG_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = (void *)sess->driver_priv_data; + internals = (struct ccp_private *)dev->data->dev_private; ret =ccp_set_session_parameters (sess_private_data, xform, internals);if (ret !=0 ) {CCP_LOG_ERR ("failed configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data);return 0 ; }static void -ccp_pmd_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -if (sess_priv) { -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); - -rte_mempool_put (sess_mp, sess_priv); -memset (sess_priv,0 ,sizeof (struct ccp_session )); -set_sym_session_private_data (sess, index,NULL ); - } -} +ccp_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}struct rte_cryptodev_ops ccp_ops = { .dev_configure = ccp_pmd_config,
drivers/crypto/ccp/ccp_pmd_private.h +0 -2
@@ -78,8 +78,6 @@ struct ccp_qp {/**< Ring for placing process packets */ struct rte_mempool *sess_mp;/**< Session Mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session Private Data Mempool */ struct rte_mempool *batch_mp;/**< Session Mempool for batch info */ struct rte_cryptodev_stats qp_stats;
drivers/crypto/ccp/rte_ccp_pmd.c +8 -21
@@ -56,33 +56,23 @@ get_ccp_session(struct ccp_qp *qp, struct rte_crypto_op *op)if (unlikely (op->sym->session ==NULL ))return NULL ; - sess = (struct ccp_session *) -get_sym_session_private_data ( - op->sym->session, - ccp_cryptodev_driver_id); + sess = (void *)op->sym->session->driver_priv_data; }else if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -void *_sess; -void *_sess_private_data =NULL ; +struct rte_cryptodev_sym_session *_sess;struct ccp_private *internals; -if (rte_mempool_get (qp->sess_mp, &_sess)) -return NULL ; -if (rte_mempool_get (qp->sess_mp, (void **)&_sess_private_data)) +if (rte_mempool_get (qp->sess_mp, (void **)&_sess))return NULL ; - sess = (struct ccp_session *)_sess_private_data; + sess = (void *)_sess->driver_priv_data; internals = (struct ccp_private *)qp->dev->data->dev_private;if (unlikely (ccp_set_session_parameters (sess, op->sym->xform, internals) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); -rte_mempool_put (qp->sess_mp_priv, _sess_private_data); sess =NULL ; } - op->sym->session = (struct rte_cryptodev_sym_session *)_sess; -set_sym_session_private_data (op->sym->session, - ccp_cryptodev_driver_id, - _sess_private_data); + op->sym->session = _sess; }return sess;
@@ -161,13 +151,10 @@ ccp_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,for (i =0 ; i < nb_dequeued; i++)if (unlikely (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS )) { -struct ccp_session *sess = (struct ccp_session *) -get_sym_session_private_data ( - ops[i]->sym->session, - ccp_cryptodev_driver_id); +struct ccp_session *sess = + (void *)ops[i]->sym->session->driver_priv_data; -rte_mempool_put (qp->sess_mp_priv, - sess); +memset (sess,0 ,sizeof (*sess));rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ;
drivers/crypto/cnxk/cn10k_cryptodev_ops.c +14 -22
@@ -22,24 +22,23 @@static inline struct cnxk_se_sess *cn10k_cpt_sym_temp_sess_create (struct cnxk_cpt_qp *qp,struct rte_crypto_op *op) { -const int driver_id = cn10k_cryptodev_driver_id;struct rte_crypto_sym_op *sym_op = op->sym;struct rte_cryptodev_sym_session *sess;struct cnxk_se_sess *priv;int ret;/* Create temporary session */ - sess =rte_cryptodev_sym_session_create (qp->sess_mp); -if (sess ==NULL ) +if (rte_mempool_get (qp->sess_mp, (void **)&sess) <0 )return NULL ; - ret =sym_session_configure (qp->lf.roc_cpt, driver_id, sym_op->xform, - sess, qp->sess_mp_priv); -if (ret) + ret =sym_session_configure (qp->lf.roc_cpt, sym_op->xform, + sess); +if (ret) { +rte_mempool_put (qp->sess_mp, (void *)sess); goto sess_put; + } - priv =get_sym_session_private_data (sess, driver_id); - + priv = (void *)sess->driver_priv_data; sym_op->session = sess;return priv;
@@ -111,8 +110,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],return 0 ; w7 = sec_sess->sa.inst.w7; }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { - sess =get_sym_session_private_data ( - sym_op->session, cn10k_cryptodev_driver_id); + sess = (void *)sym_op->session->driver_priv_data; ret =cpt_sym_inst_fill (qp, op, sess, infl_req, &inst[0 ]);if (unlikely (ret))
@@ -128,8 +126,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], ret =cpt_sym_inst_fill (qp, op, sess, infl_req, &inst[0 ]);if (unlikely (ret)) { -sym_session_clear (cn10k_cryptodev_driver_id, - op->sym->session); +sym_session_clear (op->sym->session);rte_mempool_put (qp->sess_mp, op->sym->session);return 0 ; }
@@ -296,8 +293,9 @@ cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, }else if (sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - sess, cn10k_cryptodev_driver_id); + priv = (void *)( + ((struct rte_cryptodev_sym_session *)sess)-> + driver_priv_data); priv->qp = qp; priv->cpt_inst_w2 = w2; }else
@@ -334,8 +332,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - op->sym->session, cn10k_cryptodev_driver_id); + priv = (void *)op->sym->session->driver_priv_data; *qp = priv->qp; *w2 = priv->cpt_inst_w2; }else {
@@ -543,7 +540,6 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, {const uint8_t uc_compcode = res->uc_compcode;const uint8_t compcode = res->compcode; -unsigned int sz; cop->status =RTE_CRYPTO_OP_STATUS_SUCCESS ;
@@ -620,11 +616,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, temp_sess_free:if (unlikely (cop->sess_type ==RTE_CRYPTO_OP_SESSIONLESS )) {if (cop->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) { -sym_session_clear (cn10k_cryptodev_driver_id, - cop->sym->session); - sz =rte_cryptodev_sym_get_existing_header_session_size ( - cop->sym->session); -memset (cop->sym->session,0 , sz); +sym_session_clear (cop->sym->session);rte_mempool_put (qp->sess_mp, cop->sym->session); cop->sym->session =NULL ; }
drivers/crypto/cnxk/cn9k_cryptodev_ops.c +10 -21
@@ -56,23 +56,20 @@ cn9k_cpt_sec_inst_fill(struct rte_crypto_op *op,static inline struct cnxk_se_sess *cn9k_cpt_sym_temp_sess_create (struct cnxk_cpt_qp *qp,struct rte_crypto_op *op) { -const int driver_id = cn9k_cryptodev_driver_id;struct rte_crypto_sym_op *sym_op = op->sym;struct rte_cryptodev_sym_session *sess;struct cnxk_se_sess *priv;int ret;/* Create temporary session */ - sess =rte_cryptodev_sym_session_create (qp->sess_mp); -if (sess ==NULL ) +if (rte_mempool_get (qp->sess_mp, (void **)&sess) <0 )return NULL ; - ret =sym_session_configure (qp->lf.roc_cpt, driver_id, sym_op->xform, - sess, qp->sess_mp_priv); + ret =sym_session_configure (qp->lf.roc_cpt, sym_op->xform, sess);if (ret) goto sess_put; - priv =get_sym_session_private_data (sess, driver_id); + priv = (void *)sess->driver_priv_data; sym_op->session = sess;
@@ -95,8 +92,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { sym_op = op->sym; - sess =get_sym_session_private_data ( - sym_op->session, cn9k_cryptodev_driver_id); + sess = (void *)sym_op->session->driver_priv_data; ret =cpt_sym_inst_fill (qp, op, sess, infl_req, inst); inst->w7.u64 = sess->cpt_inst_w7; }else if (op->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION )
@@ -110,8 +106,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, ret =cpt_sym_inst_fill (qp, op, sess, infl_req, inst);if (unlikely (ret)) { -sym_session_clear (cn9k_cryptodev_driver_id, - op->sym->session); +sym_session_clear (op->sym->session);rte_mempool_put (qp->sess_mp, op->sym->session); } inst->w7.u64 = sess->cpt_inst_w7;
@@ -349,8 +344,9 @@ cn9k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, }else if (sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - sess, cn9k_cryptodev_driver_id); + priv = (void *)(( + (struct rte_cryptodev_sym_session *)sess)-> + driver_priv_data); priv->qp = qp; priv->cpt_inst_w2 = w2; }else
@@ -387,8 +383,7 @@ cn9k_ca_meta_info_extract(struct rte_crypto_op *op, }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - op->sym->session, cn9k_cryptodev_driver_id); + priv = (void *)op->sym->session->driver_priv_data; *qp = priv->qp; inst->w2.u64 = priv->cpt_inst_w2; }else {
@@ -583,8 +578,6 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop,struct cpt_inflight_req *infl_req, struct cpt_cn9k_res_s *res) { - unsigned int sz; - if (likely(res->compcode == CPT_COMP_GOOD)) { if (unlikely(res->uc_compcode)) { if (res->uc_compcode == ROC_SE_ERR_GC_ICV_MISCOMPARE)
@@ -645,11 +638,7 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, temp_sess_free:if (unlikely (cop->sess_type ==RTE_CRYPTO_OP_SESSIONLESS )) {if (cop->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) { -sym_session_clear (cn9k_cryptodev_driver_id, - cop->sym->session); - sz =rte_cryptodev_sym_get_existing_header_session_size ( - cop->sym->session); -memset (cop->sym->session,0 , sz); +sym_session_clear (cop->sym->session);rte_mempool_put (qp->sess_mp, cop->sym->session); cop->sym->session =NULL ; }
drivers/crypto/cnxk/cnxk_cryptodev_ops.c +11 -44
104 lines not on this page · raw message
drivers/crypto/cnxk/cnxk_cryptodev_ops.h +5 -9
@@ -78,8 +78,6 @@ struct cnxk_cpt_qp {/**< Crypto adapter related info */ struct rte_mempool *sess_mp;/**< Session mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session private data mempool */ }; int cnxk_cpt_dev_config(struct rte_cryptodev *dev,
@@ -104,18 +102,16 @@ unsigned int cnxk_cpt_sym_session_get_size(struct rte_cryptodev *dev);int cnxk_cpt_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *pool); +struct rte_cryptodev_sym_session *sess); -int sym_session_configure (struct roc_cpt *roc_cpt,int driver_id, +int sym_session_configure (struct roc_cpt *roc_cpt,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *pool); +struct rte_cryptodev_sym_session *sess);void cnxk_cpt_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess); +struct rte_cryptodev_sym_session *sess); -void sym_session_clear (int driver_id,struct rte_cryptodev_sym_session *sess); +void sym_session_clear (struct rte_cryptodev_sym_session *sess); unsigned int cnxk_ae_session_size_get(struct rte_cryptodev *dev __rte_unused);
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +7 -24
@@ -1379,8 +1379,7 @@ build_sec_fd(struct rte_crypto_op *op,dpaa2_sec_session *sess;if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa2_sec_session *)get_sym_session_private_data ( - op->sym->session, cryptodev_driver_id); + sess = (dpaa2_sec_session *)op->sym->session->driver_priv_data;#ifdef RTE_LIB_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa2_sec_session *)get_sec_session_private_data(
@@ -1678,8 +1677,7 @@ dpaa2_sec_dump(struct rte_crypto_op *op)struct rte_crypto_sym_op *sym_op;if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa2_sec_session *)get_sym_session_private_data ( - op->sym->session, cryptodev_driver_id); + sess = (dpaa2_sec_session *)op->sym->session->driver_priv_data;#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa2_sec_session *)get_sec_session_private_data(
@@ -3754,51 +3752,36 @@ dpaa2_sec_security_session_destroy(void *dev __rte_unused, }#endif static int -dpaa2_sec_sym_session_configure (struct rte_cryptodev *dev, +dpaa2_sec_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *sess_private_data; +void *sess_private_data = (void *)sess->driver_priv_data;int ret; -if (rte_mempool_get (mempool, &sess_private_data)) { -DPAA2_SEC_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } - ret =dpaa2_sec_set_session_parameters (xform, sess_private_data);if (ret !=0 ) {DPAA2_SEC_ERR ("Failed to configure session parameters" );/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/** Clear the memory of session so it doesn't leave key material behind */ static void -dpaa2_sec_sym_session_clear (struct rte_cryptodev *dev, +dpaa2_sec_sym_session_clear(struct rte_cryptodev *dev __rte_unused,struct rte_cryptodev_sym_session *sess) {PMD_INIT_FUNC_TRACE (); -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = (void *)sess->driver_priv_data;dpaa2_sec_session *s = (dpaa2_sec_session *)sess_priv;if (sess_priv) {rte_free (s->ctxt);rte_free (s->cipher_key.data);rte_free (s->auth_key.data); -memset (s,0 ,sizeof (dpaa2_sec_session)); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); } }
drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c +1 -2
@@ -1012,8 +1012,7 @@ dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess = (dpaa2_sec_session *)get_sec_session_private_data ( session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa2_sec_session *)get_sym_session_private_data ( - session_ctx.crypto_sess, cryptodev_driver_id); + sess = (void *)session_ctx.crypto_sess->driver_priv_data; else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa2_sec_raw_dequeue_burst;
drivers/crypto/dpaa_sec/dpaa_sec.c +7 -30
@@ -670,10 +670,7 @@ dpaa_sec_dump(struct dpaa_sec_op_ctx *ctx, struct dpaa_sec_qp *qp)struct qm_sg_entry sg[2 ];if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa_sec_session *) -get_sym_session_private_data ( - op->sym->session, - dpaa_cryptodev_driver_id); + sess = (dpaa_sec_session *)op->sym->session->driver_priv_data;#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa_sec_session *)
@@ -1927,10 +1924,8 @@ dpaa_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops,switch (op->sess_type) {case RTE_CRYPTO_OP_WITH_SESSION : - ses = (dpaa_sec_session *) -get_sym_session_private_data ( - op->sym->session, - dpaa_cryptodev_driver_id); + ses = (void *) + op->sym->session->driver_priv_data;break ;#ifdef RTE_LIB_SECURITY case RTE_CRYPTO_OP_SECURITY_SESSION :
@@ -2676,31 +2671,19 @@ dpaa_sec_set_session_parameters(struct rte_cryptodev *dev,static int dpaa_sec_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *sess_private_data; +void *sess_private_data = (void *)sess->driver_priv_data;int ret;PMD_INIT_FUNC_TRACE (); -if (rte_mempool_get (mempool, &sess_private_data)) { -DPAA_SEC_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } - ret =dpaa_sec_set_session_parameters (dev, xform, sess_private_data);if (ret !=0 ) {DPAA_SEC_ERR ("failed to configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); - ret =dpaa_sec_prep_cdb (sess_private_data);if (ret) {DPAA_SEC_ERR ("Unable to prepare sec cdb" );
@@ -2714,7 +2697,6 @@ static inline void free_session_memory(struct rte_cryptodev *dev, dpaa_sec_session *s) { struct dpaa_sec_dev_private *qi = dev->data->dev_private; -struct rte_mempool *sess_mp =rte_mempool_from_obj ((void *)s);uint8_t i;for (i =0 ; i <MAX_DPAA_CORES ; i++) {
@@ -2724,7 +2706,6 @@ free_session_memory(struct rte_cryptodev *dev, dpaa_sec_session *s) s->qp[i] =NULL ; }free_session_data (s); -rte_mempool_put (sess_mp, (void *)s); }/** Clear the memory of session so it doesn't leave key material behind */
@@ -2733,14 +2714,10 @@ dpaa_sec_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) {PMD_INIT_FUNC_TRACE (); -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = (void *)sess->driver_priv_data;dpaa_sec_session *s = (dpaa_sec_session *)sess_priv; -if (sess_priv) { -free_session_memory (dev, s); -set_sym_session_private_data (sess, index,NULL ); - } +free_session_memory (dev, s); }#ifdef RTE_LIB_SECURITY
drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c +2 -2
@@ -1017,8 +1017,8 @@ dpaa_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess = (dpaa_sec_session *)get_sec_session_private_data ( session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa_sec_session *)get_sym_session_private_data ( - session_ctx.crypto_sess, dpaa_cryptodev_driver_id); + sess = (dpaa_sec_session *) + session_ctx.crypto_sess->driver_priv_data; else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa_sec_raw_dequeue_burst;
drivers/crypto/ipsec_mb/ipsec_mb_ops.c +6 -28
@@ -264,7 +264,6 @@ ipsec_mb_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, qp->pmd_type = internals->pmd_type; qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private; qp->ingress_queue =ipsec_mb_qp_create_processed_ops_ring (qp, qp_conf->nb_descriptors, socket_id);
@@ -312,9 +311,8 @@ ipsec_mb_sym_session_get_size(struct rte_cryptodev *dev)int ipsec_mb_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess,struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *sess_private_data;struct ipsec_mb_dev_private *internals = dev->data->dev_private;struct ipsec_mb_internals *pmd_data = &ipsec_mb_pmds[internals->pmd_type];
@@ -330,42 +328,22 @@ ipsec_mb_sym_session_configure(return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -IPSEC_MB_LOG (ERR ,"Couldn't get object from session mempool" ); -free_mb_mgr (mb_mgr); -return -ENOMEM ; - } - - ret = (*pmd_data->session_configure)(mb_mgr, sess_private_data, xform); + ret = (*pmd_data->session_configure)(mb_mgr, + (void *)sess->driver_priv_data, xform);if (ret !=0 ) {IPSEC_MB_LOG (ERR ,"failed configure session parameters" );/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);free_mb_mgr (mb_mgr);return ret; } -set_sym_session_private_data (sess, dev->driver_id, sess_private_data); -free_mb_mgr (mb_mgr);return 0 ; }/** Clear the session memory */ void -ipsec_mb_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -/* Zero out the whole structure */ -if (sess_priv) { -memset (sess_priv,0 ,ipsec_mb_sym_session_get_size (dev)); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); - -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +ipsec_mb_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}
drivers/crypto/ipsec_mb/ipsec_mb_private.h +19 -22
85 lines not on this page · raw message
drivers/crypto/ipsec_mb/pmd_aesni_gcm.c +14 -34
90 lines not on this page · raw message
drivers/crypto/ipsec_mb/pmd_aesni_mb.c +5 -24
@@ -1710,8 +1710,6 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) {struct rte_crypto_op *op = (struct rte_crypto_op *)job->user_data;struct aesni_mb_session *sess =NULL ; -uint32_t driver_id =ipsec_mb_get_driver_id ( -IPSEC_MB_PMD_TYPE_AESNI_MB );#ifdef AESNI_MB_DOCSIS_SEC_ENABLED uint8_t is_docsis_sec =0 ;
@@ -1725,15 +1723,7 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) sess =get_sec_session_private_data (op->sym->sec_session); } else#endif - { - sess =get_sym_session_private_data (op->sym->session, - driver_id); - } - -if (unlikely (sess ==NULL )) { - op->status =RTE_CRYPTO_OP_STATUS_INVALID_SESSION ; -return op; - } + sess = (void *)op->sym->session->driver_priv_data;if (likely (op->status ==RTE_CRYPTO_OP_STATUS_NOT_PROCESSED )) {switch (job->status) {
@@ -1771,10 +1761,6 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job)/* Free session if a session-less crypto op */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sess,0 ,sizeof (struct aesni_mb_session )); -memset (op->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - op->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sess);rte_mempool_put (qp->sess_mp, op->sym->session); op->sym->session =NULL ; }
@@ -1962,7 +1948,7 @@ aesni_mb_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,return processed_jobs; } - +#if 0 static inline void ipsec_mb_fill_error_code (struct rte_crypto_sym_vec *vec,int32_t err) {
@@ -1971,6 +1957,7 @@ ipsec_mb_fill_error_code(struct rte_crypto_sym_vec *vec, int32_t err)for (i =0 ; i != vec->num; ++i) vec->status[i] = err; } +#endif static inline int check_crypto_sgl (union rte_crypto_sym_ofs so,const struct rte_crypto_sgl *sgl)
@@ -2028,7 +2015,7 @@ verify_sync_dgst(struct rte_crypto_sym_vec *vec, }static uint32_t -aesni_mb_process_bulk (struct rte_cryptodev *dev, +aesni_mb_process_bulk(struct rte_cryptodev *dev __rte_unused,struct rte_cryptodev_sym_session *sess,union rte_crypto_sym_ofs sofs,struct rte_crypto_sym_vec *vec) {
@@ -2037,15 +2024,9 @@ aesni_mb_process_bulk(struct rte_cryptodev *dev,void *buf;IMB_JOB *job;IMB_MGR *mb_mgr; -struct aesni_mb_session *s; +struct aesni_mb_session *s = (void *)sess->driver_priv_data;uint8_t tmp_dgst[vec->num][DIGEST_LENGTH_MAX ]; - s =get_sym_session_private_data (sess, dev->driver_id); -if (s ==NULL ) { -ipsec_mb_fill_error_code (vec,EINVAL ); -return 0 ; - } -/* get per-thread MB MGR, create one if needed */ mb_mgr =get_per_thread_mb_mgr ();if (unlikely (mb_mgr ==NULL ))
drivers/crypto/ipsec_mb/pmd_chacha_poly.c +0 -4
@@ -290,10 +290,6 @@ handle_completed_chacha20_poly1305_crypto_op(struct ipsec_mb_qp *qp,/* Free session if a session-less crypto op */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sess,0 ,sizeof (struct chacha20_poly1305_session )); -memset (op->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - op->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sess);rte_mempool_put (qp->sess_mp, op->sym->session); op->sym->session =NULL ; }
drivers/crypto/ipsec_mb/pmd_kasumi.c +3 -7
@@ -231,11 +231,6 @@ process_ops(struct rte_crypto_op **ops, struct kasumi_session *session,/* Free session if a session-less crypto op. */ if (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (session,0 ,sizeof (struct kasumi_session )); -memset ( - ops[i]->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - ops[i]->sym->session)); -rte_mempool_put (qp->sess_mp_priv, session);rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ; }
@@ -287,8 +282,9 @@ process_op_bit(struct rte_crypto_op *op, struct kasumi_session *session,/* Free session if a session-less crypto op. */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -memset (op->sym->session,0 ,sizeof (struct kasumi_session )); -rte_cryptodev_sym_session_free (op->sym->session); +memset (op->sym->session->driver_priv_data,0 , +sizeof (struct kasumi_session )); +rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ; }return processed_op;
drivers/crypto/ipsec_mb/pmd_snow3g.c +3 -6
@@ -362,10 +362,6 @@ process_ops(struct rte_crypto_op **ops, struct snow3g_session *session,/* Free session if a session-less crypto op. */ if (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (session,0 ,sizeof (struct snow3g_session )); -memset (ops[i]->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - ops[i]->sym->session)); -rte_mempool_put (qp->sess_mp_priv, session);rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ; }
@@ -417,8 +413,9 @@ process_op_bit(struct rte_crypto_op *op, struct snow3g_session *session,/* Free session if a session-less crypto op. */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -memset (op->sym->session,0 ,sizeof (struct snow3g_session )); -rte_cryptodev_sym_session_free (op->sym->session); +memset (op->sym->session->driver_priv_data,0 , +sizeof (struct snow3g_session )); +rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ; }
drivers/crypto/ipsec_mb/pmd_zuc.c +0 -4
@@ -239,10 +239,6 @@ process_ops(struct rte_crypto_op **ops, enum ipsec_mb_operation op_type,/* Free session if a session-less crypto op. */ if (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sessions[i],0 ,sizeof (struct zuc_session )); -memset (ops[i]->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - ops[i]->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sessions[i]);rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ; }
drivers/crypto/mlx5/mlx5_crypto.c +6 -20
@@ -171,14 +171,13 @@ mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int mlx5_crypto_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *session, -struct rte_mempool *mp) +struct rte_cryptodev_sym_session *session) {struct mlx5_crypto_priv *priv = dev->data->dev_private; -struct mlx5_crypto_session *sess_private_data; +struct mlx5_crypto_session *sess_private_data = + (void *)session->driver_priv_data;struct rte_crypto_cipher_xform *cipher;uint8_t encryption_order; -int ret;if (unlikely (xform->next !=NULL )) {DRV_LOG (ERR ,"Xform next is not supported." );
@@ -189,17 +188,9 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev,DRV_LOG (ERR ,"Only AES-XTS algorithm is supported." );return -ENOTSUP ; } - ret =rte_mempool_get (mp, (void *)&sess_private_data); -if (ret !=0 ) { -DRV_LOG (ERR , -"Failed to get session %p private data from mempool." , - sess_private_data); -return -ENOMEM ; - } cipher = &xform->cipher; sess_private_data->dek =mlx5_crypto_dek_prepare (priv, cipher);if (sess_private_data->dek ==NULL ) { -rte_mempool_put (mp, sess_private_data);DRV_LOG (ERR ,"Failed to prepare dek." );return -ENOMEM ; }
@@ -239,8 +230,6 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev, sess_private_data->dek_id =rte_cpu_to_be_32 (sess_private_data->dek->obj->id &0xffffff ); -set_sym_session_private_data (session, dev->driver_id, - sess_private_data);DRV_LOG (DEBUG ,"Session %p was configured." , sess_private_data);return 0 ; }
@@ -250,16 +239,13 @@ mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) {struct mlx5_crypto_priv *priv = dev->data->dev_private; -struct mlx5_crypto_session *spriv =get_sym_session_private_data (sess, - dev->driver_id); +struct mlx5_crypto_session *spriv = (void *)sess->driver_priv_data;if (unlikely (spriv ==NULL )) {DRV_LOG (ERR ,"Failed to get session %p private data." , spriv);return ; }mlx5_crypto_dek_destroy (priv, spriv->dek); -set_sym_session_private_data (sess, dev->driver_id,NULL ); -rte_mempool_put (rte_mempool_from_obj (spriv), spriv);DRV_LOG (DEBUG ,"Session %p was cleared." , spriv); }
@@ -369,8 +355,8 @@ mlx5_crypto_wqe_set(struct mlx5_crypto_priv *priv,struct rte_crypto_op *op, struct mlx5_umr_wqe *umr) { - struct mlx5_crypto_session *sess = get_sym_session_private_data - (op->sym->session, mlx5_crypto_driver_id); + struct mlx5_crypto_session *sess = + (void *)op->sym->session->driver_priv_data;struct mlx5_wqe_cseg *cseg = &umr->ctr;struct mlx5_wqe_mkey_cseg *mkc = &umr->mkc;struct mlx5_wqe_dseg *klms = &umr->kseg[0 ];
drivers/crypto/mvsam/rte_mrvl_pmd.c +1 -7
@@ -597,13 +597,7 @@ mrvl_request_prepare_crp(struct sam_cio_op_params *request,return -EINVAL ; } - sess = (struct mrvl_crypto_session *)get_sym_session_private_data ( - op->sym->session, - cryptodev_driver_id); -if (unlikely (sess ==NULL )) { -MRVL_LOG (ERR ,"Session was not created for this device!" ); -return -EINVAL ; - } + sess = (struct mrvl_crypto_session *)op->sym->session->data; request->sa = sess->sam_sess; request->cookie = op;
drivers/crypto/mvsam/rte_mrvl_pmd_ops.c +3 -18
@@ -736,8 +736,7 @@ mrvl_crypto_pmd_sym_session_get_size(__rte_unused struct rte_cryptodev *dev)static int mrvl_crypto_pmd_sym_session_configure(__rte_unused struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mp) +struct rte_cryptodev_sym_session *sess) {struct mrvl_crypto_session *mrvl_sess;void *sess_private_data;
@@ -748,23 +747,15 @@ mrvl_crypto_pmd_sym_session_configure(__rte_unused struct rte_cryptodev *dev,return -EINVAL ; } -if (rte_mempool_get (mp, &sess_private_data)) { -CDEV_LOG_ERR ("Couldn't get object from session mempool." ); -return -ENOMEM ; - } - + sess_private_data = sess->data;memset (sess_private_data,0 ,sizeof (struct mrvl_crypto_session )); ret =mrvl_crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {MRVL_LOG (ERR ,"Failed to configure session parameters!" ); - -/* Return session to mempool */ -rte_mempool_put (mp, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, sess_private_data); mrvl_sess = (struct mrvl_crypto_session *)sess_private_data; if (sam_session_create(&mrvl_sess->sam_sess_params,
@@ -791,8 +782,7 @@ mrvl_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) { -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = sess->data;/* Zero out the whole structure */ if (sess_priv) {
@@ -803,11 +793,6 @@ mrvl_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev, sam_session_destroy(mrvl_sess->sam_sess) < 0) {MRVL_LOG (ERR ,"Error while destroying session!" ); } - -memset (mrvl_sess,0 ,sizeof (struct mrvl_crypto_session )); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); } }
drivers/crypto/nitrox/nitrox_sym.c +8 -31
@@ -530,24 +530,16 @@ configure_aead_ctx(struct rte_crypto_aead_xform *xform, }static int -nitrox_sym_dev_sess_configure (struct rte_cryptodev *cdev, +nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *mp_obj; -struct nitrox_crypto_ctx *ctx; +struct nitrox_crypto_ctx *ctx = (void *)sess->driver_priv_data;struct rte_crypto_cipher_xform *cipher_xform =NULL ;struct rte_crypto_auth_xform *auth_xform =NULL ;struct rte_crypto_aead_xform *aead_xform =NULL ;int ret = -EINVAL ; -if (rte_mempool_get (mempool, &mp_obj)) { -NITROX_LOG (ERR ,"Couldn't allocate context\n" ); -return -ENOMEM ; - } - - ctx = mp_obj; ctx->nitrox_chain =get_crypto_chain_order (xform);switch (ctx->nitrox_chain) {case NITROX_CHAIN_CIPHER_ONLY :
@@ -585,38 +577,23 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev, goto err; } - ctx->iova =rte_mempool_virt2iova (ctx); -set_sym_session_private_data (sess, cdev->driver_id, ctx); + ctx->iova = sess->driver_priv_data_iova;return 0 ; err: -rte_mempool_put (mempool, mp_obj);return ret; }static void -nitrox_sym_dev_sess_clear (struct rte_cryptodev *cdev, -struct rte_cryptodev_sym_session *sess) -{ -struct nitrox_crypto_ctx *ctx =get_sym_session_private_data (sess, - cdev->driver_id); -struct rte_mempool *sess_mp; - -if (!ctx) -return ; - -memset (ctx,0 ,sizeof (*ctx)); - sess_mp =rte_mempool_from_obj (ctx); -set_sym_session_private_data (sess, cdev->driver_id,NULL ); -rte_mempool_put (sess_mp, ctx); -} +nitrox_sym_dev_sess_clear(struct rte_cryptodev *cdev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}static struct nitrox_crypto_ctx *get_crypto_ctx (struct rte_crypto_op *op) {if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {if (likely (op->sym->session)) -return get_sym_session_private_data (op->sym->session, - nitrox_sym_drv_id); +return (void *)op->sym->session->driver_priv_data; }return NULL ;
drivers/crypto/null/null_crypto_pmd.c +5 -14
@@ -58,7 +58,7 @@ process_op(const struct null_crypto_qp *qp, struct rte_crypto_op *op,if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (op->sym->session,0 ,sizeof (struct null_crypto_session )); -rte_cryptodev_sym_session_free (op->sym->session); +rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ; }
@@ -78,30 +78,21 @@ get_session(struct null_crypto_qp *qp, struct rte_crypto_op *op)if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {if (likely (sym_op->session !=NULL )) sess = (struct null_crypto_session *) -get_sym_session_private_data ( - sym_op->session, cryptodev_driver_id); + sym_op->session->driver_priv_data; }else { -void *_sess =NULL ; -void *_sess_private_data =NULL ; +struct rte_cryptodev_sym_session *_sess =NULL ;if (rte_mempool_get (qp->sess_mp, (void **)&_sess))return NULL ; -if (rte_mempool_get (qp->sess_mp_priv, - (void **)&_sess_private_data)) -return NULL ; - - sess = (struct null_crypto_session *)_sess_private_data; + sess = (struct null_crypto_session *)_sess->driver_priv_data;if (unlikely (null_crypto_set_session_parameters (sess, sym_op->xform) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); -rte_mempool_put (qp->sess_mp_priv, _sess_private_data); sess =NULL ; } - sym_op->session = (struct rte_cryptodev_sym_session *)_sess; -set_sym_session_private_data (op->sym->session, - cryptodev_driver_id, _sess_private_data); + sym_op->session = _sess; }return sess;
drivers/crypto/null/null_crypto_pmd_ops.c +5 -28
@@ -233,7 +233,6 @@ null_crypto_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, } qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private;memset (&qp->qp_stats,0 ,sizeof (qp->qp_stats));
@@ -256,8 +255,7 @@ null_crypto_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int null_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mp) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;
@@ -267,43 +265,22 @@ null_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused,return -EINVAL ; } -if (rte_mempool_get (mp, &sess_private_data)) { -NULL_LOG (ERR , -"Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = (void *)sess->driver_priv_data; ret =null_crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {NULL_LOG (ERR ,"failed configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mp, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/** Clear the memory of session so it doesn't leave key material behind */ static void -null_crypto_pmd_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -/* Zero out the whole structure */ -if (sess_priv) { -memset (sess_priv,0 ,sizeof (struct null_crypto_session )); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +null_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}static struct rte_cryptodev_ops pmd_ops = { .dev_configure = null_crypto_pmd_config,
drivers/crypto/null/null_crypto_pmd_private.h +0 -2
@@ -31,8 +31,6 @@ struct null_crypto_qp {/**< Ring for placing process packets */ struct rte_mempool *sess_mp;/**< Session Mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session Mempool */ struct rte_cryptodev_stats qp_stats;/**< Queue pair statistics */ } __rte_cache_aligned;
drivers/crypto/octeontx/otx_cryptodev_hw_access.h +0 -1
@@ -49,7 +49,6 @@ struct cpt_instance {uint32_t queue_id;uintptr_t rsvd;struct rte_mempool *sess_mp; -struct rte_mempool *sess_mp_priv;struct cpt_qp_meta_info meta_info;uint8_t ca_enabled; };
drivers/crypto/octeontx/otx_cryptodev_ops.c +17 -50
166 lines not on this page · raw message
drivers/crypto/openssl/openssl_pmd_private.h +0 -2
@@ -70,8 +70,6 @@ struct openssl_qp {/**< Ring for placing process packets */ struct rte_mempool *sess_mp;/**< Session Mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session Private Data Mempool */ struct rte_cryptodev_stats stats;/**< Queue pair statistics */ uint8_t temp_digest[DIGEST_LENGTH_MAX ];
drivers/crypto/openssl/rte_openssl_pmd.c +6 -18
@@ -887,10 +887,8 @@ get_session(struct openssl_qp *qp, struct rte_crypto_op *op)if (op->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) {/* get existing session */ if (likely (op->sym->session !=NULL )) - sess = (struct openssl_session *) -get_sym_session_private_data ( - op->sym->session, - cryptodev_driver_id); + sess = (void *) + op->sym->session->driver_priv_data; }else {if (likely (op->asym->session !=NULL )) asym_sess = (struct openssl_asym_session *)
@@ -901,32 +899,26 @@ get_session(struct openssl_qp *qp, struct rte_crypto_op *op)return asym_sess;
Lines of this file are not shown. View the raw message.
drivers/crypto/openssl/rte_openssl_pmd_ops.c +5 -24
Lines of this file are not shown. View the raw message.
drivers/crypto/scheduler/scheduler_pmd_ops.c +6 -32
Lines of this file are not shown. View the raw message.
drivers/crypto/virtio/virtio_cryptodev.c +7 -33
Lines of this file are not shown. View the raw message.
examples/fips_validation/fips_dev_self_test.c +5 -25
Lines of this file are not shown. View the raw message.
examples/l2fwd-crypto/main.c +6 -48
54 lines not on this page · raw message
lib/cryptodev/rte_cryptodev.c +97 -179
276 lines not on this page · raw message
lib/cryptodev/rte_cryptodev.h +33 -90
123 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.25.1
Part of the diff is not shown here. View the raw message.
RE: [PATCH v2 1/6] cryptodev: rework session framework
Quoted 47 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Wednesday, September 21, 2022 4:03 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; De Lara Guarch, Pablo <[email protected]>; [email protected]; [email protected]; Yang, Qiming <[email protected]>; Wu, Wenjun1 <[email protected]>; [email protected]; Wu, Jingjing <[email protected]>; Xing, Beilei <[email protected]>; [email protected]; Akhil Goyal <[email protected]>; Zhang, Roy Fan <[email protected]>
Subject: [PATCH v2 1/6] cryptodev: rework session frameworkAs per current design,
rte_cryptodev_sym_session_create()andrte_cryptodev_sym_session_init()use separate mempool objects for a single session.
And structurerte_cryptodev_sym_sessionis not directly used by the application, it may cause ABI breakage if the structure is modified in future.To address these two issues, the
rte_cryptodev_sym_session_createwill take one mempool object that the session and session private data are virtually/physically contiguous, and initializes both fields. The APIrte_cryptodev_sym_session_initis removed.
rte_cryptodev_sym_session_createwill now return an opaque session pointer which will be used by the app and other APIs.In data path, opaque session pointer is attached to
rte_crypto_opand the PMD can call an internal library API to get the session private data pointer based on the driver id.Note: currently single session may be used by different device drivers, given it is initialized by them. After the change the session created by one device driver cannot be used or reinitialized by another driver.
Signed-off-by Akhil Goyal <[email protected]>Signed-off-by Fan Zhang <[email protected]>---
PATCH v2 2/6 crypto/scheduler: use unified session
In reply to Akhil GoyalFrom: Fan Zhang <[email protected]>
This patch updates the scheduler PMD to use unified session data structure. Previously thanks to the private session array in cryptodev sym session there are no necessary change needed for scheduler PMD other than the way ops are enqueued/dequeued. The patch inherits the same design in the original session data structure to the scheduler PMD so the cryptodev sym session can be as a linear buffer for both session header and driver private data.
With the change there are inevitable extra cost on both memory (64 bytes per session per driver type) and cycle count (set the correct session for each cop based on the worker before enqueue, and retrieve the original session after dequeue).
--- drivers/crypto/scheduler/scheduler_failover.c | 19 ++- .../crypto/scheduler/scheduler_multicore.c | 17 +++ .../scheduler/scheduler_pkt_size_distr.c | 84 +++++++++--- drivers/crypto/scheduler/scheduler_pmd_ops.c | 107 +++++++++++++++- .../crypto/scheduler/scheduler_pmd_private.h | 120 +++++++++++++++++- .../crypto/scheduler/scheduler_roundrobin.c | 11 +- 6 files changed, 318 insertions(+), 40 deletions(-)
drivers/crypto/scheduler/scheduler_failover.c +12 -7
@@ -16,18 +16,19 @@struct fo_scheduler_qp_ctx {struct scheduler_worker primary_worker;struct scheduler_worker secondary_worker; +uint8_t primary_worker_index; +uint8_t secondary_worker_index;uint8_t deq_idx; }; static __rte_always_inline uint16_tfailover_worker_enqueue (struct scheduler_worker *worker, -struct rte_crypto_op **ops,uint16_t nb_ops) +struct rte_crypto_op **ops,uint16_t nb_ops,uint8_t index) { -uint16_t i, processed_ops; +uint16_t processed_ops; -for (i =0 ; i < nb_ops && i <4 ; i++) -rte_prefetch0 (ops[i]->sym->session); +scheduler_set_worker_session (ops, nb_ops, index); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, ops, nb_ops);
@@ -47,13 +48,14 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops)return 0 ; enqueued_ops =failover_worker_enqueue (&qp_ctx->primary_worker, - ops, nb_ops); + ops, nb_ops,PRIMARY_WORKER_IDX );if (enqueued_ops < nb_ops) enqueued_ops +=failover_worker_enqueue ( &qp_ctx->secondary_worker, &ops[enqueued_ops], - nb_ops - enqueued_ops); + nb_ops - enqueued_ops, +SECONDARY_WORKER_IDX );return enqueued_ops; }
@@ -94,7 +96,7 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) qp_ctx->deq_idx = (~qp_ctx->deq_idx) &WORKER_SWITCH_MASK ;if (nb_deq_ops == nb_ops) -return nb_deq_ops; + goto retrieve_session; worker = workers[qp_ctx->deq_idx];
@@ -104,6 +106,9 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) worker->nb_inflight_cops -= nb_deq_ops2; } +retrieve_session: +scheduler_retrieve_session (ops, nb_deq_ops + nb_deq_ops2); +return nb_deq_ops + nb_deq_ops2; }
drivers/crypto/scheduler/scheduler_multicore.c +17 -0
@@ -183,11 +183,19 @@ mc_scheduler_worker(struct rte_cryptodev *dev)while (!mc_ctx->stop_signal) {if (pending_enq_ops) { +scheduler_set_worker_session ( + &enq_ops[pending_enq_ops_idx], pending_enq_ops, + worker_idx); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, &enq_ops[pending_enq_ops_idx], pending_enq_ops); +if (processed_ops < pending_deq_ops) +scheduler_retrieve_session ( + &enq_ops[pending_enq_ops_idx + + processed_ops], + pending_deq_ops - processed_ops); pending_enq_ops -= processed_ops; pending_enq_ops_idx += processed_ops; inflight_ops += processed_ops;
@@ -195,9 +203,16 @@ mc_scheduler_worker(struct rte_cryptodev *dev) processed_ops =rte_ring_dequeue_burst (enq_ring, (void *)enq_ops,MC_SCHED_BUFFER_SIZE ,NULL );if (processed_ops) { +scheduler_set_worker_session (enq_ops, + processed_ops, worker_idx); pending_enq_ops_idx =rte_cryptodev_enqueue_burst ( worker->dev_id, worker->qp_id, enq_ops, processed_ops); +if (pending_enq_ops_idx < processed_ops) +scheduler_retrieve_session ( + enq_ops + pending_enq_ops_idx, + processed_ops - + pending_enq_ops_idx); pending_enq_ops = processed_ops - pending_enq_ops_idx; inflight_ops += pending_enq_ops_idx; }
@@ -214,6 +229,8 @@ mc_scheduler_worker(struct rte_cryptodev *dev) worker->dev_id, worker->qp_id, deq_ops, MC_SCHED_BUFFER_SIZE);if (processed_ops) { +scheduler_retrieve_session (deq_ops, + processed_ops); inflight_ops -= processed_ops;if (reordering_enabled) {uint16_t j;
drivers/crypto/scheduler/scheduler_pkt_size_distr.c +63 -21
149 lines not on this page · raw message
drivers/crypto/scheduler/scheduler_pmd_ops.c +101 -6
126 lines not on this page · raw message
drivers/crypto/scheduler/scheduler_pmd_private.h +119 -1
144 lines not on this page · raw message
drivers/crypto/scheduler/scheduler_roundrobin.c +6 -5
@@ -23,16 +23,17 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) ((struct scheduler_qp_ctx *)qp)->private_qp_ctx;uint32_t worker_idx = rr_qp_ctx->last_enq_worker_idx;struct scheduler_worker *worker = &rr_qp_ctx->workers[worker_idx]; -uint16_t i, processed_ops; +uint16_t processed_ops;if (unlikely (nb_ops ==0 ))return 0 ; -for (i =0 ; i < nb_ops && i <4 ; i++) -rte_prefetch0 (ops[i]->sym->session); - +scheduler_set_worker_session (ops, nb_ops, worker_idx); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, ops, nb_ops); +if (processed_ops < nb_ops) +scheduler_retrieve_session (ops + processed_ops, + nb_ops - processed_ops); worker->nb_inflight_cops += processed_ops;
@@ -86,7 +87,7 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) nb_deq_ops =rte_cryptodev_dequeue_burst (worker->dev_id, worker->qp_id, ops, nb_ops); - +scheduler_retrieve_session (ops, nb_deq_ops); last_worker_idx +=1 ; last_worker_idx %= rr_qp_ctx->nb_workers;
Signature · 1 line
2.25.1
RE: [PATCH v2 2/6] crypto/scheduler: use unified session
Quoted 37 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Wednesday, September 21, 2022 4:03 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; De Lara Guarch, Pablo <[email protected]>; [email protected]; [email protected]; Yang, Qiming <[email protected]>; Wu, Wenjun1 <[email protected]>; [email protected]; Wu, Jingjing <[email protected]>; Xing, Beilei <[email protected]>; [email protected]; Zhang, Roy Fan <[email protected]>; Akhil Goyal <[email protected]>
Subject: [PATCH v2 2/6]crypto/scheduler: use unified sessionFrom: Fan Zhang <[email protected]>
This patch updates the scheduler PMD to use unified session data structure. Previously thanks to the private session array in cryptodev sym session there are no necessary change needed for scheduler PMD other than the way ops are enqueued/dequeued. The patch inherits the same design in the original session data structure to the scheduler PMD so the cryptodev sym session can be as a linear buffer for both session header and driver private data.
With the change there are inevitable extra cost on both memory
(64 bytes per session per driver type) and cycle count (set the correct session for each cop based on the worker before enqueue, and retrieve the original session after dequeue).Signed-off-by Fan Zhang <[email protected]>Signed-off-by Akhil Goyal <[email protected]>---
PATCH v2 3/6 cryptodev: hide sym session structure
In reply to Akhil GoyalStructure rte_cryptodev_sym_session is moved to internal headers which are not visible to applications.
The only field which should be used by app is opaque_data. This field can now be accessed via set/get APIs added in this patch.
Subsequent changes in app and lib are made to compile the code.
--- app/test/test_ipsec_perf.c | 4 +- doc/guides/prog_guide/cryptodev_lib.rst | 16 ++---- doc/guides/rel_notes/deprecation.rst | 9 +++ doc/guides/rel_notes/release_22_11.rst | 7 +++ drivers/crypto/bcmfs/bcmfs_sym_session.c | 5 +- drivers/crypto/caam_jr/caam_jr.c | 10 ++-- drivers/crypto/ccp/ccp_crypto.c | 30 +++++----- drivers/crypto/ccp/ccp_pmd_ops.c | 2 +- drivers/crypto/ccp/rte_ccp_pmd.c | 4 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 8 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 10 ++-- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 4 +- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 2 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 11 ++-- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 2 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 9 ++- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 2 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 2 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 4 +- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 6 +- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 4 +- drivers/crypto/ipsec_mb/pmd_kasumi.c | 2 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 2 +- drivers/crypto/mlx5/mlx5_crypto.c | 7 +-- drivers/crypto/nitrox/nitrox_sym.c | 6 +- drivers/crypto/null/null_crypto_pmd.c | 3 +- drivers/crypto/null/null_crypto_pmd_ops.c | 2 +- drivers/crypto/octeontx/otx_cryptodev_ops.c | 10 ++-- drivers/crypto/openssl/rte_openssl_pmd.c | 4 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 4 +- drivers/crypto/qat/qat_sym.c | 4 +- drivers/crypto/qat/qat_sym.h | 3 +- drivers/crypto/qat/qat_sym_session.c | 6 +- .../scheduler/scheduler_pkt_size_distr.c | 13 ++--- drivers/crypto/scheduler/scheduler_pmd_ops.c | 4 +- .../crypto/scheduler/scheduler_pmd_private.h | 30 +++++----- drivers/crypto/virtio/virtio_cryptodev.c | 4 +- drivers/crypto/virtio/virtio_rxtx.c | 2 +- examples/fips_validation/fips_dev_self_test.c | 10 ++-- lib/cryptodev/cryptodev_pmd.h | 32 +++++++++++ lib/cryptodev/rte_crypto.h | 3 +- lib/cryptodev/rte_crypto_sym.h | 7 +-- lib/cryptodev/rte_cryptodev.c | 16 ++++-- lib/cryptodev/rte_cryptodev.h | 56 +++++++++---------- lib/cryptodev/rte_cryptodev_trace.h | 8 +-- lib/ipsec/rte_ipsec_group.h | 5 +- lib/ipsec/ses.c | 3 +- 47 files changed, 208 insertions(+), 189 deletions(-)
app/test/test_ipsec_perf.c +2 -2
@@ -227,7 +227,7 @@ static int create_sa(enum rte_security_session_action_type action_type, struct ipsec_sa *sa) { - static struct rte_cryptodev_sym_session dummy_ses; + void *dummy_ses = NULL;size_t sz;int rc;
@@ -247,7 +247,7 @@ create_sa(enum rte_security_session_action_type action_type, "failed to allocate memory for rte_ipsec_sa\n"); sa->ss[0 ].type = action_type; - sa->ss[0 ].crypto.ses = &dummy_ses; + sa->ss[0 ].crypto.ses = dummy_ses; rc =rte_ipsec_sa_init (sa->ss[0 ].sa, &sa->sa_prm, sz); rc = (rc >0 && (uint32_t )rc <= sz) ?0 : -EINVAL ;
doc/guides/prog_guide/cryptodev_lib.rst +5 -11
@@ -125,13 +125,11 @@ Each queue pairs resources may be allocated on a specified socket. uint32_t nb_descriptors; /**< Number of descriptors per queue pair */ struct rte_mempool *mp_session; /**< The mempool for creating session in sessionless mode */ - struct rte_mempool *mp_session_private; - /**< The mempool for creating sess private data in sessionless mode */ }; -The fields ``mp_session`` and ``mp_session_private`` are used for creating -temporary session to process the crypto operations in the session-less mode. +The field ``mp_session`` is used for creating temporary session to process +the crypto operations in the session-less mode. They can be the same other different mempools. Please note not all Cryptodev PMDs supports session-less mode.
@@ -595,7 +593,7 @@ chain. struct rte_mbuf *m_dst; union { - struct rte_cryptodev_sym_session *session; + void *session; /**< Handle for the initialised session context */ struct rte_crypto_sym_xform *xform; /**< Session-less API Crypto operation parameters */
@@ -943,15 +941,11 @@ using one of the crypto PMDs available in DPDK. /* Create crypto session and initialize it for the crypto device. */ struct rte_cryptodev_sym_session *session; - session = rte_cryptodev_sym_session_create(session_pool); + session = rte_cryptodev_sym_session_create(cdev_id, &cipher_xform, + session_pool); if (session == NULL) rte_exit(EXIT_FAILURE, "Session could not be created\n"); - if (rte_cryptodev_sym_session_init(cdev_id, session, - &cipher_xform, session_priv_pool) < 0) - rte_exit(EXIT_FAILURE, "Session could not be initialized " - "for the crypto device\n"); - /* Get a burst of crypto operations. */ struct rte_crypto_op *crypto_ops[BURST_SIZE]; if (rte_crypto_op_bulk_alloc(crypto_op_pool,
doc/guides/rel_notes/deprecation.rst +9 -0
@@ -237,3 +237,12 @@ Deprecation Notices applications should be updated to use the ``dmadev`` library instead, with the underlying HW-functionality being provided by the ``ioat`` or ``idxd`` dma drivers + +* cryptodev: Hide structure ``rte_cryptodev_sym_session`` to remove unnecessary + indirection between session and the private data of session. An opaque pointer + can be exposed directly to application which can be attached to the + ``rte_crypto_op``. + +* cryptodev: The functions ``rte_cryptodev_sym_session_init`` and + ``rte_cryptodev_sym_session_clear`` are deprecated. The sym crypto session + opaque pointer can no longer be used by different crypto device drivers.
doc/guides/rel_notes/release_22_11.rst +7 -0
@@ -89,6 +89,13 @@ API Changes Also, make sure to start the actual text at the margin. ======================================================= +* cryptodev: The structure ``rte_cryptodev_sym_session`` was moved to + cryptodev_pmd.h and was hidden from the application. The APIs to create/init and + destroy sym crypto session were updated to take a single mempool with element size + enough to hold session data and session private data. Inline APIs was created to + get and set the session data. All sample applications were updated to attach an + opaque pointer for the session to the ``rte_crypto_op`` while enqueuing. + ABI Changes -----------
drivers/crypto/bcmfs/bcmfs_sym_session.c +2 -3
@@ -210,8 +210,7 @@ bcmfs_sym_get_session(struct rte_crypto_op *op)BCMFS_DP_LOG (ERR ,"operations op(%p) is sessionless" , op); }else if (likely (op->sym->session !=NULL )) {/* get existing session */ - sess = (struct bcmfs_sym_session *) - op->sym->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); }if (sess ==NULL )
@@ -233,7 +232,7 @@ bcmfs_sym_session_configure(struct rte_cryptodev *dev __rte_unused,return -EINVAL ; } - sess_private_data = (void *)sess->driver_priv_data; + sess_private_data =CRYPTODEV_GET_SYM_SESS_PRIV (sess); ret =crypto_set_session_parameters (sess_private_data, xform);
drivers/crypto/caam_jr/caam_jr.c +4 -6
@@ -1356,8 +1356,7 @@ caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp)switch (op->sess_type) {case RTE_CRYPTO_OP_WITH_SESSION : - ses = (struct caam_jr_session *) - op->sym->session->driver_priv_data; + ses =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);break ;case RTE_CRYPTO_OP_SECURITY_SESSION : ses = (struct caam_jr_session *)
@@ -1699,7 +1698,7 @@ caam_jr_sym_session_configure(struct rte_cryptodev *dev __rte_unused,int ret;PMD_INIT_FUNC_TRACE (); - sess_private_data = (void *)sess->driver_priv_data; + sess_private_data =CRYPTODEV_GET_SYM_SESS_PRIV (sess);memset (sess_private_data,0 ,sizeof (struct caam_jr_session )); ret =caam_jr_set_session_parameters (dev, xform, sess_private_data);if (ret !=0 ) {
@@ -1716,12 +1715,11 @@ static void caam_jr_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) { -void *sess_priv = (void *)sess->driver_priv_data; -struct caam_jr_session *s = (struct caam_jr_session *)sess_priv; +struct caam_jr_session *s =CRYPTODEV_GET_SYM_SESS_PRIV (sess);PMD_INIT_FUNC_TRACE (); -if (sess_priv) { +if (s) {rte_free (s->cipher_key.data);rte_free (s->auth_key.data); }
drivers/crypto/ccp/ccp_crypto.c +14 -16
@@ -1585,7 +1585,7 @@ ccp_perform_hmac(struct rte_crypto_op *op,void *append_ptr;uint8_t *addr; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); addr = session->auth.pre_compute; src_addr = rte_pktmbuf_iova_offset(op->sym->m_src,
@@ -1764,7 +1764,7 @@ ccp_perform_sha(struct rte_crypto_op *op,void *append_ptr;uint64_t auth_msg_bits; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); src_addr =rte_pktmbuf_iova_offset (op->sym->m_src, op->sym->auth.data.offset);
@@ -1855,7 +1855,7 @@ ccp_perform_sha3_hmac(struct rte_crypto_op *op,uint32_t tail;phys_addr_t src_addr, dest_addr, ctx_paddr, dest_addr_t; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); src_addr =rte_pktmbuf_iova_offset (op->sym->m_src, op->sym->auth.data.offset);
@@ -1999,7 +1999,7 @@ ccp_perform_sha3(struct rte_crypto_op *op,uint32_t tail;phys_addr_t src_addr, dest_addr, ctx_paddr; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); src_addr =rte_pktmbuf_iova_offset (op->sym->m_src, op->sym->auth.data.offset);
@@ -2071,7 +2071,7 @@ ccp_perform_aes_cmac(struct rte_crypto_op *op,phys_addr_t src_addr, dest_addr, key_addr;int length, non_align_len; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); key_addr =rte_mem_virt2phy (session->auth.key_ccp); src_addr = rte_pktmbuf_iova_offset(op->sym->m_src,
@@ -2232,7 +2232,7 @@ ccp_perform_aes(struct rte_crypto_op *op,phys_addr_t src_addr, dest_addr, key_addr;uint8_t *iv; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); function.raw =0 ; iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->iv.offset);
@@ -2318,7 +2318,7 @@ ccp_perform_3des(struct rte_crypto_op *op,uint8_t *iv;phys_addr_t src_addr, dest_addr, key_addr; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->iv.offset);switch (session->cipher.um.des_mode) {
@@ -2426,7 +2426,7 @@ ccp_perform_aes_gcm(struct rte_crypto_op *op, struct ccp_queue *cmd_q)phys_addr_t digest_dest_addr;int length, non_align_len; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->iv.offset); key_addr = session->cipher.key_phys;
@@ -2591,7 +2591,7 @@ ccp_crypto_cipher(struct rte_crypto_op *op,int result =0 ;struct ccp_session *session; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);switch (session->cipher.algo) {case CCP_CIPHER_ALGO_AES_CBC :
@@ -2627,7 +2627,7 @@ ccp_crypto_auth(struct rte_crypto_op *op,int result =0 ;struct ccp_session *session; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);switch (session->auth.algo) {case CCP_AUTH_ALGO_SHA1 :
@@ -2695,7 +2695,7 @@ ccp_crypto_aead(struct rte_crypto_op *op,int result =0 ;struct ccp_session *session; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);switch (session->auth.algo) {case CCP_AUTH_ALGO_AES_GCM :
@@ -2758,8 +2758,7 @@ process_ops_to_enqueue(struct ccp_qp *qp, b_info->head_offset = (uint32_t )(cmd_q->qbase_phys_addr + cmd_q->qidx *Q_DESC_SIZE );for (i = b_idx; i < (nb_ops+b_idx); i++) { - session = (struct ccp_session *) - op[i]->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op[i]->sym->session);switch (session->cmd_id) {case CCP_CMD_CIPHER : result =ccp_crypto_cipher (op[i], cmd_q, b_info);
@@ -2835,7 +2834,7 @@ static inline void ccp_auth_dq_prepare(struct rte_crypto_op *op)int offset, digest_offset;uint8_t digest_le[64 ]; - session = (struct ccp_session *)op->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);if (session->cmd_id ==CCP_CMD_COMBINED ) { digest_data = op->sym->aead.digest.data;
@@ -2909,8 +2908,7 @@ ccp_prepare_ops(struct ccp_qp *qp,for (i = b_info->b_idx; i < min_ops; i++) { op_d[i] = b_info->op[b_info->b_idx + b_info->op_idx++]; - session = (struct ccp_session *) - op_d[i]->sym->session->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (op_d[i]->sym->session);switch (session->cmd_id) {case CCP_CMD_CIPHER : op_d[i]->status =RTE_CRYPTO_OP_STATUS_SUCCESS ;
drivers/crypto/ccp/ccp_pmd_ops.c +1 -1
@@ -767,7 +767,7 @@ ccp_pmd_sym_session_configure(struct rte_cryptodev *dev,return -ENOMEM ; } - sess_private_data = (void *)sess->driver_priv_data; + sess_private_data =CRYPTODEV_GET_SYM_SESS_PRIV (sess); internals = (struct ccp_private *)dev->data->dev_private; ret =ccp_set_session_parameters (sess_private_data, xform, internals);
drivers/crypto/ccp/rte_ccp_pmd.c +2 -2
@@ -56,7 +56,7 @@ get_ccp_session(struct ccp_qp *qp, struct rte_crypto_op *op)if (unlikely (op->sym->session ==NULL ))return NULL ; - sess = (void *)op->sym->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); }else if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {struct rte_cryptodev_sym_session *_sess;struct ccp_private *internals;
@@ -152,7 +152,7 @@ ccp_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,if (unlikely (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS )) {struct ccp_session *sess = - (void *)ops[i]->sym->session->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (ops[i]->sym->session);memset (sess,0 ,sizeof (*sess)); rte_mempool_put(qp->sess_mp,
drivers/crypto/cnxk/cn10k_cryptodev_ops.c +3 -5
@@ -110,7 +110,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],return 0 ; w7 = sec_sess->sa.inst.w7; }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { - sess = (void *)sym_op->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (sym_op->session); ret =cpt_sym_inst_fill (qp, op, sess, infl_req, &inst[0 ]);if (unlikely (ret))
@@ -293,9 +293,7 @@ cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, }else if (sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv = (void *)( - ((struct rte_cryptodev_sym_session *)sess)-> - driver_priv_data); + priv =CRYPTODEV_GET_SYM_SESS_PRIV (sess); priv->qp = qp; priv->cpt_inst_w2 = w2; }else
@@ -332,7 +330,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv = (void *)op->sym->session->driver_priv_data; + priv =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); *qp = priv->qp; *w2 = priv->cpt_inst_w2; }else {
drivers/crypto/cnxk/cn9k_cryptodev_ops.c +4 -6
@@ -69,7 +69,7 @@ cn9k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op)if (ret) goto sess_put; - priv = (void *)sess->driver_priv_data; + priv =CRYPTODEV_GET_SYM_SESS_PRIV (sess); sym_op->session = sess;
@@ -92,7 +92,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { sym_op = op->sym; - sess = (void *)sym_op->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (sym_op->session); ret =cpt_sym_inst_fill (qp, op, sess, infl_req, inst); inst->w7.u64 = sess->cpt_inst_w7; }else if (op->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION )
@@ -344,9 +344,7 @@ cn9k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, }else if (sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv = (void *)(( - (struct rte_cryptodev_sym_session *)sess)-> - driver_priv_data); + priv =CRYPTODEV_GET_SYM_SESS_PRIV (sess); priv->qp = qp; priv->cpt_inst_w2 = w2; }else
@@ -383,7 +381,7 @@ cn9k_ca_meta_info_extract(struct rte_crypto_op *op, }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv = (void *)op->sym->session->driver_priv_data; + priv =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); *qp = priv->qp; inst->w2.u64 = priv->cpt_inst_w2; }else {
drivers/crypto/cnxk/cnxk_cryptodev_ops.c +2 -2
@@ -624,7 +624,7 @@ sym_session_configure(struct roc_cpt *roc_cpt, struct rte_cryptodev_sym_session *sess) {enum cpt_dp_thread_type thr_type; -struct cnxk_se_sess *sess_priv = (void *)sess->driver_priv_data; +struct cnxk_se_sess *sess_priv =CRYPTODEV_GET_SYM_SESS_PRIV (sess);int ret;memset (sess_priv,0 ,sizeof (struct cnxk_se_sess ));
@@ -693,7 +693,7 @@ cnxk_cpt_sym_session_configure(struct rte_cryptodev *dev,void sym_session_clear (struct rte_cryptodev_sym_session *sess) { -struct cnxk_se_sess *sess_priv = (void *)sess->driver_priv_data; +struct cnxk_se_sess *sess_priv =CRYPTODEV_GET_SYM_SESS_PRIV (sess);if (sess_priv->roc_se_ctx.auth_key !=NULL )plt_free (sess_priv->roc_se_ctx.auth_key);
drivers/crypto/cnxk/cnxk_cryptodev_ops.h +1 -1
@@ -5,7 +5,7 @@#ifndef _CNXK_CRYPTODEV_OPS_H_#define _CNXK_CRYPTODEV_OPS_H_ -#include <rte_cryptodev.h> +#include <cryptodev_pmd.h> #include <rte_event_crypto_adapter.h> #include "roc_api.h"
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +5 -6
@@ -1379,7 +1379,7 @@ build_sec_fd(struct rte_crypto_op *op,dpaa2_sec_session *sess;if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa2_sec_session *)op->sym->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);#ifdef RTE_LIB_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa2_sec_session *)get_sec_session_private_data(
@@ -1677,7 +1677,7 @@ dpaa2_sec_dump(struct rte_crypto_op *op)struct rte_crypto_sym_op *sym_op;if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa2_sec_session *)op->sym->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa2_sec_session *)get_sec_session_private_data(
@@ -3756,7 +3756,7 @@ dpaa2_sec_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, struct rte_cryptodev_sym_session *sess) { - void *sess_private_data = (void *)sess->driver_priv_data; + void *sess_private_data = CRYPTODEV_GET_SYM_SESS_PRIV(sess);int ret; ret =dpaa2_sec_set_session_parameters (xform, sess_private_data);
@@ -3775,10 +3775,9 @@ dpaa2_sec_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) {PMD_INIT_FUNC_TRACE (); -void *sess_priv = (void *)sess->driver_priv_data; -dpaa2_sec_session *s = (dpaa2_sec_session *)sess_priv; +dpaa2_sec_session *s =CRYPTODEV_GET_SYM_SESS_PRIV (sess); -if (sess_priv) { +if (s) {rte_free (s->ctxt);rte_free (s->cipher_key.data);rte_free (s->auth_key.data);
drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c +1 -1
@@ -1012,7 +1012,7 @@ dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess = (dpaa2_sec_session *)get_sec_session_private_data ( session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (void *)session_ctx.crypto_sess->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (session_ctx.crypto_sess); else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa2_sec_raw_dequeue_burst;
drivers/crypto/dpaa_sec/dpaa_sec.c +4 -5
@@ -670,7 +670,7 @@ dpaa_sec_dump(struct dpaa_sec_op_ctx *ctx, struct dpaa_sec_qp *qp)struct qm_sg_entry sg[2 ];if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa_sec_session *)op->sym->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa_sec_session *)
@@ -1924,8 +1924,7 @@ dpaa_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops,switch (op->sess_type) {case RTE_CRYPTO_OP_WITH_SESSION : - ses = (void *) - op->sym->session->driver_priv_data; + ses =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);break ;#ifdef RTE_LIB_SECURITY case RTE_CRYPTO_OP_SECURITY_SESSION :
@@ -2673,7 +2672,7 @@ dpaa_sec_sym_session_configure(struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, struct rte_cryptodev_sym_session *sess) { - void *sess_private_data = (void *)sess->driver_priv_data; + void *sess_private_data = CRYPTODEV_GET_SYM_SESS_PRIV(sess);int ret;PMD_INIT_FUNC_TRACE ();
@@ -2714,7 +2713,7 @@ dpaa_sec_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) {PMD_INIT_FUNC_TRACE (); -void *sess_priv = (void *)sess->driver_priv_data; +void *sess_priv =CRYPTODEV_GET_SYM_SESS_PRIV (sess);dpaa_sec_session *s = (dpaa_sec_session *)sess_priv;free_session_memory (dev, s);
drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c +1 -1
@@ -1018,7 +1018,7 @@ dpaa_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) sess = (dpaa_sec_session *) - session_ctx.crypto_sess->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (session_ctx.crypto_sess); else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa_sec_raw_dequeue_burst;
drivers/crypto/ipsec_mb/ipsec_mb_ops.c +1 -1
@@ -329,7 +329,7 @@ ipsec_mb_sym_session_configure( } ret = (*pmd_data->session_configure)(mb_mgr, - (void *)sess->driver_priv_data, xform); +CRYPTODEV_GET_SYM_SESS_PRIV (sess), xform);if (ret !=0 ) {IPSEC_MB_LOG (ERR ,"failed configure session parameters" );
drivers/crypto/ipsec_mb/ipsec_mb_private.h +2 -2
@@ -434,7 +434,7 @@ ipsec_mb_get_session_private(struct ipsec_mb_qp *qp, struct rte_crypto_op *op) }if (unlikely (pmd_data->session_configure (qp->mb_mgr, - (void *)sess->driver_priv_data, sym_op->xform) !=0 )) { +CRYPTODEV_GET_SYM_SESS_PRIV (sess), sym_op->xform) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); goto error_exit; }
@@ -447,7 +447,7 @@ ipsec_mb_get_session_private(struct ipsec_mb_qp *qp, struct rte_crypto_op *op)IPSEC_MB_LOG (ERR ,"Unrecognized session type %u" , sess_type); } -return (void *)sess->driver_priv_data; +return CRYPTODEV_GET_SYM_SESS_PRIV (sess); error_exit: op->status =RTE_CRYPTO_OP_STATUS_INVALID_SESSION ;
drivers/crypto/ipsec_mb/pmd_aesni_gcm.c +3 -3
@@ -468,7 +468,7 @@ aesni_gcm_get_session(struct ipsec_mb_qp *qp, }if (unlikely (aesni_gcm_session_configure (qp->mb_mgr, - (void *)sess->driver_priv_data, +CRYPTODEV_GET_SYM_SESS_PRIV (sess), sym_op->xform) !=0 )) {rte_mempool_put (qp->sess_mp, sess); sess =NULL ;
@@ -479,7 +479,7 @@ aesni_gcm_get_session(struct ipsec_mb_qp *qp,if (unlikely (sess ==NULL )) op->status =RTE_CRYPTO_OP_STATUS_INVALID_SESSION ; -return (void *)sess->driver_priv_data; +return CRYPTODEV_GET_SYM_SESS_PRIV (sess); } static uint16_t
@@ -704,7 +704,7 @@ aesni_gcm_process_bulk(struct rte_cryptodev *dev __rte_unused, __rte_unused union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec) { - struct aesni_gcm_session *s = (void *)sess->driver_priv_data; + struct aesni_gcm_session *s = CRYPTODEV_GET_SYM_SESS_PRIV(sess);struct gcm_context_data gdata_ctx;IMB_MGR *mb_mgr;
drivers/crypto/ipsec_mb/pmd_aesni_mb.c +2 -2
@@ -1723,7 +1723,7 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) sess =get_sec_session_private_data (op->sym->sec_session); } else#endif - sess = (void *)op->sym->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);if (likely (op->status ==RTE_CRYPTO_OP_STATUS_NOT_PROCESSED )) {switch (job->status) {
@@ -2024,7 +2024,7 @@ aesni_mb_process_bulk(struct rte_cryptodev *dev __rte_unused,void *buf;IMB_JOB *job;IMB_MGR *mb_mgr; -struct aesni_mb_session *s = (void *)sess->driver_priv_data; +struct aesni_mb_session *s =CRYPTODEV_GET_SYM_SESS_PRIV (sess);uint8_t tmp_dgst[vec->num][DIGEST_LENGTH_MAX ];/* get per-thread MB MGR, create one if needed */
drivers/crypto/ipsec_mb/pmd_kasumi.c +1 -1
@@ -282,7 +282,7 @@ process_op_bit(struct rte_crypto_op *op, struct kasumi_session *session,/* Free session if a session-less crypto op. */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -memset (op->sym->session->driver_priv_data,0 , +memset (CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session),0 ,sizeof (struct kasumi_session ));rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ;
drivers/crypto/ipsec_mb/pmd_snow3g.c +1 -1
@@ -413,7 +413,7 @@ process_op_bit(struct rte_crypto_op *op, struct snow3g_session *session,/* Free session if a session-less crypto op. */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -memset (op->sym->session->driver_priv_data,0 , +memset (CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session),0 ,sizeof (struct snow3g_session ));rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ;
drivers/crypto/mlx5/mlx5_crypto.c +3 -4
@@ -175,7 +175,7 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev, {struct mlx5_crypto_priv *priv = dev->data->dev_private;struct mlx5_crypto_session *sess_private_data = - (void *)session->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (session);struct rte_crypto_cipher_xform *cipher;uint8_t encryption_order;
@@ -239,7 +239,7 @@ mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) {struct mlx5_crypto_priv *priv = dev->data->dev_private; -struct mlx5_crypto_session *spriv = (void *)sess->driver_priv_data; +struct mlx5_crypto_session *spriv =CRYPTODEV_GET_SYM_SESS_PRIV (sess);if (unlikely (spriv ==NULL )) {DRV_LOG (ERR ,"Failed to get session %p private data." , spriv);
@@ -355,8 +355,7 @@ mlx5_crypto_wqe_set(struct mlx5_crypto_priv *priv,struct rte_crypto_op *op, struct mlx5_umr_wqe *umr) { - struct mlx5_crypto_session *sess = - (void *)op->sym->session->driver_priv_data; + struct mlx5_crypto_session *sess = CRYPTODEV_GET_SYM_SESS_PRIV(op->sym->session);struct mlx5_wqe_cseg *cseg = &umr->ctr;struct mlx5_wqe_mkey_cseg *mkc = &umr->mkc;struct mlx5_wqe_dseg *klms = &umr->kseg[0 ];
drivers/crypto/nitrox/nitrox_sym.c +3 -3
@@ -534,7 +534,7 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev __rte_unused,struct rte_crypto_sym_xform *xform, struct rte_cryptodev_sym_session *sess) { - struct nitrox_crypto_ctx *ctx = (void *)sess->driver_priv_data; + struct nitrox_crypto_ctx *ctx = CRYPTODEV_GET_SYM_SESS_PRIV(sess);struct rte_crypto_cipher_xform *cipher_xform =NULL ;struct rte_crypto_auth_xform *auth_xform =NULL ;struct rte_crypto_aead_xform *aead_xform =NULL ;
@@ -577,7 +577,7 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev __rte_unused, goto err; } - ctx->iova = sess->driver_priv_data_iova; + ctx->iova =CRYPTODEV_GET_SYM_SESS_PRIV_IOVA (sess);return 0 ; err:return ret;
@@ -593,7 +593,7 @@ get_crypto_ctx(struct rte_crypto_op *op) {if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {if (likely (op->sym->session)) -return (void *)op->sym->session->driver_priv_data; +return CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session); }return NULL ;
drivers/crypto/null/null_crypto_pmd.c +1 -2
@@ -77,8 +77,7 @@ get_session(struct null_crypto_qp *qp, struct rte_crypto_op *op)if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {if (likely (sym_op->session !=NULL )) - sess = (struct null_crypto_session *) - sym_op->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (sym_op->session); }else {struct rte_cryptodev_sym_session *_sess =NULL ;
drivers/crypto/null/null_crypto_pmd_ops.c +1 -1
@@ -265,7 +265,7 @@ null_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused,return -EINVAL ; } - sess_private_data = (void *)sess->driver_priv_data; + sess_private_data =CRYPTODEV_GET_SYM_SESS_PRIV (sess); ret =null_crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {
drivers/crypto/octeontx/otx_cryptodev_ops.c +5 -5
@@ -248,7 +248,7 @@ sym_session_configure(struct rte_crypto_sym_xform *xform,struct rte_crypto_sym_xform *temp_xform = xform;struct cpt_sess_misc *misc;vq_cmd_word3_t vq_cmd_w3; -void *priv = (void *)sess->driver_priv_data; +void *priv =CRYPTODEV_GET_SYM_SESS_PRIV (sess);int ret; ret =sym_xform_verify (xform);
@@ -294,7 +294,7 @@ sym_session_configure(struct rte_crypto_sym_xform *xform, goto priv_put; } - misc->ctx_dma_addr = sess->driver_priv_data_iova + + misc->ctx_dma_addr =CRYPTODEV_GET_SYM_SESS_PRIV_IOVA (sess) +sizeof (struct cpt_sess_misc ); vq_cmd_w3.u64 =0 ;
@@ -313,7 +313,7 @@ sym_session_configure(struct rte_crypto_sym_xform *xform,static void sym_session_clear (struct rte_cryptodev_sym_session *sess) { -void *priv = (void *)sess->driver_priv_data; +void *priv =CRYPTODEV_GET_SYM_SESS_PRIV (sess);struct cpt_sess_misc *misc;struct cpt_ctx *ctx;
@@ -507,7 +507,7 @@ otx_cpt_enq_single_sym(struct cpt_instance *instance,void *req;uint64_t cpt_op; - sess = (struct cpt_sess_misc *)sym_op->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (sym_op->session); cpt_op = sess->cpt_op;if (likely (cpt_op &CPT_OP_CIPHER_MASK ))
@@ -844,7 +844,7 @@ static inline void free_sym_session_data(const struct cpt_instance *instance, struct rte_crypto_op *cop) { - void *sess_private_data_t = (void *)cop->sym->session->driver_priv_data; + void *sess_private_data_t = CRYPTODEV_GET_SYM_SESS_PRIV(cop->sym->session);memset (sess_private_data_t,0 ,cpt_get_session_size ());rte_mempool_put (instance->sess_mp, cop->sym->session);
drivers/crypto/openssl/rte_openssl_pmd.c +2 -2
@@ -887,8 +887,8 @@ get_session(struct openssl_qp *qp, struct rte_crypto_op *op)if (op->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) {/* get existing session */ if (likely (op->sym->session !=NULL )) - sess = (void *) - op->sym->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV ( + op->sym->session); }else {if (likely (op->asym->session !=NULL )) asym_sess = (struct openssl_asym_session *)
drivers/crypto/openssl/rte_openssl_pmd_ops.c +2 -2
@@ -795,7 +795,7 @@ openssl_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, struct rte_cryptodev_sym_session *sess) { - void *sess_private_data = (void *)sess->driver_priv_data; + void *sess_private_data = CRYPTODEV_GET_SYM_SESS_PRIV(sess);int ret;if (unlikely (sess ==NULL )) {
@@ -1319,7 +1319,7 @@ static void openssl_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) { -void *sess_priv = (void *)sess->driver_priv_data; +void *sess_priv =CRYPTODEV_GET_SYM_SESS_PRIV (sess);/* Zero out the whole structure */ openssl_reset_session (sess_priv);
drivers/crypto/qat/qat_sym.c +2 -2
@@ -67,7 +67,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,return -EINVAL ;if (likely (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION )) { - ctx = (void *)op->sym->session->driver_priv_data; + ctx = (void *)CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);if (sess != (uintptr_t )ctx) {struct rte_cryptodev *cdev;struct qat_cryptodev_private *internals;
@@ -386,7 +386,7 @@ qat_sym_configure_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id,if (sess_type !=RTE_CRYPTO_OP_WITH_SESSION )return -EINVAL ; - ctx = (void *)session_ctx.crypto_sess->driver_priv_data; + ctx =CRYPTODEV_GET_SYM_SESS_PRIV (session_ctx.crypto_sess); dp_ctx->session = ctx;
drivers/crypto/qat/qat_sym.h +1 -2
@@ -316,8 +316,7 @@ qat_sym_process_response(void **op, uint8_t *resp, void *op_cookie, } else#endif { - sess = (struct qat_sym_session *) - rx_op->sym->session->driver_priv_data; + sess =CRYPTODEV_GET_SYM_SESS_PRIV (rx_op->sym->session); is_docsis_sec =0 ; }
drivers/crypto/qat/qat_sym_session.c +3 -3
@@ -222,7 +222,7 @@ void qat_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) { -struct qat_sym_session *s = (void *)sess->driver_priv_data; +struct qat_sym_session *s =CRYPTODEV_GET_SYM_SESS_PRIV (sess);if (s->bpi_ctx)bpi_cipher_ctx_free (s->bpi_ctx);
@@ -505,8 +505,8 @@ qat_sym_session_configure(struct rte_cryptodev *dev,return -EINVAL ;#endif ret =qat_sym_session_set_parameters (dev, xform, - (void *)sess->driver_priv_data, - sess->driver_priv_data_iova); +CRYPTODEV_GET_SYM_SESS_PRIV (sess), +CRYPTODEV_GET_SYM_SESS_PRIV_IOVA (sess));if (ret !=0 ) {QAT_LOG (ERR ,"Crypto QAT PMD: failed to configure session parameters" );
drivers/crypto/scheduler/scheduler_pkt_size_distr.c +5 -8
@@ -76,13 +76,10 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops)rte_prefetch0 ((uint8_t *)ops[i +7 ]->sym->session +sizeof (struct rte_cryptodev_sym_session )); - sess_ctx[0 ] = (void *)ops[i]->sym->session->driver_priv_data; - sess_ctx[1 ] = - (void *)ops[i +1 ]->sym->session->driver_priv_data; - sess_ctx[2 ] = - (void *)ops[i +2 ]->sym->session->driver_priv_data; - sess_ctx[3 ] = - (void *)ops[i +3 ]->sym->session->driver_priv_data; + sess_ctx[0 ] =CRYPTODEV_GET_SYM_SESS_PRIV (ops[i]->sym->session); + sess_ctx[1 ] =CRYPTODEV_GET_SYM_SESS_PRIV (ops[i +1 ]->sym->session); + sess_ctx[2 ] =CRYPTODEV_GET_SYM_SESS_PRIV (ops[i +2 ]->sym->session); + sess_ctx[3 ] =CRYPTODEV_GET_SYM_SESS_PRIV (ops[i +3 ]->sym->session);/* job_len is initialized as cipher data length, once * it is 0, equals to auth data length
@@ -166,7 +163,7 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops)for (; i < nb_ops; i++) {struct scheduler_session_ctx *sess_ctx = - (void *)ops[i]->sym->session->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (ops[i]->sym->session);uint32_t job_len;uint8_t target;
drivers/crypto/scheduler/scheduler_pmd_ops.c +2 -2
@@ -481,7 +481,7 @@ scheduler_pmd_sym_session_configure(struct rte_cryptodev *dev, {struct scheduler_ctx *sched_ctx = dev->data->dev_private;struct rte_mempool *mp =rte_mempool_from_obj (sess); -struct scheduler_session_ctx *sess_ctx = (void *)sess->driver_priv_data; +struct scheduler_session_ctx *sess_ctx =CRYPTODEV_GET_SYM_SESS_PRIV (sess);struct scheduler_configured_sess_info configured_sess[RTE_CRYPTODEV_SCHEDULER_MAX_NB_WORKERS ] = {0 };uint32_t i, j, n_configured_sess =0 ;
@@ -543,7 +543,7 @@ scheduler_pmd_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) {struct scheduler_ctx *sched_ctx = dev->data->dev_private; -struct scheduler_session_ctx *sess_ctx = (void *)sess->driver_priv_data; +struct scheduler_session_ctx *sess_ctx =CRYPTODEV_GET_SYM_SESS_PRIV (sess);struct scheduler_configured_sess_info deleted_sess[RTE_CRYPTODEV_SCHEDULER_MAX_NB_WORKERS ] = {0 };uint32_t i, j, n_deleted_sess =0 ;
drivers/crypto/scheduler/scheduler_pmd_private.h +15 -15
@@ -131,28 +131,28 @@ scheduler_set_worker_session(struct rte_crypto_op **ops, uint16_t nb_ops,if (op[0 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct scheduler_session_ctx *sess_ctx = - (void *)op[0 ]->sym->session->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (op[0 ]->sym->session); op[0 ]->sym->session = sess_ctx->worker_sess[worker_index]; }if (op[1 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct scheduler_session_ctx *sess_ctx = - (void *)op[1 ]->sym->session->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (op[1 ]->sym->session); op[1 ]->sym->session = sess_ctx->worker_sess[worker_index]; }if (op[2 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct scheduler_session_ctx *sess_ctx = - (void *)op[2 ]->sym->session->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (op[2 ]->sym->session); op[2 ]->sym->session = sess_ctx->worker_sess[worker_index]; }if (op[3 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct scheduler_session_ctx *sess_ctx = - (void *)op[3 ]->sym->session->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (op[3 ]->sym->session); op[3 ]->sym->session = sess_ctx->worker_sess[worker_index]; }
@@ -164,7 +164,7 @@ scheduler_set_worker_session(struct rte_crypto_op **ops, uint16_t nb_ops,while (n--) {if (op[0 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct scheduler_session_ctx *sess_ctx = - (void *)op[0 ]->sym->session->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (op[0 ]->sym->session); op[0 ]->sym->session = sess_ctx->worker_sess[worker_index];
@@ -195,17 +195,17 @@ scheduler_retrieve_session(struct rte_crypto_op **ops, uint16_t nb_ops) }if (op[0 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - op[0 ]->sym->session = - (void *)op[0 ]->sym->session->opaque_data; + op[0 ]->sym->session = (void *) +rte_cryptodev_sym_session_opaque_data_get (op[0 ]->sym->session);if (op[1 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - op[1 ]->sym->session = - (void *)op[1 ]->sym->session->opaque_data; + op[1 ]->sym->session = (void *) +rte_cryptodev_sym_session_opaque_data_get (op[1 ]->sym->session);if (op[2 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - op[2 ]->sym->session = - (void *)op[2 ]->sym->session->opaque_data; + op[2 ]->sym->session = (void *) +rte_cryptodev_sym_session_opaque_data_get (op[2 ]->sym->session);if (op[3 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - op[3 ]->sym->session = - (void *)op[3 ]->sym->session->opaque_data; + op[3 ]->sym->session = (void *) +rte_cryptodev_sym_session_opaque_data_get (op[3 ]->sym->session); op +=4 ; n -=4 ;
@@ -213,8 +213,8 @@ scheduler_retrieve_session(struct rte_crypto_op **ops, uint16_t nb_ops)while (n--) {if (op[0 ]->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - op[0 ]->sym->session = - (void *)op[0 ]->sym->session->opaque_data; + op[0 ]->sym->session = (void *) +rte_cryptodev_sym_session_opaque_data_get (op[0 ]->sym->session); op++; } }
drivers/crypto/virtio/virtio_cryptodev.c +2 -2
@@ -951,7 +951,7 @@ virtio_crypto_sym_clear_session( hw = dev->data->dev_private; vq = hw->cvq; - session = (struct virtio_crypto_session *)sess->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (sess);VIRTIO_CRYPTO_SESSION_LOG_INFO ("vq->vq_desc_head_idx = %d, " "vq = %p" , vq->vq_desc_head_idx, vq);
@@ -1318,7 +1318,7 @@ virtio_crypto_sym_configure_session(VIRTIO_CRYPTO_SESSION_LOG_ERR ("Invalid parameters" );return ret; } - session = (struct virtio_crypto_session *)sess->driver_priv_data; + session =CRYPTODEV_GET_SYM_SESS_PRIV (sess);memset (session,0 ,sizeof (struct virtio_crypto_session )); ctrl_req = &session->ctrl; ctrl_req->header.opcode =VIRTIO_CRYPTO_CIPHER_CREATE_SESSION ;
drivers/crypto/virtio/virtio_rxtx.c +1 -1
@@ -207,7 +207,7 @@ virtqueue_crypto_sym_enqueue_xmit( offsetof(struct virtio_crypto_op_cookie , iv);struct rte_crypto_sym_op *sym_op = cop->sym;struct virtio_crypto_session *session = - (void *)cop->sym->session->driver_priv_data; +CRYPTODEV_GET_SYM_SESS_PRIV (cop->sym->session);struct virtio_crypto_op_data_req *op_data_req;uint32_t hash_result_len =0 ;struct virtio_crypto_op_cookie *crypto_op_cookie;
examples/fips_validation/fips_dev_self_test.c +5 -5
@@ -980,7 +980,7 @@ typedef int (*fips_dev_self_test_prepare_xform_t)(uint8_t, uint32_t);typedef int (*fips_dev_self_test_prepare_op_t )(struct rte_crypto_op *, -struct rte_mbuf *,struct rte_cryptodev_sym_session *, +struct rte_mbuf *,void *,uint32_t ,struct fips_dev_self_test_vector *); typedef int (*fips_dev_self_test_check_result_t)(struct rte_crypto_op *,
@@ -1172,7 +1172,7 @@ prepare_aead_xform(uint8_t dev_id,static int prepare_cipher_op (struct rte_crypto_op *op,struct rte_mbuf *mbuf, -struct rte_cryptodev_sym_session *session, +void *session,uint32_t dir,struct fips_dev_self_test_vector *vec) {
@@ -1211,7 +1211,7 @@ prepare_cipher_op(struct rte_crypto_op *op,static int prepare_auth_op (struct rte_crypto_op *op,struct rte_mbuf *mbuf, -struct rte_cryptodev_sym_session *session, +void *session,uint32_t dir,struct fips_dev_self_test_vector *vec) {
@@ -1250,7 +1250,7 @@ prepare_auth_op(struct rte_crypto_op *op,static int prepare_aead_op (struct rte_crypto_op *op,struct rte_mbuf *mbuf, -struct rte_cryptodev_sym_session *session, +void *session,uint32_t dir,struct fips_dev_self_test_vector *vec) {
@@ -1462,7 +1462,7 @@ run_single_test(uint8_t dev_id, uint32_t negative_test) {struct rte_crypto_sym_xform xform; -struct rte_cryptodev_sym_session *sess; +void *sess;uint16_t n_deqd;uint8_t key[256 ];int ret;
lib/cryptodev/cryptodev_pmd.h +32 -0
@@ -132,6 +132,38 @@ struct cryptodev_driver {uint8_t id; }; +/** Cryptodev symmetric crypto session +* Each session is derived from a fixed xform chain. Therefore each session +* has a fixed algo, key, op-type, digest_len etc. +*/ +struct rte_cryptodev_sym_session { +RTE_MARKER cacheline0; +uint64_t opaque_data; +/**< Can be used for external metadata */ +uint32_t sess_data_sz; +/**< Pointer to the user data stored after sess data */ +uint16_t user_data_sz; +/**< Session user data will be placed after sess data */ +uint8_t driver_id; +/**< Driver id to get the session priv */ +rte_iova_t driver_priv_data_iova; +/**< Session driver data IOVA address */ + + RTE_MARKER cacheline1 __rte_cache_min_aligned; +/**< Second cache line - start of the driver session data */ +uint8_t driver_priv_data[0 ]; +/**< Driver specific session data, variable size */ +}; + +/** +* Helper macro to get driver private data +*/ +#define CRYPTODEV_GET_SYM_SESS_PRIV (s) \ + (void *)(((struct rte_cryptodev_sym_session *)s)->driver_priv_data) +#define CRYPTODEV_GET_SYM_SESS_PRIV_IOVA (s) \ + (((struct rte_cryptodev_sym_session *)s)->driver_priv_data_iova) + +/** * Get the rte_cryptodev structure device pointer for the device. Assumes a * valid device index.
lib/cryptodev/rte_crypto.h +1 -2
@@ -430,8 +430,7 @@ rte_crypto_op_sym_xforms_alloc(struct rte_crypto_op *op, uint8_t nb_xforms)* @param sess cryptodev session */ static inline int -rte_crypto_op_attach_sym_session (struct rte_crypto_op *op, -struct rte_cryptodev_sym_session *sess) +rte_crypto_op_attach_sym_session (struct rte_crypto_op *op,void *sess) {if (unlikely (op->type !=RTE_CRYPTO_OP_TYPE_SYMMETRIC ))return -1 ;
lib/cryptodev/rte_crypto_sym.h +2 -5
@@ -580,8 +580,6 @@ struct rte_crypto_sym_xform { }; }; -struct rte_cryptodev_sym_session ; -/** * Symmetric Cryptographic Operation. *
@@ -618,7 +616,7 @@ struct rte_crypto_sym_op {RTE_STD_C11 union { -struct rte_cryptodev_sym_session *session; +void *session;/**< Handle for the initialised session context */ struct rte_crypto_sym_xform *xform;/**< Session-less API crypto operation parameters */
@@ -923,8 +921,7 @@ __rte_crypto_sym_op_sym_xforms_alloc(struct rte_crypto_sym_op *sym_op,* @param sess cryptodev session */ static inline int -__rte_crypto_sym_op_attach_sym_session (struct rte_crypto_sym_op *sym_op, -struct rte_cryptodev_sym_session *sess) +__rte_crypto_sym_op_attach_sym_session (struct rte_crypto_sym_op *sym_op,void *sess) { sym_op->session = sess;
lib/cryptodev/rte_cryptodev.c +10 -6
@@ -1941,11 +1941,11 @@ rte_cryptodev_asym_session_create(uint8_t dev_id, }int -rte_cryptodev_sym_session_free (uint8_t dev_id, -struct rte_cryptodev_sym_session *sess) +rte_cryptodev_sym_session_free (uint8_t dev_id,void *_sess) {struct rte_cryptodev *dev;struct rte_mempool *sess_mp; +struct rte_cryptodev_sym_session *sess = _sess;struct rte_cryptodev_sym_session_pool_private_data *pool_priv;if (sess ==NULL )
@@ -2060,10 +2060,11 @@ rte_cryptodev_asym_get_private_session_size(uint8_t dev_id) }int -rte_cryptodev_sym_session_set_user_data ( -struct rte_cryptodev_sym_session *sess,void *data, +rte_cryptodev_sym_session_set_user_data (void *_sess,void *data,uint16_t size) { +struct rte_cryptodev_sym_session *sess = _sess; +if (sess ==NULL )return -EINVAL ;
@@ -2075,8 +2076,10 @@ rte_cryptodev_sym_session_set_user_data( }void * -rte_cryptodev_sym_session_get_user_data (struct rte_cryptodev_sym_session *sess) +rte_cryptodev_sym_session_get_user_data (void *_sess) { +struct rte_cryptodev_sym_session *sess = _sess; +if (sess ==NULL || sess->user_data_sz ==0 )return NULL ;
@@ -2120,10 +2123,11 @@ sym_crypto_fill_status(struct rte_crypto_sym_vec *vec, int32_t errnum)uint32_t rte_cryptodev_sym_cpu_crypto_process (uint8_t dev_id, -struct rte_cryptodev_sym_session *sess,union rte_crypto_sym_ofs ofs, +void *_sess,union rte_crypto_sym_ofs ofs,struct rte_crypto_sym_vec *vec) {struct rte_cryptodev *dev; +struct rte_cryptodev_sym_session *sess = _sess;if (!rte_cryptodev_is_valid_dev (dev_id)) {sym_crypto_fill_status (vec,EINVAL );
lib/cryptodev/rte_cryptodev.h +26 -30
95 lines not on this page · raw message
lib/cryptodev/rte_cryptodev_trace.h +2 -6
@@ -73,14 +73,10 @@ RTE_TRACE_POINT(RTE_TRACE_POINT ( rte_cryptodev_trace_sym_session_create, - RTE_TRACE_POINT_ARGS(uint8_t dev_id, - struct rte_cryptodev_sym_session *sess, void *xforms, + RTE_TRACE_POINT_ARGS(uint8_t dev_id, void *sess, void *xforms, void *mempool), rte_trace_point_emit_u8(dev_id);rte_trace_point_emit_ptr (sess); -rte_trace_point_emit_u64 (sess->opaque_data); -rte_trace_point_emit_u8 (sess->driver_id); -rte_trace_point_emit_u16 (sess->user_data_sz);rte_trace_point_emit_ptr (xforms);rte_trace_point_emit_ptr (mempool); )
@@ -108,7 +104,7 @@ RTE_TRACE_POINT(RTE_TRACE_POINT ( rte_cryptodev_trace_sym_session_free, - RTE_TRACE_POINT_ARGS(struct rte_cryptodev_sym_session *sess), +RTE_TRACE_POINT_ARGS (void *sess), rte_trace_point_emit_ptr(sess); )
lib/ipsec/rte_ipsec_group.h +3 -2
@@ -45,14 +45,15 @@ static inline struct rte_ipsec_session * rte_ipsec_ses_from_crypto(const struct rte_crypto_op *cop) {const struct rte_security_session *ss; -const struct rte_cryptodev_sym_session *cs; +void *cs;if (cop->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION ) { ss = cop->sym[0 ].sec_session;return (struct rte_ipsec_session *)(uintptr_t )ss->opaque_data; }else if (cop->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { cs = cop->sym[0 ].session; -return (struct rte_ipsec_session *)(uintptr_t )cs->opaque_data; +return (struct rte_ipsec_session *)(uintptr_t ) +rte_cryptodev_sym_session_opaque_data_get (cs); }return NULL ; }
lib/ipsec/ses.c +2 -1
@@ -45,7 +45,8 @@ rte_ipsec_session_prepare(struct rte_ipsec_session *ss) ss->pkt_func = fp;if (ss->type ==RTE_SECURITY_ACTION_TYPE_NONE ) - ss->crypto.ses->opaque_data = (uintptr_t )ss; +rte_cryptodev_sym_session_opaque_data_set (ss->crypto.ses, + (uintptr_t )ss);else ss->security.ses->opaque_data = (uintptr_t )ss;
Signature · 1 line
2.25.1
RE: [PATCH v2 3/6] cryptodev: hide sym session structure
Quoted 30 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Wednesday, September 21, 2022 4:03 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; De Lara Guarch, Pablo <[email protected]>; [email protected]; [email protected]; Yang, Qiming <[email protected]>; Wu, Wenjun1 <[email protected]>; [email protected]; Wu, Jingjing <[email protected]>; Xing, Beilei <[email protected]>; [email protected]; Akhil Goyal <[email protected]>; Zhang, Roy Fan <[email protected]>
Subject: [PATCH v2 3/6] cryptodev: hide sym session structureStructure
rte_cryptodev_sym_sessionis moved to internal headers which are not visible to applications.
The only field which should be used by app isopaque_data. This field can now be accessed via set/get APIs added in this patch.
Subsequent changes in app and lib are made to compile the code.Signed-off-by Akhil Goyal <[email protected]>Signed-off-by Fan Zhang <[email protected]>---
PATCH v2 4/6 security: remove priv mempool usage
In reply to Akhil GoyalAs per current design, rte_security_session_create() unnecessarily use 2 mempool objects for a single session.
To address this, the API will now take only 1 mempool object instead of 2. With this change, the library layer will get the object from mempool and session priv data is stored contiguously in the same mempool object.
User need to ensure that the mempool created in application is big enough for session private data as well. This can be ensured if the pool is created after getting size of session priv data using API rte_security_session_get_size().
Since set and get pkt metadata for security sessions are now made inline for Inline crypto/proto mode, a new member fast_mdata is added to the rte_security_session.
To access opaque data and fast_mdata will be accessed via inline APIs which can do pointer manipulations inside library from session_private_data pointer coming from application.
--- app/test-crypto-perf/cperf.h | 1 - app/test-crypto-perf/cperf_ops.c | 13 +- app/test-crypto-perf/cperf_ops.h | 2 +- app/test-crypto-perf/cperf_test_latency.c | 3 +- app/test-crypto-perf/cperf_test_latency.h | 1 - .../cperf_test_pmd_cyclecount.c | 3 +- .../cperf_test_pmd_cyclecount.h | 1 - app/test-crypto-perf/cperf_test_throughput.c | 3 +- app/test-crypto-perf/cperf_test_throughput.h | 1 - app/test-crypto-perf/cperf_test_verify.c | 3 +- app/test-crypto-perf/cperf_test_verify.h | 1 - app/test-crypto-perf/main.c | 3 - app/test/test_cryptodev.c | 44 +----- app/test/test_ipsec.c | 7 +- app/test/test_security.c | 146 ++---------------- app/test/test_security_inline_proto.c | 16 +- drivers/common/cnxk/roc_cpt.c | 4 +- drivers/crypto/caam_jr/caam_jr.c | 31 +--- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 7 +- drivers/crypto/cnxk/cn10k_ipsec.c | 45 ++---- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 9 +- drivers/crypto/cnxk/cn9k_ipsec.c | 54 ++----- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 29 +--- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 3 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 25 +-- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 3 +- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 26 +--- drivers/crypto/mvsam/rte_mrvl_pmd.c | 3 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 21 +-- drivers/crypto/qat/qat_sym.c | 3 +- drivers/crypto/qat/qat_sym.h | 11 +- drivers/crypto/qat/qat_sym_session.c | 27 +--- drivers/crypto/qat/qat_sym_session.h | 2 +- drivers/net/cnxk/cn10k_ethdev_sec.c | 38 ++--- drivers/net/cnxk/cn9k_ethdev_sec.c | 32 ++-- drivers/net/iavf/iavf_ipsec_crypto.c | 23 +-- drivers/net/ixgbe/ixgbe_ipsec.c | 31 ++-- drivers/net/txgbe/txgbe_ipsec.c | 32 ++-- examples/ipsec-secgw/ipsec-secgw.c | 34 ---- examples/ipsec-secgw/ipsec.c | 9 +- examples/ipsec-secgw/ipsec.h | 1 - lib/cryptodev/rte_cryptodev.h | 2 +- lib/security/rte_security.c | 20 ++- lib/security/rte_security.h | 30 ++-- lib/security/rte_security_driver.h | 13 +- 45 files changed, 193 insertions(+), 623 deletions(-)
app/test-crypto-perf/cperf.h +0 -1
@@ -15,7 +15,6 @@ struct cperf_op_fns; typedef void *(*cperf_constructor_t)( struct rte_mempool *sess_mp, - struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options,
app/test-crypto-perf/cperf_ops.c +4 -9
@@ -642,7 +642,6 @@ cperf_set_ops_aead(struct rte_crypto_op **ops,static struct rte_cryptodev_sym_session * create_ipsec_session(struct rte_mempool *sess_mp, - struct rte_mempool *priv_mp, uint8_t dev_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector,
@@ -753,13 +752,11 @@ create_ipsec_session(struct rte_mempool *sess_mp,rte_cryptodev_get_sec_ctx (dev_id);/* Create security session */ -return (void *)rte_security_session_create (ctx, - &sess_conf, sess_mp, priv_mp); +return (void *)rte_security_session_create (ctx, &sess_conf, sess_mp); }static struct rte_cryptodev_sym_session * cperf_create_session(struct rte_mempool *sess_mp, - struct rte_mempool *priv_mp, uint8_t dev_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector,
@@ -859,12 +856,11 @@ cperf_create_session(struct rte_mempool *sess_mp,rte_cryptodev_get_sec_ctx (dev_id);/* Create security session */ -return (void *)rte_security_session_create (ctx, - &sess_conf, sess_mp, priv_mp); +return (void *)rte_security_session_create (ctx, &sess_conf, sess_mp); }if (options->op_type ==CPERF_IPSEC ) { -return create_ipsec_session (sess_mp, priv_mp, dev_id, +return create_ipsec_session (sess_mp, dev_id, options, test_vector, iv_offset); }
@@ -908,8 +904,7 @@ cperf_create_session(struct rte_mempool *sess_mp,rte_cryptodev_get_sec_ctx (dev_id);/* Create security session */ -return (void *)rte_security_session_create (ctx, - &sess_conf, sess_mp, priv_mp); + return (void *)rte_security_session_create(ctx, &sess_conf, sess_mp); }#endif /*
app/test-crypto-perf/cperf_ops.h +1 -1
@@ -13,7 +13,7 @@typedef struct rte_cryptodev_sym_session *(*cperf_sessions_create_t )( -struct rte_mempool *sess_mp,struct rte_mempool *sess_priv_mp, +struct rte_mempool *sess_mp,uint8_t dev_id,const struct cperf_options *options,const struct cperf_test_vector *test_vector,uint16_t iv_offset);
app/test-crypto-perf/cperf_test_latency.c +1 -2
@@ -56,7 +56,6 @@ cperf_latency_test_free(struct cperf_latency_ctx *ctx)void * cperf_latency_test_constructor(struct rte_mempool *sess_mp, - struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector,
@@ -81,7 +80,7 @@ cperf_latency_test_constructor(struct rte_mempool *sess_mp,sizeof (struct rte_crypto_sym_op ) +sizeof (struct cperf_op_result *); - ctx->sess = op_fns->sess_create (sess_mp, sess_priv_mp, dev_id, options, + ctx->sess = op_fns->sess_create (sess_mp, dev_id, options, test_vector, iv_offset);if (ctx->sess ==NULL ) goto err;
app/test-crypto-perf/cperf_test_latency.h +0 -1
@@ -17,7 +17,6 @@void * cperf_latency_test_constructor( struct rte_mempool *sess_mp, - struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options,
app/test-crypto-perf/cperf_test_pmd_cyclecount.c +1 -2
@@ -87,7 +87,6 @@ cperf_pmd_cyclecount_test_free(struct cperf_pmd_cyclecount_ctx *ctx)void * cperf_pmd_cyclecount_test_constructor(struct rte_mempool *sess_mp, - struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector,
@@ -114,7 +113,7 @@ cperf_pmd_cyclecount_test_constructor(struct rte_mempool *sess_mp,uint16_t iv_offset =sizeof (struct rte_crypto_op ) +sizeof (struct rte_crypto_sym_op ); - ctx->sess = op_fns->sess_create (sess_mp, sess_priv_mp, dev_id, options, + ctx->sess = op_fns->sess_create (sess_mp, dev_id, options, test_vector, iv_offset);if (ctx->sess ==NULL ) goto err;
app/test-crypto-perf/cperf_test_pmd_cyclecount.h +0 -1
@@ -18,7 +18,6 @@void * cperf_pmd_cyclecount_test_constructor( struct rte_mempool *sess_mp, - struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options,
app/test-crypto-perf/cperf_test_throughput.c +1 -2
@@ -60,7 +60,6 @@ cperf_throughput_test_free(struct cperf_throughput_ctx *ctx)void * cperf_throughput_test_constructor(struct rte_mempool *sess_mp, - struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector,
@@ -83,7 +82,7 @@ cperf_throughput_test_constructor(struct rte_mempool *sess_mp,uint16_t iv_offset =sizeof (struct rte_crypto_op ) +sizeof (struct rte_crypto_sym_op ); - ctx->sess = op_fns->sess_create (sess_mp, sess_priv_mp, dev_id, options, + ctx->sess = op_fns->sess_create (sess_mp, dev_id, options, test_vector, iv_offset);if (ctx->sess ==NULL ) goto err;
app/test-crypto-perf/cperf_test_throughput.h +0 -1
@@ -18,7 +18,6 @@void * cperf_throughput_test_constructor( struct rte_mempool *sess_mp, - struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options,
app/test-crypto-perf/cperf_test_verify.c +1 -2
@@ -48,7 +48,6 @@ cperf_verify_test_free(struct cperf_verify_ctx *ctx)void * cperf_verify_test_constructor(struct rte_mempool *sess_mp, - struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector,
@@ -71,7 +70,7 @@ cperf_verify_test_constructor(struct rte_mempool *sess_mp,uint16_t iv_offset =sizeof (struct rte_crypto_op ) +sizeof (struct rte_crypto_sym_op ); - ctx->sess = op_fns->sess_create (sess_mp, sess_priv_mp, dev_id, options, + ctx->sess = op_fns->sess_create (sess_mp, dev_id, options, test_vector, iv_offset);if (ctx->sess ==NULL ) goto err;
app/test-crypto-perf/cperf_test_verify.h +0 -1
@@ -18,7 +18,6 @@void * cperf_verify_test_constructor( struct rte_mempool *sess_mp, - struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options,
app/test-crypto-perf/main.c +0 -3
@@ -23,7 +23,6 @@static struct {struct rte_mempool *sess_mp; -struct rte_mempool *priv_mp; } session_pool_socket[RTE_MAX_NUMA_NODES ];const char *cperf_test_type_strs[] = {
@@ -114,7 +113,6 @@ fill_session_pool_socket(int32_t socket_id, uint32_t session_priv_size,printf ("Allocated pool \"%s\" on socket %d\n" , mp_name, socket_id); session_pool_socket[socket_id].sess_mp = sess_mp; - session_pool_socket[socket_id].priv_mp = sess_mp; }return 0 ;
@@ -659,7 +657,6 @@ main(int argc, char **argv) ctx[i] = cperf_testmap[opts.test].constructor ( session_pool_socket[socket_id].sess_mp, - session_pool_socket[socket_id].priv_mp, cdev_id, qp_id, &opts, t_vec, &op_fns);if (ctx[i] ==NULL ) {
app/test/test_cryptodev.c +5 -39
98 lines not on this page · raw message
app/test/test_ipsec.c +3 -4
@@ -159,13 +159,12 @@ const struct supported_auth_algo auth_algos[] = {static int dummy_sec_create (void *device,struct rte_security_session_conf *conf, -struct rte_security_session *sess,struct rte_mempool *mp) +struct rte_security_session *sess) {RTE_SET_USED (device);RTE_SET_USED (conf); -RTE_SET_USED (mp); +RTE_SET_USED (sess); - sess->sess_private_data =NULL ;return 0 ; }
@@ -631,7 +630,7 @@ create_dummy_sec_session(struct ipsec_unitest_params *ut,static struct rte_security_session_conf conf; ut->ss[j].security.ses =rte_security_session_create (&dummy_sec_ctx, - &conf, qp->mp_session,NULL ); + &conf, qp->mp_session);if (ut->ss[j].security.ses ==NULL )return -ENOMEM ;
app/test/test_security.c +12 -134
392 lines not on this page · raw message
app/test/test_security_inline_proto.c +1 -15
@@ -66,7 +66,6 @@ extern struct ipsec_test_data pkt_aes_128_cbc_hmac_sha512;static struct rte_mempool *mbufpool;static struct rte_mempool *sess_pool; -static struct rte_mempool *sess_priv_pool;/* ethernet addresses of ports */ static struct rte_ether_addr ports_eth_addr[RTE_MAX_ETHPORTS ];
@@ -311,8 +310,7 @@ create_inline_ipsec_session(struct ipsec_test_data *sa, uint16_t portid,setenv ("ETH_SEC_IV_OVR" , arr,1 ); } - *sess =rte_security_session_create (sec_ctx, - sess_conf, sess_pool, sess_priv_pool); + *sess =rte_security_session_create (sec_ctx, sess_conf, sess_pool);if (*sess ==NULL ) {printf ("SEC Session init failed.\n" );return TEST_FAILED ;
@@ -482,18 +480,6 @@ init_mempools(unsigned int nb_mbuf) }printf ("Allocated sess pool\n" ); } -if (sess_priv_pool ==NULL ) { -snprintf (s,sizeof (s),"sess_priv_pool" ); - sess_priv_pool =rte_mempool_create (s, nb_sess, sess_sz, -MEMPOOL_CACHE_SIZE ,0 , -NULL ,NULL ,NULL ,NULL , -SOCKET_ID_ANY ,0 ); -if (sess_priv_pool ==NULL ) { -printf ("Cannot init sess_priv pool\n" ); -return TEST_FAILED ; - } -printf ("Allocated sess_priv pool\n" ); - }return 0 ; }
drivers/common/cnxk/roc_cpt.c +2 -2
@@ -1004,9 +1004,9 @@ roc_on_cpt_ctx_write(struct roc_cpt_lf *lf, void *sa, uint8_t opcode, inst.w4.s.param1 =0 ; inst.w4.s.param2 =0 ; inst.w4.s.dlen = ctx_len; - inst.dptr =rte_mempool_virt2iova (sa); + inst.dptr = (rte_iova_t )sa; inst.rptr =0 ; - inst.w7.s.cptr =rte_mempool_virt2iova (sa); + inst.w7.s.cptr = (rte_iova_t )sa; inst.w7.s.egrp = egrp; inst.w0.u64 =0 ;
drivers/crypto/caam_jr/caam_jr.c +6 -25
@@ -1359,9 +1359,7 @@ caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp) ses =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);break ;case RTE_CRYPTO_OP_SECURITY_SESSION : - ses = (struct caam_jr_session *) -get_sec_session_private_data ( - op->sym->sec_session); + ses =GET_SECURITY_SESS_PRIV (op->sym->sec_session);break ;default :CAAM_JR_DP_ERR ("sessionless crypto op not supported" );
@@ -1900,18 +1898,12 @@ caam_jr_set_ipsec_session(__rte_unused struct rte_cryptodev *dev,static int caam_jr_security_session_create (void *dev,struct rte_security_session_conf *conf, -struct rte_security_session *sess, -struct rte_mempool *mempool) +struct rte_security_session *sess) { -void *sess_private_data; +void *sess_private_data =GET_SECURITY_SESS_PRIV (sess);struct rte_cryptodev *cdev = (struct rte_cryptodev *)dev;int ret; -if (rte_mempool_get (mempool, &sess_private_data)) { -CAAM_JR_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } -switch (conf->protocol) {case RTE_SECURITY_PROTOCOL_IPSEC : ret = caam_jr_set_ipsec_session(cdev, conf,
@@ -1924,13 +1916,8 @@ caam_jr_security_session_create(void *dev, }if (ret !=0 ) {CAAM_JR_ERR ("failed to configure session parameters" ); -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data); -return ret; } -set_sec_session_private_data (sess, sess_private_data); -return ret; }
@@ -1940,18 +1927,12 @@ caam_jr_security_session_destroy(void *dev __rte_unused, struct rte_security_session *sess) {PMD_INIT_FUNC_TRACE (); -void *sess_priv =get_sec_session_private_data (sess); - -struct caam_jr_session *s = (struct caam_jr_session *)sess_priv; - -if (sess_priv) { -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); +struct caam_jr_session *s =GET_SECURITY_SESS_PRIV (sess); +if (s) {rte_free (s->cipher_key.data);rte_free (s->auth_key.data); -memset (sess,0 ,sizeof (struct caam_jr_session )); -set_sec_session_private_data (sess,NULL ); -rte_mempool_put (sess_mp, sess_priv); +memset (s,0 ,sizeof (struct caam_jr_session )); }return 0 ; }
drivers/crypto/cnxk/cn10k_cryptodev_ops.c +3 -4
@@ -103,8 +103,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],if (op->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) {if (op->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION ) { - sec_sess =get_sec_session_private_data ( - sym_op->sec_session); + sec_sess =GET_SECURITY_SESS_PRIV (sym_op->sec_session); ret =cpt_sec_inst_fill (qp, op, sec_sess, &inst[0 ]);if (unlikely (ret))return 0 ;
@@ -286,7 +285,7 @@ cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused,struct cn10k_sec_session *priv;struct cn10k_ipsec_sa *sa; - priv =get_sec_session_private_data (sess); + priv =GET_SECURITY_SESS_PRIV (sess); sa = &priv->sa; sa->qp = qp; sa->inst.w2 = w2;
@@ -323,7 +322,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op,struct cn10k_sec_session *priv;struct cn10k_ipsec_sa *sa; - priv =get_sec_session_private_data (op->sym->sec_session); + priv =GET_SECURITY_SESS_PRIV (op->sym->sec_session); sa = &priv->sa; *qp = sa->qp; *w2 = sa->inst.w2;
drivers/crypto/cnxk/cn10k_ipsec.c +9 -36
99 lines not on this page · raw message
drivers/crypto/cnxk/cn9k_cryptodev_ops.c +4 -5
@@ -27,7 +27,7 @@ cn9k_cpt_sec_inst_fill(struct rte_crypto_op *op,struct cn9k_ipsec_sa *sa;int ret; - priv =get_sec_session_private_data (op->sym->sec_session); + priv =GET_SECURITY_SESS_PRIV (op->sym->sec_session); sa = &priv->sa;if (unlikely (sym_op->m_dst && sym_op->m_dst != sym_op->m_src)) {
@@ -337,7 +337,7 @@ cn9k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused,struct cn9k_sec_session *priv;struct cn9k_ipsec_sa *sa; - priv =get_sec_session_private_data (sess); + priv =GET_SECURITY_SESS_PRIV (sess); sa = &priv->sa; sa->qp = qp; sa->inst.w2 = w2;
@@ -374,7 +374,7 @@ cn9k_ca_meta_info_extract(struct rte_crypto_op *op,struct cn9k_sec_session *priv;struct cn9k_ipsec_sa *sa; - priv =get_sec_session_private_data (op->sym->sec_session); + priv =GET_SECURITY_SESS_PRIV (op->sym->sec_session); sa = &priv->sa; *qp = sa->qp; inst->w2.u64 = sa->inst.w2;
@@ -539,8 +539,7 @@ cn9k_cpt_sec_post_process(struct rte_crypto_op *cop, CPT_OP_FLAGS_IPSEC_INB_REPLAY)) {int ret; - priv =get_sec_session_private_data ( - sym_op->sec_session); + priv =GET_SECURITY_SESS_PRIV (sym_op->sec_session); sa = &priv->sa; ret = ipsec_antireplay_check(
drivers/crypto/cnxk/cn9k_ipsec.c +14 -40
122 lines not on this page · raw message
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +6 -23
@@ -1382,8 +1382,7 @@ build_sec_fd(struct rte_crypto_op *op, sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);#ifdef RTE_LIB_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) - sess = (dpaa2_sec_session *)get_sec_session_private_data ( - op->sym->sec_session); + sess =GET_SECURITY_SESS_PRIV (op->sym->sec_session);#endif else {DPAA2_SEC_DP_ERR ("Session type invalid\n" );
@@ -1583,8 +1582,7 @@ sec_simple_fd_to_mbuf(const struct qbman_fd *fd) mbuf->buf_iova = op->sym->aead.digest.phys_addr; op->sym->aead.digest.phys_addr =0L ; - sess_priv = (dpaa2_sec_session *)get_sec_session_private_data ( - op->sym->sec_session); + sess_priv =GET_SECURITY_SESS_PRIV (op->sym->sec_session);if (sess_priv->dir ==DIR_ENC ) mbuf->data_off +=SEC_FLC_DHR_OUTBOUND ;else
@@ -1680,8 +1678,7 @@ dpaa2_sec_dump(struct rte_crypto_op *op) sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) - sess = (dpaa2_sec_session *)get_sec_session_private_data ( - op->sym->sec_session); + sess =GET_SECURITY_SESS_PRIV (op->sym->sec_session);#endif if (sess ==NULL )
@@ -3690,18 +3687,12 @@ dpaa2_sec_set_pdcp_session(struct rte_cryptodev *dev,static int dpaa2_sec_security_session_create (void *dev,struct rte_security_session_conf *conf, -struct rte_security_session *sess, -struct rte_mempool *mempool) +struct rte_security_session *sess) { -void *sess_private_data; +void *sess_private_data =GET_SECURITY_SESS_PRIV (sess);struct rte_cryptodev *cdev = (struct rte_cryptodev *)dev;int ret; -if (rte_mempool_get (mempool, &sess_private_data)) { -DPAA2_SEC_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } -switch (conf->protocol) {case RTE_SECURITY_PROTOCOL_IPSEC : ret = dpaa2_sec_set_ipsec_session(cdev, conf,
@@ -3718,13 +3709,9 @@ dpaa2_sec_security_session_create(void *dev, }if (ret !=0 ) {DPAA2_SEC_ERR ("Failed to configure session parameters" ); -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sec_session_private_data (sess, sess_private_data); -return ret; }
@@ -3734,19 +3721,15 @@ dpaa2_sec_security_session_destroy(void *dev __rte_unused, struct rte_security_session *sess) {PMD_INIT_FUNC_TRACE (); -void *sess_priv =get_sec_session_private_data (sess); +void *sess_priv =GET_SECURITY_SESS_PRIV (sess);dpaa2_sec_session *s = (dpaa2_sec_session *)sess_priv;if (sess_priv) { -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -rte_free (s->ctxt);rte_free (s->cipher_key.data);rte_free (s->auth_key.data);memset (s,0 ,sizeof (dpaa2_sec_session)); -set_sec_session_private_data (sess,NULL ); -rte_mempool_put (sess_mp, sess_priv); }return 0 ; }
drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c +1 -2
@@ -1009,8 +1009,7 @@ dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, }if (sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION ) - sess = (dpaa2_sec_session *)get_sec_session_private_data ( - session_ctx.sec_sess); + sess =GET_SECURITY_SESS_PRIV (session_ctx.sec_sess);else if (sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) sess =CRYPTODEV_GET_SYM_SESS_PRIV (session_ctx.crypto_sess);else
drivers/crypto/dpaa_sec/dpaa_sec.c +5 -20
@@ -673,9 +673,7 @@ dpaa_sec_dump(struct dpaa_sec_op_ctx *ctx, struct dpaa_sec_qp *qp) sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) - sess = (dpaa_sec_session *) -get_sec_session_private_data ( - op->sym->sec_session); + sess =GET_SECURITY_SESS_PRIV (op->sym->sec_session);#endif if (sess ==NULL ) {printf ("session is NULL\n" );
@@ -1928,9 +1926,7 @@ dpaa_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops,break ;#ifdef RTE_LIB_SECURITY case RTE_CRYPTO_OP_SECURITY_SESSION : - ses = (dpaa_sec_session *) -get_sec_session_private_data ( - op->sym->sec_session); + ses =GET_SECURITY_SESS_PRIV (op->sym->sec_session);break ;#endif default :
@@ -3245,18 +3241,12 @@ dpaa_sec_set_pdcp_session(struct rte_cryptodev *dev,static int dpaa_sec_security_session_create (void *dev,struct rte_security_session_conf *conf, -struct rte_security_session *sess, -struct rte_mempool *mempool) +struct rte_security_session *sess) { -void *sess_private_data; +void *sess_private_data =GET_SECURITY_SESS_PRIV (sess);struct rte_cryptodev *cdev = (struct rte_cryptodev *)dev;int ret; -if (rte_mempool_get (mempool, &sess_private_data)) { -DPAA_SEC_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } -switch (conf->protocol) {case RTE_SECURITY_PROTOCOL_IPSEC : ret = dpaa_sec_set_ipsec_session(cdev, conf,
@@ -3273,13 +3263,9 @@ dpaa_sec_security_session_create(void *dev, }if (ret !=0 ) {DPAA_SEC_ERR ("failed to configure session parameters" ); -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sec_session_private_data (sess, sess_private_data); - ret =dpaa_sec_prep_cdb (sess_private_data);if (ret) {DPAA_SEC_ERR ("Unable to prepare sec cdb" );
@@ -3295,12 +3281,11 @@ dpaa_sec_security_session_destroy(void *dev __rte_unused, struct rte_security_session *sess) {PMD_INIT_FUNC_TRACE (); -void *sess_priv =get_sec_session_private_data (sess); +void *sess_priv =GET_SECURITY_SESS_PRIV (sess);dpaa_sec_session *s = (dpaa_sec_session *)sess_priv;if (sess_priv) {free_session_memory ((struct rte_cryptodev *)dev, s); -set_sec_session_private_data (sess,NULL ); }return 0 ; }
drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c +1 -2
@@ -1014,8 +1014,7 @@ dpaa_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, }if (sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION ) - sess = (dpaa_sec_session *)get_sec_session_private_data ( - session_ctx.sec_sess); + sess =GET_SECURITY_SESS_PRIV (session_ctx.sec_sess);else if (sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) sess = (dpaa_sec_session *)CRYPTODEV_GET_SYM_SESS_PRIV (session_ctx.crypto_sess);
drivers/crypto/ipsec_mb/pmd_aesni_mb.c +5 -21
@@ -1584,8 +1584,7 @@ set_sec_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, op->status =RTE_CRYPTO_OP_STATUS_INVALID_SESSION ;return -1 ; } - session = (struct aesni_mb_session *) -get_sec_session_private_data (op->sym->sec_session); + session =GET_SECURITY_SESS_PRIV (op->sym->sec_session);if (unlikely (session ==NULL )) { op->status =RTE_CRYPTO_OP_STATUS_INVALID_SESSION ;
@@ -1720,7 +1719,7 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job)* this is for DOCSIS */ is_docsis_sec =1 ; - sess =get_sec_session_private_data (op->sym->sec_session); + sess =GET_SECURITY_SESS_PRIV (op->sym->sec_session); } else#endif sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);
@@ -2106,10 +2105,9 @@ struct rte_cryptodev_ops aesni_mb_pmd_ops = {*/ static int aesni_mb_pmd_sec_sess_create (void *dev,struct rte_security_session_conf *conf, -struct rte_security_session *sess, -struct rte_mempool *mempool) +struct rte_security_session *sess) { -void *sess_private_data; +void *sess_private_data =GET_SECURITY_SESS_PRIV (sess);struct rte_cryptodev *cdev = (struct rte_cryptodev *)dev;int ret;
@@ -2119,24 +2117,14 @@ aesni_mb_pmd_sec_sess_create(void *dev, struct rte_security_session_conf *conf,return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -IPSEC_MB_LOG (ERR ,"Couldn't get object from session mempool" ); -return -ENOMEM ; - } - ret =aesni_mb_set_docsis_sec_session_parameters (cdev, conf, sess_private_data);if (ret !=0 ) {IPSEC_MB_LOG (ERR ,"Failed to configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sec_session_private_data (sess, sess_private_data); -return ret; }
@@ -2145,14 +2133,10 @@ static int aesni_mb_pmd_sec_sess_destroy(void *dev __rte_unused, struct rte_security_session *sess) { -void *sess_priv =get_sec_session_private_data (sess); +void *sess_priv =GET_SECURITY_SESS_PRIV (sess);if (sess_priv) { -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -memset (sess_priv,0 ,sizeof (struct aesni_mb_session )); -set_sec_session_private_data (sess,NULL ); -rte_mempool_put (sess_mp, sess_priv); }return 0 ; }
drivers/crypto/mvsam/rte_mrvl_pmd.c +1 -2
@@ -767,8 +767,7 @@ mrvl_request_prepare_sec(struct sam_cio_ipsec_params *request,return -EINVAL ; } - sess = (struct mrvl_crypto_session *)get_sec_session_private_data ( - op->sym->sec_session); + sess =GET_SECURITY_SESS_PRIV (op->sym->sec_session);if (unlikely (sess ==NULL )) {MRVL_LOG (ERR ,"Session was not created for this device! %d" , cryptodev_driver_id);
drivers/crypto/mvsam/rte_mrvl_pmd_ops.c +3 -18
@@ -833,11 +833,10 @@ struct rte_cryptodev_ops *rte_mrvl_crypto_pmd_ops = &mrvl_crypto_pmd_ops;static int mrvl_crypto_pmd_security_session_create(__rte_unused void *dev,struct rte_security_session_conf *conf, -struct rte_security_session *sess, -struct rte_mempool *mempool) +struct rte_security_session *sess) {struct mrvl_crypto_session *mrvl_sess; -void *sess_private_data; +void *sess_private_data =GET_SECURITY_SESS_PRIV (sess);int ret;if (sess ==NULL ) {
@@ -845,11 +844,6 @@ mrvl_crypto_pmd_security_session_create(__rte_unused void *dev,return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -MRVL_LOG (ERR ,"Couldn't get object from session mempool." ); -return -ENOMEM ; - } -switch (conf->protocol) {case RTE_SECURITY_PROTOCOL_IPSEC : mrvl_sess = (struct mrvl_crypto_session *)sess_private_data;
@@ -863,8 +857,6 @@ mrvl_crypto_pmd_security_session_create(__rte_unused void *dev,if (ret !=0 ) {MRVL_LOG (ERR ,"Failed to configure session parameters." ); -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; }
@@ -878,8 +870,6 @@ mrvl_crypto_pmd_security_session_create(__rte_unused void *dev, &mrvl_sess->sam_sess);if (ret <0 ) {MRVL_LOG (ERR ,"PMD: failed to create IPSEC session." ); -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; }break ;
@@ -889,8 +879,6 @@ mrvl_crypto_pmd_security_session_create(__rte_unused void *dev,return -EINVAL ; } -set_sec_session_private_data (sess, sess_private_data); -return ret; }
@@ -899,13 +887,12 @@ static int mrvl_crypto_pmd_security_session_destroy(void *dev __rte_unused, struct rte_security_session *sess) { -void *sess_priv =get_sec_session_private_data (sess); +void *sess_priv =GET_SECURITY_SESS_PRIV (sess);/* Zero out the whole structure */ if (sess_priv) {struct mrvl_crypto_session *mrvl_sess = (struct mrvl_crypto_session *)sess_priv; -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv);if (mrvl_sess->sam_sess &&sam_session_destroy (mrvl_sess->sam_sess) <0 ) {
@@ -916,8 +903,6 @@ mrvl_crypto_pmd_security_session_destroy(void *dev __rte_unused,rte_free (mrvl_sess->sam_sess_params.auth_key);rte_free (mrvl_sess->sam_sess_params.cipher_iv);memset (sess,0 ,sizeof (struct rte_security_session )); -set_sec_session_private_data (sess,NULL ); -rte_mempool_put (sess_mp, sess_priv); }return 0 ; }
drivers/crypto/qat/qat_sym.c +2 -1
@@ -9,6 +9,7 @@#include <rte_crypto_sym.h> #include <rte_bus_pci.h> #include <rte_byteorder.h> +#include <rte_security_driver.h> #include "qat_sym.h" #include "qat_crypto.h"
@@ -100,7 +101,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,#ifdef RTE_LIB_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { - ctx =get_sec_session_private_data (op->sym->sec_session); + ctx =GET_SECURITY_SESS_PRIV (op->sym->sec_session);if (unlikely (!ctx)) {QAT_DP_LOG (ERR ,"No session for this device" );return -EINVAL ;
drivers/crypto/qat/qat_sym.h +5 -6
@@ -12,6 +12,9 @@#ifdef BUILD_QAT_SYM #include <openssl/evp.h> +#ifdef RTE_LIB_SECURITY +#include <rte_security_driver.h> +#endif #include "qat_common.h" #include "qat_sym_session.h"
@@ -274,9 +277,7 @@ qat_sym_preprocess_requests(void **ops, uint16_t nb_ops) op = (struct rte_crypto_op *)ops[i];if (op->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION ) { - ctx = (struct qat_sym_session *) -get_sec_session_private_data ( - op->sym->sec_session); + ctx =GET_SECURITY_SESS_PRIV (op->sym->sec_session);if (ctx ==NULL || ctx->bpi_ctx ==NULL )continue ;
@@ -309,9 +310,7 @@ qat_sym_process_response(void **op, uint8_t *resp, void *op_cookie,* Assuming at this point that if it's a security * op, that this is for DOCSIS */ - sess = (struct qat_sym_session *) -get_sec_session_private_data ( - rx_op->sym->sec_session); + sess =GET_SECURITY_SESS_PRIV (rx_op->sym->sec_session); is_docsis_sec =1 ; } else#endif
drivers/crypto/qat/qat_sym_session.c +7 -20
@@ -25,7 +25,7 @@#include <rte_malloc.h> #include <rte_crypto_sym.h> #ifdef RTE_LIB_SECURITY -#include <rte_security.h> +#include <rte_security_driver.h> #endif #include "qat_logs.h"
@@ -2541,7 +2541,8 @@ qat_sec_session_check_docsis(struct rte_security_session_conf *conf)static int qat_sec_session_set_docsis_parameters (struct rte_cryptodev *dev, -struct rte_security_session_conf *conf,void *session_private) +struct rte_security_session_conf *conf,void *session_private, +rte_iova_t session_paddr) {int ret;int qat_cmd_id;
@@ -2560,7 +2561,6 @@ qat_sec_session_set_docsis_parameters(struct rte_cryptodev *dev, xform = conf->crypto_xform;/* Verify the session physical address is known */ -rte_iova_t session_paddr =rte_mempool_virt2iova (session);if (session_paddr ==0 || session_paddr ==RTE_BAD_IOVA ) {QAT_LOG (ERR ,"Session physical address unknown. Bad memory pool." );
@@ -2590,10 +2590,9 @@ qat_sec_session_set_docsis_parameters(struct rte_cryptodev *dev,int qat_security_session_create (void *dev,struct rte_security_session_conf *conf, -struct rte_security_session *sess, -struct rte_mempool *mempool) +struct rte_security_session *sess) { -void *sess_private_data; +void *sess_private_data =GET_SECURITY_SESS_PRIV (sess);struct rte_cryptodev *cdev = (struct rte_cryptodev *)dev;struct qat_cryptodev_private *internals = cdev->data->dev_private;enum qat_device_gen qat_dev_gen = internals->qat_dev->qat_dev_gen;
@@ -2606,25 +2605,17 @@ qat_security_session_create(void *dev,return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -QAT_LOG (ERR ,"Couldn't get object from session mempool" ); -return -ENOMEM ; - } -#if (OPENSSL_VERSION_NUMBER >=0x30000000L )if (ossl_legacy_provider_load ())return -EINVAL ;#endif ret =qat_sec_session_set_docsis_parameters (cdev, conf, - sess_private_data); + sess_private_data,GET_SECURITY_SESS_PRIV_IOVA (sess));if (ret !=0 ) {QAT_LOG (ERR ,"Failed to configure session parameters" ); -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sec_session_private_data (sess, sess_private_data); sym_session = (struct qat_sym_session *)sess_private_data; sym_session->dev_id = internals->dev_id;
@@ -2636,17 +2627,13 @@ int qat_security_session_destroy(void *dev __rte_unused, struct rte_security_session *sess) { -void *sess_priv =get_sec_session_private_data (sess); +void *sess_priv =GET_SECURITY_SESS_PRIV (sess);struct qat_sym_session *s = (struct qat_sym_session *)sess_priv;if (sess_priv) {if (s->bpi_ctx)bpi_cipher_ctx_free (s->bpi_ctx);memset (s,0 ,qat_sym_session_get_private_size (dev)); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); - -set_sec_session_private_data (sess,NULL ); -rte_mempool_put (sess_mp, sess_priv); }# if (OPENSSL_VERSION_NUMBER >=0x30000000L )
drivers/crypto/qat/qat_sym_session.h +1 -1
@@ -177,7 +177,7 @@ qat_sym_validate_zuc_key(int key_len, enum icp_qat_hw_cipher_algo *alg);#ifdef RTE_LIB_SECURITY int qat_security_session_create (void *dev,struct rte_security_session_conf *conf, -struct rte_security_session *sess,struct rte_mempool *mempool); +struct rte_security_session *sess);int qat_security_session_destroy (void *dev,struct rte_security_session *sess);#endif
drivers/net/cnxk/cn10k_ethdev_sec.c +13 -25
@@ -583,15 +583,14 @@ cn10k_eth_sec_outb_sa_misc_fill(struct roc_nix *roc_nix,static int cn10k_eth_sec_session_create (void *device,struct rte_security_session_conf *conf, -struct rte_security_session *sess, -struct rte_mempool *mempool) +struct rte_security_session *sess) {struct rte_eth_dev *eth_dev = (struct rte_eth_dev *)device;struct cnxk_eth_dev *dev =cnxk_eth_pmd_priv (eth_dev);struct rte_security_ipsec_xform *ipsec;struct cn10k_sec_sess_priv sess_priv;struct rte_crypto_sym_xform *crypto; -struct cnxk_eth_sec_sess *eth_sec; +struct cnxk_eth_sec_sess *eth_sec =GET_SECURITY_SESS_PRIV (sess);struct roc_nix *nix = &dev->nix;bool inbound, inl_dev;rte_spinlock_t *lock;
@@ -626,11 +625,6 @@ cn10k_eth_sec_session_create(void *device,return -EEXIST ; } -if (rte_mempool_get (mempool, (void **)ð_sec)) { -plt_err ("Could not allocate security session private data" ); -return -ENOMEM ; - } -memset (eth_sec,0 ,sizeof (struct cnxk_eth_sec_sess )); sess_priv.u64 =0 ;
@@ -659,12 +653,12 @@ cn10k_eth_sec_session_create(void *device,"Failed to create ingress sa, inline dev " "not found or spi not in range"); rc = -ENOTSUP ; - goto mempool_put; + goto err; }else if (!sa) {snprintf (tbuf,sizeof (tbuf),"Failed to create ingress sa" ); rc = -EFAULT ; - goto mempool_put; + goto err; } inb_sa = (struct roc_ot_ipsec_inb_sa *)sa;
@@ -675,7 +669,7 @@ cn10k_eth_sec_session_create(void *device,"Inbound SA with SPI %u already in use" , ipsec->spi); rc = -EBUSY ; - goto mempool_put; + goto err; } inb_sa_dptr = (struct roc_ot_ipsec_inb_sa *)dev->inb.sa_dptr;
@@ -687,7 +681,7 @@ cn10k_eth_sec_session_create(void *device,if (rc) {snprintf (tbuf,sizeof (tbuf),"Failed to init inbound sa, rc=%d" , rc); - goto mempool_put; + goto err; } inb_priv =roc_nix_inl_ot_ipsec_inb_sa_sw_rsvd (inb_sa);
@@ -724,7 +718,7 @@ cn10k_eth_sec_session_create(void *device, eth_sec->inb, sizeof(struct roc_ot_ipsec_inb_sa));if (rc) - goto mempool_put; + goto err;if (conf->ipsec.options.ip_reassembly_en) { inb_priv->reass_dynfield_off = dev->reass_dynfield_off;
@@ -745,7 +739,7 @@ cn10k_eth_sec_session_create(void *device,/* Alloc an sa index */ rc =cnxk_eth_outb_sa_idx_get (dev, &sa_idx, ipsec->spi);if (rc) - goto mempool_put; + goto err; outb_sa =roc_nix_inl_ot_ipsec_outb_sa (sa_base, sa_idx); outb_priv =roc_nix_inl_ot_ipsec_outb_sa_sw_rsvd (outb_sa);
@@ -760,7 +754,7 @@ cn10k_eth_sec_session_create(void *device,snprintf (tbuf,sizeof (tbuf),"Failed to init outbound sa, rc=%d" , rc); rc |=cnxk_eth_outb_sa_idx_put (dev, sa_idx); - goto mempool_put; + goto err; }if (conf->ipsec.options.iv_gen_disable ==1 ) {
@@ -776,7 +770,7 @@ cn10k_eth_sec_session_create(void *device,"Failed to init outb sa misc params, rc=%d" , rc); rc |=cnxk_eth_outb_sa_idx_put (dev, sa_idx); - goto mempool_put; + goto err; }/* Save userdata */
@@ -818,7 +812,7 @@ cn10k_eth_sec_session_create(void *device, eth_sec->inb, sizeof(struct roc_ot_ipsec_outb_sa));if (rc) - goto mempool_put; + goto err; }if (inbound && inl_dev)roc_nix_inl_dev_unlock ();
@@ -830,15 +824,14 @@ cn10k_eth_sec_session_create(void *device,/* * Update fast path info in priv area. */ -set_sec_session_private_data (sess, (void *)sess_priv.u64); + sess->fast_mdata = sess_priv.u64;return 0 ; -mempool_put: +err:if (inbound && inl_dev)roc_nix_inl_dev_unlock ();rte_spinlock_unlock (lock); -rte_mempool_put (mempool, eth_sec);if (rc)plt_err ("%s" , tbuf);return rc;
@@ -850,7 +843,6 @@ cn10k_eth_sec_session_destroy(void *device, struct rte_security_session *sess)struct rte_eth_dev *eth_dev = (struct rte_eth_dev *)device;struct cnxk_eth_dev *dev =cnxk_eth_pmd_priv (eth_dev);struct cnxk_eth_sec_sess *eth_sec; -struct rte_mempool *mp;rte_spinlock_t *lock;void *sa_dptr;
@@ -896,10 +888,6 @@ cn10k_eth_sec_session_destroy(void *device, struct rte_security_session *sess) eth_sec->inb ?"inbound" :"outbound" , eth_sec->spi, eth_sec->sa_idx, eth_sec->inl_dev); -/* Put eth_sec object back to pool */ - mp =rte_mempool_from_obj (eth_sec); -set_sec_session_private_data (sess,NULL ); -rte_mempool_put (mp, eth_sec);return 0 ; }
drivers/net/cnxk/cn9k_ethdev_sec.c +10 -22
@@ -137,15 +137,14 @@ ar_window_init(struct cn9k_inb_priv_data *inb_priv)static int cn9k_eth_sec_session_create (void *device,struct rte_security_session_conf *conf, -struct rte_security_session *sess, -struct rte_mempool *mempool) +struct rte_security_session *sess) { +struct cnxk_eth_sec_sess *eth_sec =GET_SECURITY_SESS_PRIV (sess);struct rte_eth_dev *eth_dev = (struct rte_eth_dev *)device;struct cnxk_eth_dev *dev =cnxk_eth_pmd_priv (eth_dev);struct rte_security_ipsec_xform *ipsec;struct cn9k_sec_sess_priv sess_priv;struct rte_crypto_sym_xform *crypto; -struct cnxk_eth_sec_sess *eth_sec;struct roc_nix *nix = &dev->nix;rte_spinlock_t *lock;char tbuf[128 ] = {0 };
@@ -172,11 +171,6 @@ cn9k_eth_sec_session_create(void *device,return -EEXIST ; } -if (rte_mempool_get (mempool, (void **)ð_sec)) { -plt_err ("Could not allocate security session private data" ); -return -ENOMEM ; - } - lock = inbound ? &dev->inb.lock : &dev->outb.lock;rte_spinlock_lock (lock);
@@ -202,7 +196,7 @@ cn9k_eth_sec_session_create(void *device,snprintf (tbuf,sizeof (tbuf),"Failed to create ingress sa" ); rc = -EFAULT ; - goto mempool_put; + goto err; }/* Check if SA is already in use */
@@ -211,7 +205,7 @@ cn9k_eth_sec_session_create(void *device,"Inbound SA with SPI %u already in use" , ipsec->spi); rc = -EBUSY ; - goto mempool_put; + goto err; }memset (inb_sa,0 ,sizeof (struct roc_onf_ipsec_inb_sa ));
@@ -221,7 +215,7 @@ cn9k_eth_sec_session_create(void *device,if (rc) {snprintf (tbuf,sizeof (tbuf),"Failed to init inbound sa, rc=%d" , rc); - goto mempool_put; + goto err; } inb_priv =roc_nix_inl_onf_ipsec_inb_sa_sw_rsvd (inb_sa);
@@ -235,7 +229,7 @@ cn9k_eth_sec_session_create(void *device,if (inb_priv->replay_win_sz) { rc =ar_window_init (inb_priv);if (rc) - goto mempool_put; + goto err; }/* Prepare session priv */
@@ -264,7 +258,7 @@ cn9k_eth_sec_session_create(void *device,/* Alloc an sa index */ rc =cnxk_eth_outb_sa_idx_get (dev, &sa_idx,0 );if (rc) - goto mempool_put; + goto err; outb_sa =roc_nix_inl_onf_ipsec_outb_sa (sa_base, sa_idx); outb_priv =roc_nix_inl_onf_ipsec_outb_sa_sw_rsvd (outb_sa);
@@ -278,7 +272,7 @@ cn9k_eth_sec_session_create(void *device,snprintf (tbuf,sizeof (tbuf),"Failed to init outbound sa, rc=%d" , rc); rc |=cnxk_eth_outb_sa_idx_put (dev, sa_idx); - goto mempool_put; + goto err; }/* Save userdata */
@@ -321,12 +315,11 @@ cn9k_eth_sec_session_create(void *device,/* * Update fast path info in priv area. */ -set_sec_session_private_data (sess, (void *)sess_priv.u64); + sess->fast_mdata = sess_priv.u64;return 0 ; -mempool_put: +err:rte_spinlock_unlock (lock); -rte_mempool_put (mempool, eth_sec);if (rc)plt_err ("%s" , tbuf);return rc;
@@ -340,7 +333,6 @@ cn9k_eth_sec_session_destroy(void *device, struct rte_security_session *sess)struct roc_onf_ipsec_outb_sa *outb_sa;struct roc_onf_ipsec_inb_sa *inb_sa;struct cnxk_eth_sec_sess *eth_sec; -struct rte_mempool *mp;rte_spinlock_t *lock; eth_sec =cnxk_eth_sec_sess_get_by_sess (dev, sess);
@@ -377,10 +369,6 @@ cn9k_eth_sec_session_destroy(void *device, struct rte_security_session *sess) eth_sec->inb ?"inbound" :"outbound" , eth_sec->spi, eth_sec->sa_idx); -/* Put eth_sec object back to pool */ - mp =rte_mempool_from_obj (eth_sec); -set_sec_session_private_data (sess,NULL ); -rte_mempool_put (mp, eth_sec);return 0 ; }
drivers/net/iavf/iavf_ipsec_crypto.c +6 -17
@@ -640,14 +640,13 @@ set_session_parameter(struct iavf_security_ctx *iavf_sctx,static int iavf_ipsec_crypto_session_create (void *device,struct rte_security_session_conf *conf, -struct rte_security_session *session, -struct rte_mempool *mempool) +struct rte_security_session *session) {struct rte_eth_dev *ethdev = device;struct iavf_adapter *adapter =IAVF_DEV_PRIVATE_TO_ADAPTER (ethdev->data->dev_private);struct iavf_security_ctx *iavf_sctx = adapter->security_ctx; -struct iavf_security_session *iavf_session =NULL ; +struct iavf_security_session *iavf_session =GET_SECURITY_SESS_PRIV (session);int sa_idx;int ret =0 ;
@@ -656,12 +655,6 @@ iavf_ipsec_crypto_session_create(void *device,if (ret)return ret; -/* allocate session context */ -if (rte_mempool_get (mempool, (void **)&iavf_session)) { -PMD_DRV_LOG (ERR ,"Cannot get object from sess mempool" ); -return -ENOMEM ; - } -/* add SA to hardware database */ sa_idx =iavf_ipsec_crypto_security_association_add (adapter, conf);if (sa_idx <0 ) {
@@ -675,16 +668,12 @@ iavf_ipsec_crypto_session_create(void *device,RTE_SECURITY_IPSEC_SA_DIR_INGRESS ? "inbound" : "outbound"); -rte_mempool_put (mempool, iavf_session);return -EFAULT ; }/* save data plane required session parameters */ set_session_parameter (iavf_sctx, iavf_session, conf, sa_idx); -/* save to security session private data */ -set_sec_session_private_data (session, iavf_session); -return 0 ; }
@@ -702,7 +691,7 @@ iavf_ipsec_crypto_action_valid(struct rte_eth_dev *ethdev, {struct iavf_adapter *adapter =IAVF_DEV_PRIVATE_TO_ADAPTER (ethdev->data->dev_private); -struct iavf_security_session *sess = session->sess_private_data; +const struct iavf_security_session *sess = (const void *)session->driver_priv_data;/* verify we have a valid session and that it belong to this adapter */ if (unlikely (sess ==NULL || sess->adapter != adapter))
@@ -880,7 +869,7 @@ iavf_ipsec_crypto_session_update(void *device,int rc =0 ; adapter =IAVF_DEV_PRIVATE_TO_ADAPTER (eth_dev->data->dev_private); - iavf_sess = (struct iavf_security_session *)session->sess_private_data; + iavf_sess =GET_SECURITY_SESS_PRIV (session);/* verify we have a valid session and that it belong to this adapter */ if (unlikely (iavf_sess ==NULL || iavf_sess->adapter != adapter))
@@ -1045,7 +1034,7 @@ iavf_ipsec_crypto_session_destroy(void *device,int ret; adapter =IAVF_DEV_PRIVATE_TO_ADAPTER (eth_dev->data->dev_private); - iavf_sess = (struct iavf_security_session *)session->sess_private_data; + iavf_sess =GET_SECURITY_SESS_PRIV (session);/* verify we have a valid session and that it belong to this adapter */ if (unlikely (iavf_sess ==NULL || iavf_sess->adapter != adapter))
@@ -1140,7 +1129,7 @@ iavf_ipsec_crypto_pkt_metadata_set(void *device,struct iavf_adapter *adapter =IAVF_DEV_PRIVATE_TO_ADAPTER (ethdev->data->dev_private);struct iavf_security_ctx *iavf_sctx = adapter->security_ctx; -struct iavf_security_session *iavf_sess = session->sess_private_data; +struct iavf_security_session *iavf_sess =GET_SECURITY_SESS_PRIV (session);struct iavf_ipsec_crypto_pkt_metadata *md;struct rte_esp_tail *esp_tail;uint64_t *sqn = params;
drivers/net/ixgbe/ixgbe_ipsec.c +10 -21
@@ -369,24 +369,17 @@ ixgbe_crypto_remove_sa(struct rte_eth_dev *dev,static int ixgbe_crypto_create_session (void *device,struct rte_security_session_conf *conf, -struct rte_security_session *session, -struct rte_mempool *mempool) +struct rte_security_session *session) {struct rte_eth_dev *eth_dev = (struct rte_eth_dev *)device; -struct ixgbe_crypto_session *ic_session =NULL ; +struct ixgbe_crypto_session *ic_session =GET_SECURITY_SESS_PRIV (session);struct rte_crypto_aead_xform *aead_xform;struct rte_eth_conf *dev_conf = ð_dev->data->dev_conf; -if (rte_mempool_get (mempool, (void **)&ic_session)) { -PMD_DRV_LOG (ERR ,"Cannot get object from ic_session mempool" ); -return -ENOMEM ; - } -if (conf->crypto_xform->type !=RTE_CRYPTO_SYM_XFORM_AEAD || conf->crypto_xform->aead.algo !=RTE_CRYPTO_AEAD_AES_GCM ) {PMD_DRV_LOG (ERR ,"Unsupported crypto transformation mode\n" ); -rte_mempool_put (mempool, (void *)ic_session);return -ENOTSUP ; } aead_xform = &conf->crypto_xform->aead;
@@ -396,7 +389,6 @@ ixgbe_crypto_create_session(void *device, ic_session->op =IXGBE_OP_AUTHENTICATED_DECRYPTION ; }else {PMD_DRV_LOG (ERR ,"IPsec decryption not enabled\n" ); -rte_mempool_put (mempool, (void *)ic_session);return -ENOTSUP ; } }else {
@@ -404,7 +396,6 @@ ixgbe_crypto_create_session(void *device, ic_session->op =IXGBE_OP_AUTHENTICATED_ENCRYPTION ; }else {PMD_DRV_LOG (ERR ,"IPsec encryption not enabled\n" ); -rte_mempool_put (mempool, (void *)ic_session);return -ENOTSUP ; } }
@@ -416,12 +407,9 @@ ixgbe_crypto_create_session(void *device, ic_session->spi = conf->ipsec.spi; ic_session->dev = eth_dev; -set_sec_session_private_data (session, ic_session); -if (ic_session->op ==IXGBE_OP_AUTHENTICATED_ENCRYPTION ) {if (ixgbe_crypto_add_sa (ic_session)) {PMD_DRV_LOG (ERR ,"Failed to add SA\n" ); -rte_mempool_put (mempool, (void *)ic_session);return -EPERM ; } }
@@ -440,9 +428,7 @@ ixgbe_crypto_remove_session(void *device, struct rte_security_session *session) {struct rte_eth_dev *eth_dev = device; -struct ixgbe_crypto_session *ic_session = - (struct ixgbe_crypto_session *) -get_sec_session_private_data (session); +struct ixgbe_crypto_session *ic_session =GET_SECURITY_SESS_PRIV (session);struct rte_mempool *mempool =rte_mempool_from_obj (ic_session);if (eth_dev != ic_session->dev) {
@@ -480,8 +466,7 @@ ixgbe_crypto_update_mb(void *device __rte_unused,struct rte_security_session *session, struct rte_mbuf *m, void *params __rte_unused) { - struct ixgbe_crypto_session *ic_session = -get_sec_session_private_data (session); + struct ixgbe_crypto_session *ic_session = GET_SECURITY_SESS_PRIV(session);if (ic_session->op ==IXGBE_OP_AUTHENTICATED_ENCRYPTION ) {union ixgbe_crypto_tx_desc_md *mdata = (union ixgbe_crypto_tx_desc_md *)
@@ -685,8 +670,12 @@ ixgbe_crypto_add_ingress_sa_from_flow(const void *sess, const void *ip_spec, uint8_t is_ipv6) { -struct ixgbe_crypto_session *ic_session - =get_sec_session_private_data (sess); +/** +* FIXME Updating the session priv data when the session is const. +* Typecasting done here is wrong and the implementation need to be corrected. +*/ +struct ixgbe_crypto_session *ic_session = (void *)(uint64_t ) + ((const struct rte_security_session *)sess)->driver_priv_data;if (ic_session->op ==IXGBE_OP_AUTHENTICATED_DECRYPTION ) {if (is_ipv6) {
drivers/net/txgbe/txgbe_ipsec.c +11 -21
@@ -349,24 +349,17 @@ txgbe_crypto_remove_sa(struct rte_eth_dev *dev,static int txgbe_crypto_create_session (void *device,struct rte_security_session_conf *conf, -struct rte_security_session *session, -struct rte_mempool *mempool) +struct rte_security_session *session) {struct rte_eth_dev *eth_dev = (struct rte_eth_dev *)device; -struct txgbe_crypto_session *ic_session =NULL ; +struct txgbe_crypto_session *ic_session =GET_SECURITY_SESS_PRIV (session);struct rte_crypto_aead_xform *aead_xform;struct rte_eth_conf *dev_conf = ð_dev->data->dev_conf; -if (rte_mempool_get (mempool, (void **)&ic_session)) { -PMD_DRV_LOG (ERR ,"Cannot get object from ic_session mempool" ); -return -ENOMEM ; - } -if (conf->crypto_xform->type !=RTE_CRYPTO_SYM_XFORM_AEAD || conf->crypto_xform->aead.algo !=RTE_CRYPTO_AEAD_AES_GCM ) {PMD_DRV_LOG (ERR ,"Unsupported crypto transformation mode\n" ); -rte_mempool_put (mempool, (void *)ic_session);return -ENOTSUP ; } aead_xform = &conf->crypto_xform->aead;
@@ -376,7 +369,6 @@ txgbe_crypto_create_session(void *device, ic_session->op =TXGBE_OP_AUTHENTICATED_DECRYPTION ; }else {PMD_DRV_LOG (ERR ,"IPsec decryption not enabled\n" ); -rte_mempool_put (mempool, (void *)ic_session);return -ENOTSUP ; } }else {
@@ -384,7 +376,6 @@ txgbe_crypto_create_session(void *device, ic_session->op =TXGBE_OP_AUTHENTICATED_ENCRYPTION ; }else {PMD_DRV_LOG (ERR ,"IPsec encryption not enabled\n" ); -rte_mempool_put (mempool, (void *)ic_session);return -ENOTSUP ; } }
@@ -396,12 +387,9 @@ txgbe_crypto_create_session(void *device, ic_session->spi = conf->ipsec.spi; ic_session->dev = eth_dev; -set_sec_session_private_data (session, ic_session); -if (ic_session->op ==TXGBE_OP_AUTHENTICATED_ENCRYPTION ) {if (txgbe_crypto_add_sa (ic_session)) {PMD_DRV_LOG (ERR ,"Failed to add SA\n" ); -rte_mempool_put (mempool, (void *)ic_session);return -EPERM ; } }
@@ -420,9 +408,7 @@ txgbe_crypto_remove_session(void *device, struct rte_security_session *session) {struct rte_eth_dev *eth_dev = device; -struct txgbe_crypto_session *ic_session = - (struct txgbe_crypto_session *) -get_sec_session_private_data (session); +struct txgbe_crypto_session *ic_session =GET_SECURITY_SESS_PRIV (session);struct rte_mempool *mempool =rte_mempool_from_obj (ic_session);if (eth_dev != ic_session->dev) {
@@ -460,8 +446,8 @@ txgbe_crypto_update_mb(void *device __rte_unused,struct rte_security_session *session, struct rte_mbuf *m, void *params __rte_unused) { - struct txgbe_crypto_session *ic_session = -get_sec_session_private_data (session); + struct txgbe_crypto_session *ic_session = GET_SECURITY_SESS_PRIV(session); +if (ic_session->op ==TXGBE_OP_AUTHENTICATED_ENCRYPTION ) {union txgbe_crypto_tx_desc_md *mdata = (union txgbe_crypto_tx_desc_md *)
@@ -661,8 +647,12 @@ txgbe_crypto_add_ingress_sa_from_flow(const void *sess, const void *ip_spec, uint8_t is_ipv6) { -struct txgbe_crypto_session *ic_session = -get_sec_session_private_data (sess); +/** +* FIXME Updating the session priv data when the session is const. +* Typecasting done here is wrong and the implementation need to be corrected. +*/ +struct txgbe_crypto_session *ic_session = (void *)(uint64_t ) + ((const struct rte_security_session *)sess)->driver_priv_data;if (ic_session->op ==TXGBE_OP_AUTHENTICATED_DECRYPTION ) {if (is_ipv6) {
examples/ipsec-secgw/ipsec-secgw.c +0 -34
@@ -2110,38 +2110,6 @@ session_pool_init(struct socket_ctx *ctx, int32_t socket_id, size_t sess_sz)printf ("Allocated session pool on socket %d\n" , socket_id); } -static void -session_priv_pool_init (struct socket_ctx *ctx,int32_t socket_id, -size_t sess_sz) -{ -char mp_name[RTE_MEMPOOL_NAMESIZE ]; -struct rte_mempool *sess_mp; -uint32_t nb_sess; - -snprintf (mp_name,RTE_MEMPOOL_NAMESIZE , -"sess_mp_priv_%u" , socket_id); - nb_sess = (get_nb_crypto_sessions () +CDEV_MP_CACHE_SZ * -rte_lcore_count ()); - nb_sess =RTE_MAX (nb_sess,CDEV_MP_CACHE_SZ * -CDEV_MP_CACHE_MULTIPLIER ); - sess_mp =rte_mempool_create (mp_name, - nb_sess, - sess_sz, -CDEV_MP_CACHE_SZ , -0 ,NULL ,NULL ,NULL , -NULL , socket_id, -0 ); - ctx->session_priv_pool = sess_mp; - -if (ctx->session_priv_pool ==NULL ) -rte_exit (EXIT_FAILURE , -"Cannot init session priv pool on socket %d\n" , - socket_id); -else -printf ("Allocated session priv pool on socket %d\n" , - socket_id); -} -static void pool_init (struct socket_ctx *ctx,int32_t socket_id,int portid,uint32_t nb_mbuf)
@@ -3022,8 +2990,6 @@ main(int32_t argc, char **argv)continue ;session_pool_init (&socket_ctx[socket_id], socket_id, sess_sz); -session_priv_pool_init (&socket_ctx[socket_id], socket_id, - sess_sz); }printf ("Number of mbufs in packet pool %d\n" , nb_bufs_in_pool);
examples/ipsec-secgw/ipsec.c +3 -6
@@ -151,8 +151,7 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx_lcore[],set_ipsec_conf (sa, &(sess_conf.ipsec)); ips->security.ses =rte_security_session_create (ctx, - &sess_conf, skt_ctx->session_pool, - skt_ctx->session_priv_pool); + &sess_conf, skt_ctx->session_pool);if (ips->security.ses ==NULL ) {RTE_LOG (ERR ,IPSEC ,"SEC Session init failed: err: %d\n" , ret);
@@ -278,8 +277,7 @@ create_inline_session(struct socket_ctx *skt_ctx, struct ipsec_sa *sa, } ips->security.ses =rte_security_session_create (sec_ctx, - &sess_conf, skt_ctx->session_pool, - skt_ctx->session_priv_pool); + &sess_conf, skt_ctx->session_pool);if (ips->security.ses ==NULL ) {RTE_LOG (ERR ,IPSEC ,"SEC Session init failed: err: %d\n" , ret);
@@ -478,8 +476,7 @@ create_inline_session(struct socket_ctx *skt_ctx, struct ipsec_sa *sa, sess_conf.userdata = (void *) sa; ips->security.ses =rte_security_session_create (sec_ctx, - &sess_conf, skt_ctx->session_pool, - skt_ctx->session_priv_pool); + &sess_conf, skt_ctx->session_pool);if (ips->security.ses ==NULL ) {RTE_LOG (ERR ,IPSEC ,"SEC Session init failed: err: %d\n" , ret);
examples/ipsec-secgw/ipsec.h +0 -1
@@ -258,7 +258,6 @@ struct socket_ctx {struct rte_mempool *mbuf_pool[RTE_MAX_ETHPORTS ];struct rte_mempool *mbuf_pool_indir;struct rte_mempool *session_pool; -struct rte_mempool *session_priv_pool; };struct cnt_blk {
lib/cryptodev/rte_cryptodev.h +1 -1
@@ -967,7 +967,7 @@ rte_cryptodev_asym_session_pool_create(const char *name, uint32_t nb_elts,* @param dev_id ID of device that we want the session to be used on * @param xforms Symmetric crypto transform operations to apply on flow * processed with this session -* @param mempool Mempool where the private data is allocated. +* @param mp Mempool where the private data is allocated. * * @return * - On success return pointer to sym-session.
lib/security/rte_security.c +14 -6
@@ -42,21 +42,28 @@ rte_security_dynfield_register(void)struct rte_security_session *rte_security_session_create (struct rte_security_ctx *instance,struct rte_security_session_conf *conf, -struct rte_mempool *mp, -struct rte_mempool *priv_mp) +struct rte_mempool *mp) {struct rte_security_session *sess =NULL ; +uint32_t sess_priv_size;RTE_PTR_CHAIN3_OR_ERR_RET (instance, ops, session_create,NULL ,NULL );RTE_PTR_OR_ERR_RET (conf,NULL );RTE_PTR_OR_ERR_RET (mp,NULL ); -RTE_PTR_OR_ERR_RET (priv_mp,NULL ); + + sess_priv_size = instance->ops->session_get_size (instance->device); +if (mp->elt_size < (sizeof (struct rte_security_session ) + sess_priv_size)) +return NULL ;if (rte_mempool_get (mp, (void **)&sess))return NULL ; -if (instance->ops->session_create (instance->device, conf, - sess, priv_mp)) { +/* Clear session priv data */ +memset (sess->driver_priv_data,0 , sess_priv_size); + + sess->driver_priv_data_iova =rte_mempool_virt2iova (sess) + + offsetof(struct rte_security_session , driver_priv_data); +if (instance->ops->session_create (instance->device, conf, sess)) {rte_mempool_put (mp, (void *)sess);return NULL ; }
@@ -83,7 +90,8 @@ rte_security_session_get_size(struct rte_security_ctx *instance) {RTE_PTR_CHAIN3_OR_ERR_RET (instance, ops, session_get_size,0 ,0 ); -return instance->ops->session_get_size (instance->device); +return (sizeof (struct rte_security_session ) + + instance->ops->session_get_size (instance->device)); }int
lib/security/rte_security.h +11 -19
@@ -519,10 +519,17 @@ struct rte_security_session_conf { };struct rte_security_session { -void *sess_private_data; -/**< Private session material */ +RTE_MARKER cacheline0;uint64_t opaque_data;/**< Opaque user defined data */ +uint64_t fast_mdata; +/**< Fast metadata to be used for inline path */ +rte_iova_t driver_priv_data_iova; +/**< session private data IOVA address */ + + RTE_MARKER cacheline1 __rte_cache_min_aligned; +uint8_t driver_priv_data[0 ]; +/**< Private session material, variable size (depends on driver) */ };/**
@@ -531,7 +538,6 @@ struct rte_security_session {* @param instance security instance * @param conf session configuration parameters * @param mp mempool to allocate session objects from -* @param priv_mp mempool to allocate session private data objects from * @return * - On success, pointer to session * - On failure, NULL
@@ -539,8 +545,7 @@ struct rte_security_session {struct rte_security_session *rte_security_session_create (struct rte_security_ctx *instance,struct rte_security_session_conf *conf, -struct rte_mempool *mp, -struct rte_mempool *priv_mp); +struct rte_mempool *mp);/** * Update security session as specified by the session configuration
@@ -656,7 +661,7 @@ rte_security_set_pkt_metadata(struct rte_security_ctx *instance,/* Fast Path */ if (instance->flags &RTE_SEC_CTX_F_FAST_SET_MDATA ) { *rte_security_dynfield (mb) = - (rte_security_dynfield_t)(sess->sess_private_data); + (rte_security_dynfield_t)(sess->fast_mdata);return 0 ; }
@@ -713,19 +718,6 @@ __rte_security_attach_session(struct rte_crypto_sym_op *sym_op,return 0 ; } -static inline void * -get_sec_session_private_data (const struct rte_security_session *sess) -{ -return sess->sess_private_data; -} - -static inline void -set_sec_session_private_data (struct rte_security_session *sess, -void *private_data) -{ - sess->sess_private_data = private_data; -} -/** * Attach a session to a crypto operation. * This API is needed only in case of RTE_SECURITY_SESS_CRYPTO_PROTO_OFFLOAD
lib/security/rte_security_driver.h +9 -4
@@ -19,24 +19,29 @@ extern "C" {#include "rte_security.h" +/** +* Helper macro to get driver private data +*/ +#define GET_SECURITY_SESS_PRIV (s) \ + (void *)(((struct rte_security_session *)s)->driver_priv_data) +#define GET_SECURITY_SESS_PRIV_IOVA (s) \ + (((struct rte_security_session *)s)->driver_priv_data_iova) +/** * Configure a security session on a device. * * @param device Crypto/eth device pointer * @param conf Security session configuration * @param sess Pointer to Security private session structure -* @param mp Mempool where the private session is allocated * * @return * - Returns 0 if private session structure have been created successfully. * - Returns -EINVAL if input parameters are invalid. * - Returns -ENOTSUP if crypto device does not support the crypto transform. -* - Returns -ENOMEM if the private session could not be allocated. */ typedef int (*security_session_create_t )(void *device,struct rte_security_session_conf *conf, -struct rte_security_session *sess, -struct rte_mempool *mp); +struct rte_security_session *sess);/** * Free driver private session data.
Signature · 1 line
2.25.1
PATCH v2 5/6 drivers/crypto: support security session get size op
In reply to Akhil GoyalAdded the support for rte_security_op.session_get_size() in all the PMDs which support rte_security sessions and the op was not supported.
--- drivers/crypto/caam_jr/caam_jr.c | 6 ++++++ drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 7 +++++++ drivers/crypto/dpaa_sec/dpaa_sec.c | 8 ++++++++ drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 7 +++++++ drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 7 +++++++ drivers/crypto/qat/dev/qat_sym_pmd_gen1.c | 1 + drivers/crypto/qat/qat_sym_session.c | 6 ++++++ drivers/crypto/qat/qat_sym_session.h | 2 ++ 8 files changed, 44 insertions(+)
drivers/crypto/caam_jr/caam_jr.c +6 -0
@@ -1937,6 +1937,11 @@ caam_jr_security_session_destroy(void *dev __rte_unused,return 0 ; } +static unsigned int +caam_jr_security_session_get_size(void *device __rte_unused) +{ +return sizeof (struct caam_jr_session ); +} static int caam_jr_dev_configure(struct rte_cryptodev *dev,
@@ -2031,6 +2036,7 @@ static struct rte_cryptodev_ops caam_jr_ops = {static struct rte_security_ops caam_jr_security_ops = { .session_create = caam_jr_security_session_create, .session_update =NULL , + .session_get_size = caam_jr_security_session_get_size, .session_stats_get =NULL , .session_destroy = caam_jr_security_session_destroy, .set_pkt_metadata =NULL ,
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +7 -0
@@ -3733,6 +3733,12 @@ dpaa2_sec_security_session_destroy(void *dev __rte_unused, }return 0 ; } + +static unsigned int +dpaa2_sec_security_session_get_size(void *device __rte_unused) +{ +return sizeof (dpaa2_sec_session); +}#endif static int dpaa2_sec_sym_session_configure(struct rte_cryptodev *dev __rte_unused,
@@ -4184,6 +4190,7 @@ dpaa2_sec_capabilities_get(void *device __rte_unused)static const struct rte_security_ops dpaa2_sec_security_ops = { .session_create = dpaa2_sec_security_session_create, .session_update =NULL , + .session_get_size = dpaa2_sec_security_session_get_size, .session_stats_get =NULL , .session_destroy = dpaa2_sec_security_session_destroy, .set_pkt_metadata =NULL ,
drivers/crypto/dpaa_sec/dpaa_sec.c +8 -0
@@ -3289,6 +3289,13 @@ dpaa_sec_security_session_destroy(void *dev __rte_unused, }return 0 ; } + +static unsigned int +dpaa_sec_security_session_get_size(void *device __rte_unused) +{ +return sizeof (dpaa_sec_session); +} +#endif static int dpaa_sec_dev_configure(struct rte_cryptodev *dev __rte_unused,
@@ -3547,6 +3554,7 @@ dpaa_sec_capabilities_get(void *device __rte_unused)static const struct rte_security_ops dpaa_sec_security_ops = { .session_create = dpaa_sec_security_session_create, .session_update =NULL , + .session_get_size = dpaa_sec_security_session_get_size, .session_stats_get =NULL , .session_destroy = dpaa_sec_security_session_destroy, .set_pkt_metadata =NULL ,
drivers/crypto/ipsec_mb/pmd_aesni_mb.c +7 -0
@@ -2141,6 +2141,12 @@ aesni_mb_pmd_sec_sess_destroy(void *dev __rte_unused,return 0 ; } +static unsigned int +aesni_mb_pmd_sec_sess_get_size(void *device __rte_unused) +{ +return sizeof (struct aesni_mb_session ); +} +/** Get security capabilities for aesni multi-buffer */ static const struct rte_security_capability * aesni_mb_pmd_sec_capa_get(void *device __rte_unused)
@@ -2151,6 +2157,7 @@ aesni_mb_pmd_sec_capa_get(void *device __rte_unused)static struct rte_security_ops aesni_mb_pmd_sec_ops = { .session_create = aesni_mb_pmd_sec_sess_create, .session_update =NULL , + .session_get_size = aesni_mb_pmd_sec_sess_get_size, .session_stats_get =NULL , .session_destroy = aesni_mb_pmd_sec_sess_destroy, .set_pkt_metadata =NULL ,
drivers/crypto/mvsam/rte_mrvl_pmd_ops.c +7 -0
@@ -907,6 +907,12 @@ mrvl_crypto_pmd_security_session_destroy(void *dev __rte_unused,return 0 ; } +static unsigned int +mrvl_crypto_pmd_security_session_create(void *device __rte_unused) +{ +return sizeof (struct mrvl_crypto_session ); +} +static const struct rte_security_capability mrvl_crypto_pmd_sec_security_cap[] = { {/* IPsec Lookaside Protocol offload ESP Tunnel Egress */
@@ -971,6 +977,7 @@ mrvl_crypto_pmd_security_capabilities_get(void *device __rte_unused)struct rte_security_ops mrvl_sec_security_pmd_ops = { .session_create = mrvl_crypto_pmd_security_session_create, .session_update =NULL , + .session_get_size = mrvl_crypto_pmd_security_session_get_size, .session_stats_get =NULL , .session_destroy = mrvl_crypto_pmd_security_session_destroy, .set_pkt_metadata =NULL ,
drivers/crypto/qat/dev/qat_sym_pmd_gen1.c +1 -0
@@ -428,6 +428,7 @@ qat_security_cap_get_gen1(void *dev __rte_unused)struct rte_security_ops security_qat_ops_gen1 = { .session_create = qat_security_session_create, .session_update =NULL , + .session_get_size = qat_security_session_get_size, .session_stats_get =NULL , .session_destroy = qat_security_session_destroy, .set_pkt_metadata =NULL ,
drivers/crypto/qat/qat_sym_session.c +6 -0
@@ -2641,4 +2641,10 @@ qat_security_session_destroy(void *dev __rte_unused,# endif return 0 ; } + +unsigned int +qat_security_session_get_size(void *device __rte_unused) +{ +return sizeof (struct qat_sym_session ); +}#endif
drivers/crypto/qat/qat_sym_session.h +2 -0
@@ -180,6 +180,8 @@ qat_security_session_create(void *dev, struct rte_security_session_conf *conf, struct rte_security_session *sess);int qat_security_session_destroy (void *dev,struct rte_security_session *sess); +unsigned int +qat_security_session_get_size(void *dev __rte_unused);#endif #endif /* _QAT_SYM_SESSION_H_ */
Signature · 1 line
2.25.1
RE: [PATCH v2 5/6] drivers/crypto: support security session get size op
Quoted 25 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Wednesday, September 21, 2022 4:03 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; De Lara Guarch, Pablo <[email protected]>; [email protected]; [email protected]; Yang, Qiming <[email protected]>; Wu, Wenjun1 <[email protected]>; [email protected]; Wu, Jingjing <[email protected]>; Xing, Beilei <[email protected]>; [email protected]; Akhil Goyal <[email protected]>
Subject: [PATCH v2 5/6]drivers/crypto: support security session get size opAdded the support for
rte_security_op.session_get_size()
in all the PMDs which supportrte_securitysessions and the op was not supported.Signed-off-by Akhil Goyal <[email protected]>---
PATCH v2 6/6 security: hide session structure
In reply to Akhil GoyalStructure rte_security_session is moved to internal headers which are not visible to applications.
The only field which should be used by app is opaque_data. This field can now be accessed via set/get APIs added in this patch.
Subsequent changes in app and lib are made to compile the code.
--- app/test-crypto-perf/cperf_ops.c | 6 +- .../cperf_test_pmd_cyclecount.c | 2 +- app/test-crypto-perf/cperf_test_throughput.c | 2 +- app/test/test_cryptodev.c | 2 +- app/test/test_cryptodev_security_ipsec.c | 2 +- app/test/test_cryptodev_security_ipsec.h | 2 +- app/test/test_security.c | 32 ++++---- app/test/test_security_inline_proto.c | 10 +-- drivers/crypto/caam_jr/caam_jr.c | 2 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 4 +- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 6 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 6 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 2 +- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 4 +- drivers/crypto/qat/qat_sym.c | 4 +- drivers/crypto/qat/qat_sym.h | 4 +- drivers/net/iavf/iavf_ipsec_crypto.h | 2 +- examples/ipsec-secgw/ipsec_worker.c | 2 +- lib/cryptodev/rte_crypto_sym.h | 4 +- lib/ipsec/rte_ipsec_group.h | 12 +-- lib/ipsec/ses.c | 2 +- lib/security/rte_security.c | 13 ++- lib/security/rte_security.h | 80 ++++++++++++------- lib/security/rte_security_driver.h | 18 +++++ 24 files changed, 131 insertions(+), 92 deletions(-)
app/test-crypto-perf/cperf_ops.c +2 -4
@@ -65,8 +65,7 @@ cperf_set_ops_security(struct rte_crypto_op **ops,for (i =0 ; i < nb_ops; i++) {struct rte_crypto_sym_op *sym_op = ops[i]->sym; -struct rte_security_session *sec_sess = - (struct rte_security_session *)sess; +void *sec_sess = (void *)sess;uint32_t buf_sz; uint32_t *per_pkt_hfn = rte_crypto_op_ctod_offset(ops[i],
@@ -131,8 +130,7 @@ cperf_set_ops_security_ipsec(struct rte_crypto_op **ops, uint16_t iv_offset __rte_unused, uint32_t *imix_idx, uint64_t *tsc_start) { -struct rte_security_session *sec_sess = - (struct rte_security_session *)sess; +void *sec_sess = sess;const uint32_t test_buffer_size = options->test_buffer_size;const uint32_t headroom_sz = options->headroom_sz;const uint32_t segment_sz = options->segment_sz;
app/test-crypto-perf/cperf_test_pmd_cyclecount.c +1 -1
@@ -70,7 +70,7 @@ cperf_pmd_cyclecount_test_free(struct cperf_pmd_cyclecount_ctx *ctx) (struct rte_security_ctx *)rte_cryptodev_get_sec_ctx (ctx->dev_id);rte_security_session_destroy (sec_ctx, - (struct rte_security_session *)ctx->sess); + (void *)ctx->sess); } else#endif rte_cryptodev_sym_session_free (ctx->dev_id, ctx->sess);
app/test-crypto-perf/cperf_test_throughput.c +1 -1
@@ -47,7 +47,7 @@ cperf_throughput_test_free(struct cperf_throughput_ctx *ctx)rte_cryptodev_get_sec_ctx (ctx->dev_id);rte_security_session_destroy ( sec_ctx, - (struct rte_security_session *)ctx->sess); + (void *)ctx->sess); }#endif else
app/test/test_cryptodev.c +1 -1
@@ -84,7 +84,7 @@ struct crypto_unittest_params {union {struct rte_cryptodev_sym_session *sess;#ifdef RTE_LIB_SECURITY -struct rte_security_session *sec_session; +void *sec_session;#endif };#ifdef RTE_LIB_SECURITY
app/test/test_cryptodev_security_ipsec.c +1 -1
@@ -1126,7 +1126,7 @@ test_ipsec_status_check(const struct ipsec_test_data *td,int test_ipsec_stats_verify (struct rte_security_ctx *ctx, -struct rte_security_session *sess, +void *sess,const struct ipsec_test_flags *flags,enum rte_security_ipsec_sa_direction dir) {
app/test/test_cryptodev_security_ipsec.h +1 -1
@@ -277,7 +277,7 @@ int test_ipsec_status_check(const struct ipsec_test_data *td, int pkt_num);int test_ipsec_stats_verify (struct rte_security_ctx *ctx, -struct rte_security_session *sess, +void *sess,const struct ipsec_test_flags *flags,enum rte_security_ipsec_sa_direction dir);
app/test/test_security.c +16 -16
@@ -234,7 +234,7 @@ static struct mock_session_create_data { void *device; struct rte_security_session_conf *conf; - struct rte_security_session *sess; + void *sess; struct rte_mempool *mp; struct rte_mempool *priv_mp;
@@ -268,7 +268,7 @@ mock_session_create(void *device,*/ static struct mock_session_update_data { void *device; - struct rte_security_session *sess; + void *sess; struct rte_security_session_conf *conf; int ret;
@@ -322,7 +322,7 @@ mock_session_get_size(void *device)*/ static struct mock_session_stats_get_data { void *device; - struct rte_security_session *sess; + void *sess; struct rte_security_stats *stats; int ret;
@@ -352,7 +352,7 @@ mock_session_stats_get(void *device,*/ static struct mock_session_destroy_data { void *device; - struct rte_security_session *sess; + void *sess; int ret;
@@ -377,7 +377,7 @@ mock_session_destroy(void *device, struct rte_security_session *sess)*/ static struct mock_set_pkt_metadata_data { void *device; - struct rte_security_session *sess; + void *sess; struct rte_mbuf *m; void *params;
@@ -513,7 +513,7 @@ static struct security_testsuite_params {static struct security_unittest_params {struct rte_security_ctx ctx;struct rte_security_session_conf conf; -struct rte_security_session *sess; +void *sess; } unittest_params = { .ctx = { .device =NULL ,
@@ -650,7 +650,7 @@ ut_setup_with_session(void) {struct security_unittest_params *ut_params = &unittest_params;struct security_testsuite_params *ts_params = &testsuite_params; -struct rte_security_session *sess; +void *sess;int ret =ut_setup ();if (ret !=TEST_SUCCESS )
@@ -701,7 +701,7 @@ test_session_create_inv_context(void) {struct security_testsuite_params *ts_params = &testsuite_params;struct security_unittest_params *ut_params = &unittest_params; -struct rte_security_session *sess; +void *sess; sess =rte_security_session_create (NULL , &ut_params->conf, ts_params->session_mpool);
@@ -723,7 +723,7 @@ test_session_create_inv_context_ops(void) {struct security_testsuite_params *ts_params = &testsuite_params;struct security_unittest_params *ut_params = &unittest_params; -struct rte_security_session *sess; +void *sess; ut_params->ctx.ops =NULL ;
@@ -747,7 +747,7 @@ test_session_create_inv_context_ops_fun(void) {struct security_testsuite_params *ts_params = &testsuite_params;struct security_unittest_params *ut_params = &unittest_params; -struct rte_security_session *sess; +void *sess; ut_params->ctx.ops = &empty_ops;
@@ -770,7 +770,7 @@ test_session_create_inv_configuration(void) {struct security_testsuite_params *ts_params = &testsuite_params;struct security_unittest_params *ut_params = &unittest_params; -struct rte_security_session *sess; +void *sess; sess =rte_security_session_create (&ut_params->ctx,NULL , ts_params->session_mpool);
@@ -791,7 +791,7 @@ static inttest_session_create_inv_mempool (void) {struct security_unittest_params *ut_params = &unittest_params; -struct rte_security_session *sess; +void *sess; sess =rte_security_session_create (&ut_params->ctx, &ut_params->conf,NULL ); TEST_ASSERT_MOCK_FUNCTION_CALL_RET(rte_security_session_create,
@@ -812,8 +812,8 @@ test_session_create_mempool_empty(void) {struct security_testsuite_params *ts_params = &testsuite_params;struct security_unittest_params *ut_params = &unittest_params; -struct rte_security_session *tmp[SECURITY_TEST_MEMPOOL_SIZE ]; -struct rte_security_session *sess; +void *tmp[SECURITY_TEST_MEMPOOL_SIZE ]; +void *sess;/* Get all available objects from mempool. */ int i, ret;
@@ -853,7 +853,7 @@ test_session_create_ops_failure(void) {struct security_testsuite_params *ts_params = &testsuite_params;struct security_unittest_params *ut_params = &unittest_params; -struct rte_security_session *sess; +void *sess; mock_session_create_exp.device =NULL ; mock_session_create_exp.conf = &ut_params->conf;
@@ -879,7 +879,7 @@ test_session_create_success(void) {struct security_testsuite_params *ts_params = &testsuite_params;struct security_unittest_params *ut_params = &unittest_params; -struct rte_security_session *sess; +void *sess; mock_session_create_exp.device =NULL ; mock_session_create_exp.conf = &ut_params->conf;
app/test/test_security_inline_proto.c +5 -5
@@ -119,7 +119,7 @@ static struct rte_flow *default_flow[RTE_MAX_ETHPORTS];/* Create Inline IPsec session */ static int create_inline_ipsec_session (struct ipsec_test_data *sa,uint16_t portid, -struct rte_security_session **sess,struct rte_security_ctx **ctx, +void **sess,struct rte_security_ctx **ctx,uint32_t *ol_flags,const struct ipsec_test_flags *flags,struct rte_security_session_conf *sess_conf) {
@@ -681,8 +681,8 @@ static int test_ipsec_with_reassembly(struct reassembly_vector *vector, const struct ipsec_test_flags *flags) { - struct rte_security_session *out_ses[ENCAP_DECAP_BURST_SZ] = {0}; -struct rte_security_session *in_ses[ENCAP_DECAP_BURST_SZ ] = {0 }; + void *out_ses[ENCAP_DECAP_BURST_SZ] = {0}; +void *in_ses[ENCAP_DECAP_BURST_SZ ] = {0 };struct rte_eth_ip_reassembly_params reass_capa = {0 };struct rte_security_session_conf sess_conf_out = {0 };struct rte_security_session_conf sess_conf_in = {0 };
@@ -944,7 +944,7 @@ test_ipsec_inline_proto_process(struct ipsec_test_data *td,struct rte_crypto_sym_xform cipher = {0 };struct rte_crypto_sym_xform auth = {0 };struct rte_crypto_sym_xform aead = {0 }; -struct rte_security_session *ses; +void *ses;struct rte_security_ctx *ctx;int nb_rx =0 , nb_sent;uint32_t ol_flags;
@@ -1177,7 +1177,7 @@ test_ipsec_inline_proto_process_with_esn(struct ipsec_test_data td[],struct rte_mbuf *rx_pkt =NULL ;struct rte_mbuf *tx_pkt =NULL ;int nb_rx, nb_sent; -struct rte_security_session *ses; +void *ses;struct rte_security_ctx *ctx;uint32_t ol_flags;int i, ret;
drivers/crypto/caam_jr/caam_jr.c +1 -1
@@ -1359,7 +1359,7 @@ caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp) ses =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);break ;case RTE_CRYPTO_OP_SECURITY_SESSION : - ses =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + ses =GET_SECURITY_SESS_PRIV (op->sym->session);break ;default :CAAM_JR_DP_ERR ("sessionless crypto op not supported" );
drivers/crypto/cnxk/cn10k_cryptodev_ops.c +2 -2
@@ -103,7 +103,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],if (op->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) {if (op->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION ) { - sec_sess =GET_SECURITY_SESS_PRIV (sym_op->sec_session); + sec_sess =GET_SECURITY_SESS_PRIV (sym_op->session); ret =cpt_sec_inst_fill (qp, op, sec_sess, &inst[0 ]);if (unlikely (ret))return 0 ;
@@ -322,7 +322,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op,struct cn10k_sec_session *priv;struct cn10k_ipsec_sa *sa; - priv =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + priv =GET_SECURITY_SESS_PRIV (op->sym->session); sa = &priv->sa; *qp = sa->qp; *w2 = sa->inst.w2;
drivers/crypto/cnxk/cn9k_cryptodev_ops.c +3 -3
@@ -27,7 +27,7 @@ cn9k_cpt_sec_inst_fill(struct rte_crypto_op *op,struct cn9k_ipsec_sa *sa;int ret; - priv =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + priv =GET_SECURITY_SESS_PRIV (op->sym->session); sa = &priv->sa;if (unlikely (sym_op->m_dst && sym_op->m_dst != sym_op->m_src)) {
@@ -374,7 +374,7 @@ cn9k_ca_meta_info_extract(struct rte_crypto_op *op,struct cn9k_sec_session *priv;struct cn9k_ipsec_sa *sa; - priv =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + priv =GET_SECURITY_SESS_PRIV (op->sym->session); sa = &priv->sa; *qp = sa->qp; inst->w2.u64 = sa->inst.w2;
@@ -539,7 +539,7 @@ cn9k_cpt_sec_post_process(struct rte_crypto_op *cop, CPT_OP_FLAGS_IPSEC_INB_REPLAY)) {int ret; - priv =GET_SECURITY_SESS_PRIV (sym_op->sec_session); + priv =GET_SECURITY_SESS_PRIV (sym_op->session); sa = &priv->sa; ret = ipsec_antireplay_check(
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +3 -3
@@ -1382,7 +1382,7 @@ build_sec_fd(struct rte_crypto_op *op, sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);#ifdef RTE_LIB_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) - sess =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + sess =GET_SECURITY_SESS_PRIV (op->sym->session);#endif else {DPAA2_SEC_DP_ERR ("Session type invalid\n" );
@@ -1582,7 +1582,7 @@ sec_simple_fd_to_mbuf(const struct qbman_fd *fd) mbuf->buf_iova = op->sym->aead.digest.phys_addr; op->sym->aead.digest.phys_addr =0L ; - sess_priv =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + sess_priv =GET_SECURITY_SESS_PRIV (op->sym->session);if (sess_priv->dir ==DIR_ENC ) mbuf->data_off +=SEC_FLC_DHR_OUTBOUND ;else
@@ -1678,7 +1678,7 @@ dpaa2_sec_dump(struct rte_crypto_op *op) sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) - sess =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + sess =GET_SECURITY_SESS_PRIV (op->sym->session);#endif if (sess ==NULL )
drivers/crypto/dpaa_sec/dpaa_sec.c +1 -1
@@ -1926,7 +1926,7 @@ dpaa_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops,break ;#ifdef RTE_LIB_SECURITY case RTE_CRYPTO_OP_SECURITY_SESSION : - ses =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + ses =GET_SECURITY_SESS_PRIV (op->sym->session);break ;#endif default :
drivers/crypto/ipsec_mb/pmd_aesni_mb.c +2 -2
@@ -1584,7 +1584,7 @@ set_sec_mb_job_params(IMB_JOB *job, struct ipsec_mb_qp *qp, op->status =RTE_CRYPTO_OP_STATUS_INVALID_SESSION ;return -1 ; } - session =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + session =GET_SECURITY_SESS_PRIV (op->sym->session);if (unlikely (session ==NULL )) { op->status =RTE_CRYPTO_OP_STATUS_INVALID_SESSION ;
@@ -1719,7 +1719,7 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job)* this is for DOCSIS */ is_docsis_sec =1 ; - sess =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + sess =GET_SECURITY_SESS_PRIV (op->sym->session); } else#endif sess =CRYPTODEV_GET_SYM_SESS_PRIV (op->sym->session);
drivers/crypto/qat/qat_sym.c +2 -2
@@ -101,7 +101,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,#ifdef RTE_LIB_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { - ctx =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + ctx =GET_SECURITY_SESS_PRIV (op->sym->session);if (unlikely (!ctx)) {QAT_DP_LOG (ERR ,"No session for this device" );return -EINVAL ;
@@ -145,7 +145,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg, } } - sess = (uintptr_t )op->sym->sec_session; + sess = (uintptr_t )op->sym->session; build_request = ctx->build_request[proc_type]; opaque[0 ] = sess; opaque[1 ] = (uintptr_t )build_request;
drivers/crypto/qat/qat_sym.h +2 -2
@@ -277,7 +277,7 @@ qat_sym_preprocess_requests(void **ops, uint16_t nb_ops) op = (struct rte_crypto_op *)ops[i];if (op->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION ) { - ctx =GET_SECURITY_SESS_PRIV (op->sym->sec_session); + ctx =GET_SECURITY_SESS_PRIV (op->sym->session);if (ctx ==NULL || ctx->bpi_ctx ==NULL )continue ;
@@ -310,7 +310,7 @@ qat_sym_process_response(void **op, uint8_t *resp, void *op_cookie,* Assuming at this point that if it's a security * op, that this is for DOCSIS */ - sess =GET_SECURITY_SESS_PRIV (rx_op->sym->sec_session); + sess =GET_SECURITY_SESS_PRIV (rx_op->sym->session); is_docsis_sec =1 ; } else#endif
drivers/net/iavf/iavf_ipsec_crypto.h +1 -1
@@ -5,7 +5,7 @@#ifndef _IAVF_IPSEC_CRYPTO_H_#define _IAVF_IPSEC_CRYPTO_H_ -#include <rte_security.h> +#include <rte_security_driver.h> #include "iavf.h"
examples/ipsec-secgw/ipsec_worker.c +1 -1
@@ -17,7 +17,7 @@#endif struct port_drv_mode_data { -struct rte_security_session *sess; +void *sess;struct rte_security_ctx *ctx; };
lib/cryptodev/rte_crypto_sym.h +1 -3
@@ -617,11 +617,9 @@ struct rte_crypto_sym_op {RTE_STD_C11 union {void *session; -/**< Handle for the initialised session context */ +/**< Handle for the initialised crypto/security session context */ struct rte_crypto_sym_xform *xform;/**< Session-less API crypto operation parameters */ -struct rte_security_session *sec_session; -/**< Handle for the initialised security session context */ };RTE_STD_C11
lib/ipsec/rte_ipsec_group.h +6 -6
@@ -44,16 +44,16 @@ struct rte_ipsec_group {static inline struct rte_ipsec_session *rte_ipsec_ses_from_crypto (const struct rte_crypto_op *cop) { -const struct rte_security_session *ss; -void *cs; +void *ses;if (cop->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION ) { - ss = cop->sym[0 ].sec_session; -return (struct rte_ipsec_session *)(uintptr_t )ss->opaque_data; + ses = cop->sym[0 ].session; +return (struct rte_ipsec_session *)(uintptr_t ) +rte_security_session_opaque_data_get (ses); }else if (cop->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { - cs = cop->sym[0 ].session; + ses = cop->sym[0 ].session;return (struct rte_ipsec_session *)(uintptr_t ) -rte_cryptodev_sym_session_opaque_data_get (cs); +rte_cryptodev_sym_session_opaque_data_get (ses); }return NULL ; }
lib/ipsec/ses.c +1 -1
@@ -48,7 +48,7 @@ rte_ipsec_session_prepare(struct rte_ipsec_session *ss)rte_cryptodev_sym_session_opaque_data_set (ss->crypto.ses, (uintptr_t )ss); else - ss->security.ses->opaque_data = (uintptr_t)ss; + rte_security_session_opaque_data_set(ss->security.ses, (uintptr_t)ss);return 0 ; }
lib/security/rte_security.c +6 -7
@@ -39,7 +39,7 @@ rte_security_dynfield_register(void)return rte_security_dynfield_offset; } -struct rte_security_session * +void *rte_security_session_create (struct rte_security_ctx *instance,struct rte_security_session_conf *conf,struct rte_mempool *mp)
@@ -69,12 +69,12 @@ rte_security_session_create(struct rte_security_ctx *instance, } instance->sess_cnt++; -return sess; +return (void *)sess; }int rte_security_session_update (struct rte_security_ctx *instance, -struct rte_security_session *sess, +void *sess,struct rte_security_session_conf *conf) { RTE_PTR_CHAIN3_OR_ERR_RET(instance, ops, session_update, -EINVAL,
@@ -96,7 +96,7 @@ rte_security_session_get_size(struct rte_security_ctx *instance)int rte_security_session_stats_get (struct rte_security_ctx *instance, -struct rte_security_session *sess, +void *sess,struct rte_security_stats *stats) { RTE_PTR_CHAIN3_OR_ERR_RET(instance, ops, session_stats_get, -EINVAL,
@@ -108,8 +108,7 @@ rte_security_session_stats_get(struct rte_security_ctx *instance, }int -rte_security_session_destroy (struct rte_security_ctx *instance, -struct rte_security_session *sess) +rte_security_session_destroy (struct rte_security_ctx *instance,void *sess) {int ret;
@@ -131,7 +130,7 @@ rte_security_session_destroy(struct rte_security_ctx *instance,int __rte_security_set_pkt_metadata (struct rte_security_ctx *instance, -struct rte_security_session *sess, +void *sess,struct rte_mbuf *m,void *params) {#ifdef RTE_DEBUG
lib/security/rte_security.h +53 -27
142 lines not on this page · raw message
lib/security/rte_security_driver.h +18 -0
@@ -19,6 +19,24 @@ extern "C" {#include "rte_security.h" +/** +* @internal +* Security session to be used by library for internal usage +*/ +struct rte_security_session { +RTE_MARKER cacheline0; +uint64_t opaque_data; +/**< Opaque user defined data */ +uint64_t fast_mdata; +/**< Fast metadata to be used for inline path */ +rte_iova_t driver_priv_data_iova; +/**< session private data IOVA address */ + + RTE_MARKER cacheline1 __rte_cache_min_aligned; +uint8_t driver_priv_data[0 ]; +/**< Private session material, variable size (depends on driver) */ +}; +/** * Helper macro to get driver private data */
Signature · 1 line
2.25.1
RE: [PATCH v2 0/6] crypto/security session framework rework
In reply to Akhil Goyal++ Fan Zhang
I think I missed adding one of the major contributor for this cleanup.
Quoted 181 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Wednesday, September 21, 2022 8:33 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; Vamsi Krishna Attunuru <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Igor Russkikh <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Ankur Dwivedi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; Anoob Joseph <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Akhil Goyal <[email protected]>
Subject: [PATCH v2 0/6]crypto/securitysession framework reworkThis patchset reworks the symmetric crypto and security session data structure to use a single virtual/physical contiguous buffer for symmetric
crypto/securitysession and driver private data. In addition the session data structure is now private. The session is represented as an opaque pointer in the application.With the change the session is no longer supported to be accessed by multiple device drivers. For the same reason rte_cryptodev_sym_session_init/clear APIs are deprecated as rte_cryptodev_sym_session_create/free will initialize and clear the driver specific data field.
The change was also submitted last year during DPDK 21.11 timeframe also[1], but was not applied due to lack of feedback from community. Please help in getting this cleanup merged in this cycle.
Now the similar work was already done for asymmetric crypto. This patchset is rebased over current tree and fixes all the issues reported so far.
This patchset is a v2 for the patch that was sent by Fan Zhang(Intel) with a few changes
- Added security session rework also.
- fixed issues in [2] reported on mailing list.
- few other fixes.Please review and provide feedback as soon as possible as this is intended to be merged in DPDK 22.11 RC1.
Currently the cnxk platform is tested with this change. Request everyone to review and test on their platform.
Special note to ixgbe and txgbe maintainers.
There is a wrong implementation for flow creation. Please check. A hack is added to bypass it. Please fix it separately.[1] https://patches.dpdk.org/project/dpdk/cover/20211018213452.2734720-1-
[email protected]/
[2] https://patches.dpdk.org/project/dpdk/cover/20220829160645.378406-1-
[email protected]/Akhil Goyal (5):
cryptodev: rework session framework
cryptodev: hide sym session structure
security: remove priv mempool usagedrivers/crypto: support security session get size op
security: hide session structureFan Zhang (1):
crypto/scheduler: use unified sessionapp/test-crypto-perf/cperf.h | 1 - app/test-crypto-perf/cperf_ops.c | 40 +-- app/test-crypto-perf/cperf_ops.h | 2 +- app/test-crypto-perf/cperf_test_latency.c | 9 +- app/test-crypto-perf/cperf_test_latency.h | 1 - .../cperf_test_pmd_cyclecount.c | 10 +- .../cperf_test_pmd_cyclecount.h | 1 - app/test-crypto-perf/cperf_test_throughput.c | 11 +- app/test-crypto-perf/cperf_test_throughput.h | 1 - app/test-crypto-perf/cperf_test_verify.c | 9 +- app/test-crypto-perf/cperf_test_verify.h | 1 - app/test-crypto-perf/main.c | 30 +- app/test-eventdev/test_perf_common.c | 35 +- app/test-eventdev/test_perf_common.h | 1 - app/test/test_cryptodev.c | 336 +++++------------- app/test/test_cryptodev_blockcipher.c | 16 +- app/test/test_cryptodev_security_ipsec.c | 2 +- app/test/test_cryptodev_security_ipsec.h | 2 +- app/test/test_event_crypto_adapter.c | 35 +- app/test/test_ipsec.c | 47 +-- app/test/test_ipsec_perf.c | 4 +- app/test/test_security.c | 178 ++-------- app/test/test_security_inline_proto.c | 26 +- doc/guides/prog_guide/cryptodev_lib.rst | 16 +- doc/guides/rel_notes/deprecation.rst | 9 + doc/guides/rel_notes/release_22_11.rst | 7 + drivers/common/cnxk/roc_cpt.c | 4 +- drivers/crypto/armv8/rte_armv8_pmd.c | 21 +- drivers/crypto/armv8/rte_armv8_pmd_ops.c | 32 +- drivers/crypto/bcmfs/bcmfs_sym_session.c | 39 +- drivers/crypto/bcmfs/bcmfs_sym_session.h | 3 +- drivers/crypto/caam_jr/caam_jr.c | 69 +--- drivers/crypto/ccp/ccp_crypto.c | 56 +-- drivers/crypto/ccp/ccp_pmd_ops.c | 32 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 - drivers/crypto/ccp/rte_ccp_pmd.c | 29 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 41 +-- drivers/crypto/cnxk/cn10k_ipsec.c | 45 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 38 +- drivers/crypto/cnxk/cn9k_ipsec.c | 54 +-- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 55 +-- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 16 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 70 ++-- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 6 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 69 +--- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 7 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 34 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 41 +-- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 48 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 62 +--- drivers/crypto/ipsec_mb/pmd_chacha_poly.c | 4 - drivers/crypto/ipsec_mb/pmd_kasumi.c | 10 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 9 +- drivers/crypto/ipsec_mb/pmd_zuc.c | 4 - drivers/crypto/mlx5/mlx5_crypto.c | 25 +- drivers/crypto/mvsam/rte_mrvl_pmd.c | 11 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 49 +-- drivers/crypto/nitrox/nitrox_sym.c | 39 +- drivers/crypto/null/null_crypto_pmd.c | 20 +- drivers/crypto/null/null_crypto_pmd_ops.c | 33 +- drivers/crypto/null/null_crypto_pmd_private.h | 2 - .../crypto/octeontx/otx_cryptodev_hw_access.h | 1 - drivers/crypto/octeontx/otx_cryptodev_ops.c | 67 +--- drivers/crypto/openssl/openssl_pmd_private.h | 2 - drivers/crypto/openssl/rte_openssl_pmd.c | 24 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 29 +- drivers/crypto/qat/dev/qat_sym_pmd_gen1.c | 1 + drivers/crypto/qat/qat_sym.c | 15 +- drivers/crypto/qat/qat_sym.h | 16 +- drivers/crypto/qat/qat_sym_session.c | 73 ++-- drivers/crypto/qat/qat_sym_session.h | 10 +- drivers/crypto/scheduler/scheduler_failover.c | 19 +- .../crypto/scheduler/scheduler_multicore.c | 17 + .../scheduler/scheduler_pkt_size_distr.c | 81 +++-- drivers/crypto/scheduler/scheduler_pmd_ops.c | 93 ++++- .../crypto/scheduler/scheduler_pmd_private.h | 120 ++++++- .../crypto/scheduler/scheduler_roundrobin.c | 11 +- drivers/crypto/virtio/virtio_cryptodev.c | 40 +-- drivers/crypto/virtio/virtio_rxtx.c | 3 +- drivers/net/cnxk/cn10k_ethdev_sec.c | 38 +- drivers/net/cnxk/cn9k_ethdev_sec.c | 32 +- drivers/net/iavf/iavf_ipsec_crypto.c | 23 +- drivers/net/iavf/iavf_ipsec_crypto.h | 2 +- drivers/net/ixgbe/ixgbe_ipsec.c | 31 +- drivers/net/txgbe/txgbe_ipsec.c | 32 +- examples/fips_validation/fips_dev_self_test.c | 40 +-- examples/fips_validation/main.c | 35 +- examples/ipsec-secgw/ipsec-secgw.c | 44 +-- examples/ipsec-secgw/ipsec.c | 16 +- examples/ipsec-secgw/ipsec.h | 1 - examples/ipsec-secgw/ipsec_worker.c | 2 +- examples/l2fwd-crypto/main.c | 54 +-- examples/vhost_crypto/main.c | 16 +- lib/cryptodev/cryptodev_pmd.h | 60 ++-- lib/cryptodev/cryptodev_trace_points.c | 6 - lib/cryptodev/rte_crypto.h | 3 +- lib/cryptodev/rte_crypto_sym.h | 11 +- lib/cryptodev/rte_cryptodev.c | 284 ++++++--------- lib/cryptodev/rte_cryptodev.h | 153 +++----- lib/cryptodev/rte_cryptodev_trace.h | 35 +- lib/cryptodev/version.map | 6 - lib/ipsec/rte_ipsec_group.h | 13 +- lib/ipsec/ses.c | 5 +- lib/pipeline/rte_table_action.c | 10 +- lib/security/rte_security.c | 33 +- lib/security/rte_security.h | 90 +++-- lib/security/rte_security_driver.h | 31 +- lib/vhost/rte_vhost_crypto.h | 3 - lib/vhost/vhost_crypto.c | 28 +- 109 files changed, 1255 insertions(+), 2413 deletions(-)--
2.25.1
RE: [PATCH v2 0/6] crypto/security session framework rework
Hi Akhil/Fan
Patchset verified for QAT and AESNI_MB sessions, with particular focus on Security Cipher-CRC
For the series,
Regards,
David
Quoted 210 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Wednesday, September 21, 2022 4:11 PM
To: Akhil Goyal <[email protected]>; [email protected]; Zhang, Roy Fan <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Vamsi Krishna Attunuru <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Igor Russkikh <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Ankur Dwivedi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; Anoob Joseph <[email protected]>; De Lara Guarch, Pablo <[email protected]>; [email protected]; [email protected]; Yang, Qiming <[email protected]>; Wu, Wenjun1 <[email protected]>; [email protected]; Wu, Jingjing <[email protected]>; Xing, Beilei <[email protected]>; Nithin Kumar Dabilpuram <[email protected]>
Subject: RE: [PATCH v2 0/6]crypto/securitysession framework rework++ Fan Zhang
I think I missed adding one of the major contributor for this cleanup.Quoted 4 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Wednesday, September 21, 2022 8:33 PM
To: [email protected]Quoted 8 lines
Cc: [email protected]; [email protected]; [email protected]; Vamsi Krishna Attunuru <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Igor Russkikh <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Ankur Dwivedi <[email protected]>; [email protected]; [email protected];
Quoted 3 lines
[email protected]; Anoob Joseph <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected];
Quoted 169 lines
[email protected]; [email protected]; Nithin Kumar Dabilpuram <[email protected]>; Akhil Goyal <[email protected]>
Subject: [PATCH v2 0/6]crypto/securitysession framework reworkThis patchset reworks the symmetric crypto and security session data structure to use a single virtual/physical contiguous buffer for symmetric
crypto/securitysession and driver private data. In addition the session data structure is now private. The session is represented as an opaque pointer in the application.With the change the session is no longer supported to be accessed by multiple device drivers. For the same reason rte_cryptodev_sym_session_init/clear APIs are deprecated as rte_cryptodev_sym_session_create/free will initialize and clear the driver specific data field.
The change was also submitted last year during DPDK 21.11 timeframe also[1], but was not applied due to lack of feedback from community. Please help in getting this cleanup merged in this cycle.
Now the similar work was already done for asymmetric crypto. This patchset is rebased over current tree and fixes all the issues reported so far.
This patchset is a v2 for the patch that was sent by Fan Zhang(Intel) with a few changes
- Added security session rework also.
- fixed issues in [2] reported on mailing list.
- few other fixes.Please review and provide feedback as soon as possible as this is intended to be merged in DPDK 22.11 RC1.
Currently the cnxk platform is tested with this change. Request everyone to review and test on their platform.
Special note to ixgbe and txgbe maintainers.
There is a wrong implementation for flow creation. Please check. A hack is added to bypass it. Please fix it separately.[1]
https://patches.dpdk.org/project/dpdk/cover/20211018213452.2734720-1-
[email protected]/
[2]
https://patches.dpdk.org/project/dpdk/cover/20220829160645.378406-1-
[email protected]/Akhil Goyal (5):
cryptodev: rework session framework
cryptodev: hide sym session structure
security: remove priv mempool usagedrivers/crypto: support security session get size op
security: hide session structureFan Zhang (1):
crypto/scheduler: use unified sessionapp/test-crypto-perf/cperf.h | 1 - app/test-crypto-perf/cperf_ops.c | 40 +-- app/test-crypto-perf/cperf_ops.h | 2 +- app/test-crypto-perf/cperf_test_latency.c | 9 +- app/test-crypto-perf/cperf_test_latency.h | 1 - .../cperf_test_pmd_cyclecount.c | 10 +- .../cperf_test_pmd_cyclecount.h | 1 - app/test-crypto-perf/cperf_test_throughput.c | 11 +- app/test-crypto-perf/cperf_test_throughput.h | 1 - app/test-crypto-perf/cperf_test_verify.c | 9 +- app/test-crypto-perf/cperf_test_verify.h | 1 - app/test-crypto-perf/main.c | 30 +- app/test-eventdev/test_perf_common.c | 35 +- app/test-eventdev/test_perf_common.h | 1 - app/test/test_cryptodev.c | 336 +++++------------- app/test/test_cryptodev_blockcipher.c | 16 +- app/test/test_cryptodev_security_ipsec.c | 2 +- app/test/test_cryptodev_security_ipsec.h | 2 +- app/test/test_event_crypto_adapter.c | 35 +- app/test/test_ipsec.c | 47 +-- app/test/test_ipsec_perf.c | 4 +- app/test/test_security.c | 178 ++-------- app/test/test_security_inline_proto.c | 26 +- doc/guides/prog_guide/cryptodev_lib.rst | 16 +- doc/guides/rel_notes/deprecation.rst | 9 + doc/guides/rel_notes/release_22_11.rst | 7 + drivers/common/cnxk/roc_cpt.c | 4 +- drivers/crypto/armv8/rte_armv8_pmd.c | 21 +- drivers/crypto/armv8/rte_armv8_pmd_ops.c | 32 +- drivers/crypto/bcmfs/bcmfs_sym_session.c | 39 +- drivers/crypto/bcmfs/bcmfs_sym_session.h | 3 +- drivers/crypto/caam_jr/caam_jr.c | 69 +--- drivers/crypto/ccp/ccp_crypto.c | 56 +-- drivers/crypto/ccp/ccp_pmd_ops.c | 32 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 - drivers/crypto/ccp/rte_ccp_pmd.c | 29 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 41 +-- drivers/crypto/cnxk/cn10k_ipsec.c | 45 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 38 +- drivers/crypto/cnxk/cn9k_ipsec.c | 54 +-- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 55 +-- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 16 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 70 ++-- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 6 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 69 +--- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 7 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 34 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 41 +-- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 48 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 62 +--- drivers/crypto/ipsec_mb/pmd_chacha_poly.c | 4 - drivers/crypto/ipsec_mb/pmd_kasumi.c | 10 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 9 +- drivers/crypto/ipsec_mb/pmd_zuc.c | 4 - drivers/crypto/mlx5/mlx5_crypto.c | 25 +- drivers/crypto/mvsam/rte_mrvl_pmd.c | 11 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 49 +-- drivers/crypto/nitrox/nitrox_sym.c | 39 +- drivers/crypto/null/null_crypto_pmd.c | 20 +- drivers/crypto/null/null_crypto_pmd_ops.c | 33 +- drivers/crypto/null/null_crypto_pmd_private.h | 2 - .../crypto/octeontx/otx_cryptodev_hw_access.h | 1 - drivers/crypto/octeontx/otx_cryptodev_ops.c | 67 +--- drivers/crypto/openssl/openssl_pmd_private.h | 2 - drivers/crypto/openssl/rte_openssl_pmd.c | 24 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 29 +- drivers/crypto/qat/dev/qat_sym_pmd_gen1.c | 1 + drivers/crypto/qat/qat_sym.c | 15 +- drivers/crypto/qat/qat_sym.h | 16 +- drivers/crypto/qat/qat_sym_session.c | 73 ++-- drivers/crypto/qat/qat_sym_session.h | 10 +- drivers/crypto/scheduler/scheduler_failover.c | 19 +- .../crypto/scheduler/scheduler_multicore.c | 17 + .../scheduler/scheduler_pkt_size_distr.c | 81 +++-- drivers/crypto/scheduler/scheduler_pmd_ops.c | 93 ++++- .../crypto/scheduler/scheduler_pmd_private.h | 120 ++++++- .../crypto/scheduler/scheduler_roundrobin.c | 11 +- drivers/crypto/virtio/virtio_cryptodev.c | 40 +-- drivers/crypto/virtio/virtio_rxtx.c | 3 +- drivers/net/cnxk/cn10k_ethdev_sec.c | 38 +- drivers/net/cnxk/cn9k_ethdev_sec.c | 32 +- drivers/net/iavf/iavf_ipsec_crypto.c | 23 +- drivers/net/iavf/iavf_ipsec_crypto.h | 2 +- drivers/net/ixgbe/ixgbe_ipsec.c | 31 +- drivers/net/txgbe/txgbe_ipsec.c | 32 +- examples/fips_validation/fips_dev_self_test.c | 40 +-- examples/fips_validation/main.c | 35 +- examples/ipsec-secgw/ipsec-secgw.c | 44 +-- examples/ipsec-secgw/ipsec.c | 16 +- examples/ipsec-secgw/ipsec.h | 1 - examples/ipsec-secgw/ipsec_worker.c | 2 +- examples/l2fwd-crypto/main.c | 54 +-- examples/vhost_crypto/main.c | 16 +- lib/cryptodev/cryptodev_pmd.h | 60 ++-- lib/cryptodev/cryptodev_trace_points.c | 6 - lib/cryptodev/rte_crypto.h | 3 +- lib/cryptodev/rte_crypto_sym.h | 11 +- lib/cryptodev/rte_cryptodev.c | 284 ++++++--------- lib/cryptodev/rte_cryptodev.h | 153 +++----- lib/cryptodev/rte_cryptodev_trace.h | 35 +- lib/cryptodev/version.map | 6 - lib/ipsec/rte_ipsec_group.h | 13 +- lib/ipsec/ses.c | 5 +- lib/pipeline/rte_table_action.c | 10 +- lib/security/rte_security.c | 33 +- lib/security/rte_security.h | 90 +++-- lib/security/rte_security_driver.h | 31 +- lib/vhost/rte_vhost_crypto.h | 3 - lib/vhost/vhost_crypto.c | 28 +- 109 files changed, 1255 insertions(+), 2413 deletions(-)--
2.25.1
RE: [PATCH v2 0/6] crypto/security session framework rework
In reply to Akhil GoyalHi Akhil,
We have verified this patch series on DPAA1 and DPAA2 platforms using test and ipsec-secgw applications.
Series Tested-by: Gagandeep Singh <[email protected]>
Regards,
Gagan
Quoted 195 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Wednesday, September 21, 2022 8:33 PM
To: [email protected]
Cc: [email protected]; [email protected]; Hemant Agrawal <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Gagandeep Singh <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Akhil Goyal <[email protected]>
Subject: [PATCH v2 0/6]crypto/securitysession framework reworkThis patchset reworks the symmetric crypto and security session data structure to use a single virtual/physical contiguous buffer for symmetric
crypto/securitysession and driver private data.
In addition the session data structure is now private.
The session is represented as an opaque pointer in the application.With the change the session is no longer supported to be accessed by multiple device drivers. For the same reason rte_cryptodev_sym_session_init/clear APIs are deprecated as rte_cryptodev_sym_session_create/free will initialize and clear the driver specific data field.
The change was also submitted last year during DPDK 21.11 timeframe also[1], but was not applied due to lack of feedback from community. Please help in getting this cleanup merged in this cycle.
Now the similar work was already done for asymmetric crypto.
This patchset is rebased over current tree and fixes all the issues reported so far.
This patchset is a v2 for the patch that was sent by Fan Zhang(Intel) with a few changes
- Added security session rework also.
- fixed issues in [2] reported on mailing list.
- few other fixes.Please review and provide feedback as soon as possible as this is intended to be merged in DPDK 22.11 RC1.
Currently the cnxk platform is tested with this change. Request everyone to review and test on their platform.
Special note to ixgbe and txgbe maintainers.
There is a wrong implementation for flow creation. Please check. A hack is added to bypass it. Please fix it separately.[1]
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatch
es.dpdk.org%2Fproject%2Fdpdk%2Fcover%2F20211018213452.2734720-1- gakhil%40marvell.com%2F&data=05%7C01%7Cg.singh%40nxp.com%7 C4654103c909a4dd3031208da9be279f0%7C686ea1d3bc2b4c6fa92cd99c5c3 01635%7C0%7C0%7C637993694272898803%7CUnknown%7CTWFpbGZsb3d 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% 3D%7C3000%7C%7C%7C&sdata=NkLq49IFdRzKdHwZ5mvLJhLKi5pxsV15 t5ZsC8lK1Xc%3D&reserved=0
[2]
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatch
es.dpdk.org%2Fproject%2Fdpdk%2Fcover%2F20220829160645.378406-1- roy.fan.zhang%40intel.com%2F&data=05%7C01%7Cg.singh%40nxp.co m%7C4654103c909a4dd3031208da9be279f0%7C686ea1d3bc2b4c6fa92cd99 c5c301635%7C0%7C0%7C637993694272898803%7CUnknown%7CTWFpbGZ sb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6 Mn0%3D%7C3000%7C%7C%7C&sdata=WLJbXuU%2FLDeV9QlEUhqfNF 7XAG8itcI%2BMdHgecv%2BaN4%3D&reserved=0Akhil Goyal (5):
cryptodev: rework session framework
cryptodev: hide sym session structure
security: remove priv mempool usagedrivers/crypto: support security session get size op
security: hide session structureFan Zhang (1):
crypto/scheduler: use unified sessionapp/test-crypto-perf/cperf.h | 1 - app/test-crypto-perf/cperf_ops.c | 40 +-- app/test-crypto-perf/cperf_ops.h | 2 +- app/test-crypto-perf/cperf_test_latency.c | 9 +- app/test-crypto-perf/cperf_test_latency.h | 1 - .../cperf_test_pmd_cyclecount.c | 10 +- .../cperf_test_pmd_cyclecount.h | 1 - app/test-crypto-perf/cperf_test_throughput.c | 11 +- app/test-crypto-perf/cperf_test_throughput.h | 1 - app/test-crypto-perf/cperf_test_verify.c | 9 +- app/test-crypto-perf/cperf_test_verify.h | 1 - app/test-crypto-perf/main.c | 30 +- app/test-eventdev/test_perf_common.c | 35 +- app/test-eventdev/test_perf_common.h | 1 - app/test/test_cryptodev.c | 336 +++++------------- app/test/test_cryptodev_blockcipher.c | 16 +- app/test/test_cryptodev_security_ipsec.c | 2 +- app/test/test_cryptodev_security_ipsec.h | 2 +- app/test/test_event_crypto_adapter.c | 35 +- app/test/test_ipsec.c | 47 +-- app/test/test_ipsec_perf.c | 4 +- app/test/test_security.c | 178 ++-------- app/test/test_security_inline_proto.c | 26 +- doc/guides/prog_guide/cryptodev_lib.rst | 16 +- doc/guides/rel_notes/deprecation.rst | 9 + doc/guides/rel_notes/release_22_11.rst | 7 + drivers/common/cnxk/roc_cpt.c | 4 +- drivers/crypto/armv8/rte_armv8_pmd.c | 21 +- drivers/crypto/armv8/rte_armv8_pmd_ops.c | 32 +- drivers/crypto/bcmfs/bcmfs_sym_session.c | 39 +- drivers/crypto/bcmfs/bcmfs_sym_session.h | 3 +- drivers/crypto/caam_jr/caam_jr.c | 69 +--- drivers/crypto/ccp/ccp_crypto.c | 56 +-- drivers/crypto/ccp/ccp_pmd_ops.c | 32 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 - drivers/crypto/ccp/rte_ccp_pmd.c | 29 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 41 +-- drivers/crypto/cnxk/cn10k_ipsec.c | 45 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 38 +- drivers/crypto/cnxk/cn9k_ipsec.c | 54 +-- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 55 +-- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 16 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 70 ++-- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 6 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 69 +--- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 7 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 34 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 41 +-- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 48 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 62 +--- drivers/crypto/ipsec_mb/pmd_chacha_poly.c | 4 - drivers/crypto/ipsec_mb/pmd_kasumi.c | 10 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 9 +- drivers/crypto/ipsec_mb/pmd_zuc.c | 4 - drivers/crypto/mlx5/mlx5_crypto.c | 25 +- drivers/crypto/mvsam/rte_mrvl_pmd.c | 11 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 49 +-- drivers/crypto/nitrox/nitrox_sym.c | 39 +- drivers/crypto/null/null_crypto_pmd.c | 20 +- drivers/crypto/null/null_crypto_pmd_ops.c | 33 +- drivers/crypto/null/null_crypto_pmd_private.h | 2 - .../crypto/octeontx/otx_cryptodev_hw_access.h | 1 - drivers/crypto/octeontx/otx_cryptodev_ops.c | 67 +--- drivers/crypto/openssl/openssl_pmd_private.h | 2 - drivers/crypto/openssl/rte_openssl_pmd.c | 24 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 29 +- drivers/crypto/qat/dev/qat_sym_pmd_gen1.c | 1 + drivers/crypto/qat/qat_sym.c | 15 +- drivers/crypto/qat/qat_sym.h | 16 +- drivers/crypto/qat/qat_sym_session.c | 73 ++-- drivers/crypto/qat/qat_sym_session.h | 10 +- drivers/crypto/scheduler/scheduler_failover.c | 19 +- .../crypto/scheduler/scheduler_multicore.c | 17 + .../scheduler/scheduler_pkt_size_distr.c | 81 +++-- drivers/crypto/scheduler/scheduler_pmd_ops.c | 93 ++++- .../crypto/scheduler/scheduler_pmd_private.h | 120 ++++++- .../crypto/scheduler/scheduler_roundrobin.c | 11 +- drivers/crypto/virtio/virtio_cryptodev.c | 40 +-- drivers/crypto/virtio/virtio_rxtx.c | 3 +- drivers/net/cnxk/cn10k_ethdev_sec.c | 38 +- drivers/net/cnxk/cn9k_ethdev_sec.c | 32 +- drivers/net/iavf/iavf_ipsec_crypto.c | 23 +- drivers/net/iavf/iavf_ipsec_crypto.h | 2 +- drivers/net/ixgbe/ixgbe_ipsec.c | 31 +- drivers/net/txgbe/txgbe_ipsec.c | 32 +- examples/fips_validation/fips_dev_self_test.c | 40 +-- examples/fips_validation/main.c | 35 +- examples/ipsec-secgw/ipsec-secgw.c | 44 +-- examples/ipsec-secgw/ipsec.c | 16 +- examples/ipsec-secgw/ipsec.h | 1 - examples/ipsec-secgw/ipsec_worker.c | 2 +- examples/l2fwd-crypto/main.c | 54 +-- examples/vhost_crypto/main.c | 16 +- lib/cryptodev/cryptodev_pmd.h | 60 ++-- lib/cryptodev/cryptodev_trace_points.c | 6 - lib/cryptodev/rte_crypto.h | 3 +- lib/cryptodev/rte_crypto_sym.h | 11 +- lib/cryptodev/rte_cryptodev.c | 284 ++++++--------- lib/cryptodev/rte_cryptodev.h | 153 +++----- lib/cryptodev/rte_cryptodev_trace.h | 35 +- lib/cryptodev/version.map | 6 - lib/ipsec/rte_ipsec_group.h | 13 +- lib/ipsec/ses.c | 5 +- lib/pipeline/rte_table_action.c | 10 +- lib/security/rte_security.c | 33 +- lib/security/rte_security.h | 90 +++-- lib/security/rte_security_driver.h | 31 +- lib/vhost/rte_vhost_crypto.h | 3 - lib/vhost/vhost_crypto.c | 28 +- 109 files changed, 1255 insertions(+), 2413 deletions(-)--
2.25.1
v3 cover crypto/security session framework rework
In reply to Akhil GoyalThis patchset reworks the symmetric crypto and security session data structure to use a single virtual/physical contiguous buffer for symmetric crypto/security session and driver private data. In addition the session data structure is now private. The session is represented as an opaque pointer in the application.
With the change the session is no longer supported to be accessed by multiple device drivers. For the same reason rte_cryptodev_sym_session_init/clear APIs are deprecated as rte_cryptodev_sym_session_create/free will initialize and clear the driver specific data field.
The change was also submitted last year during DPDK 21.11 timeframe also[1], but was not applied due to lack of feedback from community. Please help in getting this cleanup merged in this cycle.
Now the similar work was already done for asymmetric crypto. This patchset is rebased over current tree and fixes all the issues reported so far.
Changes in v3:
- Updated release notes
- fixed checkpatch issues
- renamed macro to get sess priv data to align with crypto macro
- added acked-by/tested-by
Changes in v2:
This patchset is a v2 for the patch that was sent by Fan Zhang(Intel) with a few changes
- Added security session rework also.
- fixed issues in [2] reported on mailing list.
- few other fixes.
Please review and provide feedback as soon as possible as this is intended to be merged in DPDK 22.11 RC1.
Currently the cnxk platform is tested with this change. Request everyone to review and test on their platform.
Special note to ixgbe and txgbe maintainers.
There is a wrong implementation for flow creation. Please check. A hack is added to bypass it. Please fix it separately.
[1] https://patches.dpdk.org/project/dpdk/cover/[email protected]/
[2] https://patches.dpdk.org/project/dpdk/cover/[email protected]/
Akhil Goyal (5):
cryptodev: rework session framework
cryptodev: hide sym session structure
security: remove priv mempool usage drivers/crypto: support security session get size op
security: hide session structure
Fan Zhang (1):
crypto/scheduler: use unified session
app/test-crypto-perf/cperf.h | 1 - app/test-crypto-perf/cperf_ops.c | 40 +-- app/test-crypto-perf/cperf_ops.h | 2 +- app/test-crypto-perf/cperf_test_latency.c | 9 +- app/test-crypto-perf/cperf_test_latency.h | 1 - .../cperf_test_pmd_cyclecount.c | 10 +- .../cperf_test_pmd_cyclecount.h | 1 - app/test-crypto-perf/cperf_test_throughput.c | 11 +- app/test-crypto-perf/cperf_test_throughput.h | 1 - app/test-crypto-perf/cperf_test_verify.c | 9 +- app/test-crypto-perf/cperf_test_verify.h | 1 - app/test-crypto-perf/main.c | 30 +- app/test-eventdev/test_perf_common.c | 35 +- app/test-eventdev/test_perf_common.h | 1 - app/test/test_cryptodev.c | 336 +++++------------- app/test/test_cryptodev_blockcipher.c | 16 +- app/test/test_cryptodev_security_ipsec.c | 2 +- app/test/test_cryptodev_security_ipsec.h | 2 +- app/test/test_event_crypto_adapter.c | 35 +- app/test/test_ipsec.c | 47 +-- app/test/test_ipsec_perf.c | 4 +- app/test/test_security.c | 178 ++-------- app/test/test_security_inline_proto.c | 26 +- doc/guides/prog_guide/cryptodev_lib.rst | 16 +- doc/guides/rel_notes/deprecation.rst | 9 - doc/guides/rel_notes/release_22_11.rst | 14 + drivers/common/cnxk/roc_cpt.c | 4 +- drivers/crypto/armv8/rte_armv8_pmd.c | 21 +- drivers/crypto/armv8/rte_armv8_pmd_ops.c | 32 +- drivers/crypto/bcmfs/bcmfs_sym_session.c | 39 +- drivers/crypto/bcmfs/bcmfs_sym_session.h | 3 +- drivers/crypto/caam_jr/caam_jr.c | 69 +--- drivers/crypto/ccp/ccp_crypto.c | 56 +-- drivers/crypto/ccp/ccp_pmd_ops.c | 32 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 - drivers/crypto/ccp/rte_ccp_pmd.c | 29 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 41 +-- drivers/crypto/cnxk/cn10k_ipsec.c | 45 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 38 +- drivers/crypto/cnxk/cn9k_ipsec.c | 54 +-- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 55 +-- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 16 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 70 ++-- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 6 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 69 +--- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 7 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 34 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 41 +-- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 48 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 69 +--- drivers/crypto/ipsec_mb/pmd_chacha_poly.c | 4 - drivers/crypto/ipsec_mb/pmd_kasumi.c | 10 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 9 +- drivers/crypto/ipsec_mb/pmd_zuc.c | 4 - drivers/crypto/mlx5/mlx5_crypto.c | 25 +- drivers/crypto/mvsam/rte_mrvl_pmd.c | 11 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 49 +-- drivers/crypto/nitrox/nitrox_sym.c | 39 +- drivers/crypto/null/null_crypto_pmd.c | 20 +- drivers/crypto/null/null_crypto_pmd_ops.c | 33 +- drivers/crypto/null/null_crypto_pmd_private.h | 2 - .../crypto/octeontx/otx_cryptodev_hw_access.h | 1 - drivers/crypto/octeontx/otx_cryptodev_ops.c | 67 +--- drivers/crypto/openssl/openssl_pmd_private.h | 2 - drivers/crypto/openssl/rte_openssl_pmd.c | 24 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 29 +- drivers/crypto/qat/dev/qat_sym_pmd_gen1.c | 1 + drivers/crypto/qat/qat_sym.c | 15 +- drivers/crypto/qat/qat_sym.h | 16 +- drivers/crypto/qat/qat_sym_session.c | 73 ++-- drivers/crypto/qat/qat_sym_session.h | 10 +- drivers/crypto/scheduler/scheduler_failover.c | 19 +- .../crypto/scheduler/scheduler_multicore.c | 17 + .../scheduler/scheduler_pkt_size_distr.c | 81 +++-- drivers/crypto/scheduler/scheduler_pmd_ops.c | 93 ++++- .../crypto/scheduler/scheduler_pmd_private.h | 120 ++++++- .../crypto/scheduler/scheduler_roundrobin.c | 11 +- drivers/crypto/virtio/virtio_cryptodev.c | 40 +-- drivers/crypto/virtio/virtio_rxtx.c | 3 +- drivers/net/cnxk/cn10k_ethdev_sec.c | 38 +- drivers/net/cnxk/cn9k_ethdev_sec.c | 32 +- drivers/net/iavf/iavf_ipsec_crypto.c | 23 +- drivers/net/iavf/iavf_ipsec_crypto.h | 2 +- drivers/net/ixgbe/ixgbe_ipsec.c | 31 +- drivers/net/txgbe/txgbe_ipsec.c | 32 +- examples/fips_validation/fips_dev_self_test.c | 40 +-- examples/fips_validation/main.c | 35 +- examples/ipsec-secgw/ipsec-secgw.c | 44 +-- examples/ipsec-secgw/ipsec.c | 16 +- examples/ipsec-secgw/ipsec.h | 1 - examples/ipsec-secgw/ipsec_worker.c | 2 +- examples/l2fwd-crypto/main.c | 54 +-- examples/vhost_crypto/main.c | 16 +- lib/cryptodev/cryptodev_pmd.h | 60 ++-- lib/cryptodev/cryptodev_trace_points.c | 6 - lib/cryptodev/rte_crypto.h | 3 +- lib/cryptodev/rte_crypto_sym.h | 11 +- lib/cryptodev/rte_cryptodev.c | 284 ++++++--------- lib/cryptodev/rte_cryptodev.h | 153 +++----- lib/cryptodev/rte_cryptodev_trace.h | 35 +- lib/cryptodev/version.map | 6 - lib/ipsec/rte_ipsec_group.h | 13 +- lib/ipsec/ses.c | 5 +- lib/pipeline/rte_table_action.c | 10 +- lib/security/rte_security.c | 33 +- lib/security/rte_security.h | 90 +++-- lib/security/rte_security_driver.h | 31 +- lib/vhost/rte_vhost_crypto.h | 3 - lib/vhost/vhost_crypto.c | 28 +- 109 files changed, 1251 insertions(+), 2431 deletions(-)
Signature · 1 line
2.25.1
PATCH v3 1/6 cryptodev: rework session framework
As per current design, rte_cryptodev_sym_session_create() and rte_cryptodev_sym_session_init() use separate mempool objects for a single session.
And structure rte_cryptodev_sym_session is not directly used by the application, it may cause ABI breakage if the structure is modified in future.
To address these two issues, the rte_cryptodev_sym_session_create will take one mempool object that the session and session private data are virtually/physically contiguous, and initializes both fields. The API rte_cryptodev_sym_session_init is removed.
rte_cryptodev_sym_session_create will now return an opaque session pointer which will be used by the app and other APIs.
In data path, opaque session pointer is attached to rte_crypto_op and the PMD can call an internal library API to get the session private data pointer based on the driver id.
Note: currently single session may be used by different device drivers, given it is initialized by them. After the change the session created by one device driver cannot be used or reinitialized by another driver.
--- app/test-crypto-perf/cperf_ops.c | 21 +- app/test-crypto-perf/cperf_test_latency.c | 6 +- .../cperf_test_pmd_cyclecount.c | 5 +- app/test-crypto-perf/cperf_test_throughput.c | 6 +- app/test-crypto-perf/cperf_test_verify.c | 6 +- app/test-crypto-perf/main.c | 29 +- app/test-eventdev/test_perf_common.c | 35 +- app/test-eventdev/test_perf_common.h | 1 - app/test/test_cryptodev.c | 302 +++++------------- app/test/test_cryptodev_blockcipher.c | 16 +- app/test/test_event_crypto_adapter.c | 35 +- app/test/test_ipsec.c | 42 +-- drivers/crypto/armv8/rte_armv8_pmd.c | 21 +- drivers/crypto/armv8/rte_armv8_pmd_ops.c | 32 +- drivers/crypto/bcmfs/bcmfs_sym_session.c | 38 +-- drivers/crypto/bcmfs/bcmfs_sym_session.h | 3 +- drivers/crypto/caam_jr/caam_jr.c | 28 +- drivers/crypto/ccp/ccp_crypto.c | 58 +--- drivers/crypto/ccp/ccp_pmd_ops.c | 32 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 - drivers/crypto/ccp/rte_ccp_pmd.c | 29 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 36 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 31 +- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 55 +--- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 14 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 31 +- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 3 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 37 +-- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 4 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 34 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 41 ++- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 48 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 36 +-- drivers/crypto/ipsec_mb/pmd_chacha_poly.c | 4 - drivers/crypto/ipsec_mb/pmd_kasumi.c | 10 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 9 +- drivers/crypto/ipsec_mb/pmd_zuc.c | 4 - drivers/crypto/mlx5/mlx5_crypto.c | 26 +- drivers/crypto/mvsam/rte_mrvl_pmd.c | 8 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 21 +- drivers/crypto/nitrox/nitrox_sym.c | 39 +-- drivers/crypto/null/null_crypto_pmd.c | 19 +- drivers/crypto/null/null_crypto_pmd_ops.c | 33 +- drivers/crypto/null/null_crypto_pmd_private.h | 2 - .../crypto/octeontx/otx_cryptodev_hw_access.h | 1 - drivers/crypto/octeontx/otx_cryptodev_ops.c | 67 +--- drivers/crypto/openssl/openssl_pmd_private.h | 2 - drivers/crypto/openssl/rte_openssl_pmd.c | 24 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 29 +- drivers/crypto/qat/qat_sym.c | 10 +- drivers/crypto/qat/qat_sym.h | 4 +- drivers/crypto/qat/qat_sym_session.c | 40 +-- drivers/crypto/qat/qat_sym_session.h | 6 +- drivers/crypto/scheduler/scheduler_pmd_ops.c | 38 +-- drivers/crypto/virtio/virtio_cryptodev.c | 40 +-- drivers/crypto/virtio/virtio_rxtx.c | 3 +- examples/fips_validation/fips_dev_self_test.c | 30 +- examples/fips_validation/main.c | 35 +- examples/ipsec-secgw/ipsec-secgw.c | 10 +- examples/ipsec-secgw/ipsec.c | 7 +- examples/l2fwd-crypto/main.c | 54 +--- examples/vhost_crypto/main.c | 16 +- lib/cryptodev/cryptodev_pmd.h | 28 +- lib/cryptodev/cryptodev_trace_points.c | 6 - lib/cryptodev/rte_cryptodev.c | 276 ++++++---------- lib/cryptodev/rte_cryptodev.h | 123 ++----- lib/cryptodev/rte_cryptodev_trace.h | 33 +- lib/cryptodev/version.map | 6 - lib/pipeline/rte_table_action.c | 10 +- lib/vhost/rte_vhost_crypto.h | 3 - lib/vhost/vhost_crypto.c | 28 +- 71 files changed, 551 insertions(+), 1670 deletions(-)
app/test-crypto-perf/cperf_ops.c +10 -11
@@ -912,7 +912,6 @@ cperf_create_session(struct rte_mempool *sess_mp, &sess_conf, sess_mp, priv_mp); }#endif - sess =rte_cryptodev_sym_session_create (sess_mp);/* * cipher only */
@@ -937,8 +936,8 @@ cperf_create_session(struct rte_mempool *sess_mp, cipher_xform.cipher.iv.length =0 ; }/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, sess, &cipher_xform, - priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, &cipher_xform, + sess_mp);/* * auth only */
@@ -965,8 +964,8 @@ cperf_create_session(struct rte_mempool *sess_mp, auth_xform.auth.iv.length =0 ; }/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, sess, &auth_xform, - priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, &auth_xform, + sess_mp);/* * cipher and auth */
@@ -1024,13 +1023,13 @@ cperf_create_session(struct rte_mempool *sess_mp,if (options->op_type ==CPERF_CIPHER_THEN_AUTH ) { cipher_xform.next = &auth_xform;/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, - sess, &cipher_xform, priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, + &cipher_xform, sess_mp); }else {/* auth then cipher */ auth_xform.next = &cipher_xform;/* create crypto session */ -rte_cryptodev_sym_session_init (dev_id, - sess, &auth_xform, priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, + &auth_xform, sess_mp); } }else {/* options->op_type == CPERF_AEAD */ aead_xform.type =RTE_CRYPTO_SYM_XFORM_AEAD ;
@@ -1050,8 +1049,8 @@ cperf_create_session(struct rte_mempool *sess_mp, options->aead_aad_sz;/* Create crypto session */ -rte_cryptodev_sym_session_init (dev_id, - sess, &aead_xform, priv_mp); + sess =rte_cryptodev_sym_session_create (dev_id, &aead_xform, + sess_mp); }return sess;
app/test-crypto-perf/cperf_test_latency.c +2 -4
@@ -44,10 +44,8 @@ static void cperf_latency_test_free(struct cperf_latency_ctx *ctx) {if (ctx) { -if (ctx->sess) { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } +if (ctx->sess) +rte_cryptodev_sym_session_free (ctx->dev_id, ctx->sess);rte_mempool_free (ctx->pool);
app/test-crypto-perf/cperf_test_pmd_cyclecount.c +1 -4
@@ -74,10 +74,7 @@ cperf_pmd_cyclecount_test_free(struct cperf_pmd_cyclecount_ctx *ctx) (struct rte_security_session *)ctx->sess); } else#endif - { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } +rte_cryptodev_sym_session_free (ctx->dev_id, ctx->sess); }rte_mempool_free (ctx->pool);
app/test-crypto-perf/cperf_test_throughput.c +2 -4
@@ -52,10 +52,8 @@ cperf_throughput_test_free(struct cperf_throughput_ctx *ctx) (struct rte_security_session *)ctx->sess); }#endif - else { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } + else + rte_cryptodev_sym_session_free(ctx->dev_id, ctx->sess); }rte_mempool_free (ctx->pool);
app/test-crypto-perf/cperf_test_verify.c +2 -4
@@ -39,10 +39,8 @@ static void cperf_verify_test_free(struct cperf_verify_ctx *ctx) {if (ctx) { -if (ctx->sess) { -rte_cryptodev_sym_session_clear (ctx->dev_id, ctx->sess); -rte_cryptodev_sym_session_free (ctx->sess); - } +if (ctx->sess) +rte_cryptodev_sym_session_free (ctx->dev_id, ctx->sess);rte_mempool_free (ctx->pool);
app/test-crypto-perf/main.c +3 -26
@@ -97,35 +97,14 @@ fill_session_pool_socket(int32_t socket_id, uint32_t session_priv_size,char mp_name[RTE_MEMPOOL_NAMESIZE ];struct rte_mempool *sess_mp; -if (session_pool_socket[socket_id].priv_mp ==NULL ) { -snprintf (mp_name,RTE_MEMPOOL_NAMESIZE , -"priv_sess_mp_%u" , socket_id); - - sess_mp =rte_mempool_create (mp_name, - nb_sessions, - session_priv_size, -0 ,0 ,NULL ,NULL ,NULL , -NULL , socket_id, -0 ); - -if (sess_mp ==NULL ) { -printf ("Cannot create pool \"%s\" on socket %d\n" , - mp_name, socket_id); -return -ENOMEM ; - } - -printf ("Allocated pool \"%s\" on socket %d\n" , - mp_name, socket_id); - session_pool_socket[socket_id].priv_mp = sess_mp; - } -if (session_pool_socket[socket_id].sess_mp ==NULL ) {snprintf (mp_name,RTE_MEMPOOL_NAMESIZE ,"sess_mp_%u" , socket_id); sess_mp =rte_cryptodev_sym_session_pool_create (mp_name, - nb_sessions,0 ,0 ,0 , socket_id); + nb_sessions, session_priv_size,0 ,0 , + socket_id);if (sess_mp ==NULL ) { printf("Cannot create pool \"%s\" on socket %d\n",
@@ -136,6 +115,7 @@ fill_session_pool_socket(int32_t socket_id, uint32_t session_priv_size,printf ("Allocated pool \"%s\" on socket %d\n" , mp_name, socket_id); session_pool_socket[socket_id].sess_mp = sess_mp; + session_pool_socket[socket_id].priv_mp = sess_mp; }return 0 ;
@@ -323,12 +303,9 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs)return ret; qp_conf.mp_session = session_pool_socket[socket_id].sess_mp; - qp_conf.mp_session_private = - session_pool_socket[socket_id].priv_mp;if (opts->op_type ==CPERF_ASYM_MODEX ) { qp_conf.mp_session =NULL ; - qp_conf.mp_session_private =NULL ; } ret =rte_cryptodev_configure (cdev_id, &conf);
app/test-eventdev/test_perf_common.c +8 -27
@@ -864,18 +864,13 @@ cryptodev_sym_sess_create(struct prod_data *p, struct test_perf *t) cipher_xform.cipher.op =RTE_CRYPTO_CIPHER_OP_ENCRYPT ; cipher_xform.next =NULL ; - sess =rte_cryptodev_sym_session_create (t->ca_sess_pool); + sess =rte_cryptodev_sym_session_create (p->ca.cdev_id, &cipher_xform, + t->ca_sess_pool);if (sess ==NULL ) {evt_err ("Failed to create sym session" );return NULL ; } -if (rte_cryptodev_sym_session_init (p->ca.cdev_id, sess, &cipher_xform, - t->ca_sess_priv_pool)) { -evt_err ("Failed to init session" ); -return NULL ; - } -return sess; }
@@ -1382,15 +1377,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) goto err; } - t->ca_sess_pool =rte_cryptodev_sym_session_pool_create ( -"ca_sess_pool" , nb_sessions,0 ,0 , -sizeof (union rte_event_crypto_metadata ),SOCKET_ID_ANY ); -if (t->ca_sess_pool ==NULL ) { -evt_err ("Failed to create sym session pool" ); - ret = -ENOMEM ; - goto err; - } - max_session_size =0 ;for (cdev_id =0 ; cdev_id < cdev_count; cdev_id++) {unsigned int session_size;
@@ -1401,12 +1387,11 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) max_session_size = session_size; } - max_session_size +=sizeof (union rte_event_crypto_metadata ); - t->ca_sess_priv_pool =rte_mempool_create ( -"ca_sess_priv_pool" , nb_sessions, max_session_size,0 ,0 ,NULL , -NULL ,NULL ,NULL ,SOCKET_ID_ANY ,0 ); -if (t->ca_sess_priv_pool ==NULL ) { -evt_err ("failed to create sym session private pool" ); + t->ca_sess_pool =rte_cryptodev_sym_session_pool_create ( +"ca_sess_pool" , nb_sessions, max_session_size,0 , +sizeof (union rte_event_crypto_metadata ),SOCKET_ID_ANY ); +if (t->ca_sess_pool ==NULL ) { +evt_err ("Failed to create sym session pool" ); ret = -ENOMEM ; goto err; }
@@ -1446,7 +1431,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) qp_conf.nb_descriptors =NB_CRYPTODEV_DESCRIPTORS ; qp_conf.mp_session = t->ca_sess_pool; - qp_conf.mp_session_private = t->ca_sess_priv_pool;for (qp_id =0 ; qp_id < conf.nb_queue_pairs; qp_id++) { ret = rte_cryptodev_queue_pair_setup(
@@ -1467,7 +1451,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt)rte_mempool_free (t->ca_op_pool);rte_mempool_free (t->ca_sess_pool); -rte_mempool_free (t->ca_sess_priv_pool);rte_mempool_free (t->ca_asym_sess_pool);return ret;
@@ -1492,8 +1475,7 @@ perf_cryptodev_destroy(struct evt_test *test, struct evt_options *opt)for (flow_id =0 ; flow_id < t->nb_flows; flow_id++) { sess = p->ca.crypto_sess[flow_id]; cdev_id = p->ca.cdev_id; -rte_cryptodev_sym_session_clear (cdev_id, sess); -rte_cryptodev_sym_session_free (sess); +rte_cryptodev_sym_session_free (cdev_id, sess); } rte_event_crypto_adapter_queue_pair_del(
@@ -1509,7 +1491,6 @@ perf_cryptodev_destroy(struct evt_test *test, struct evt_options *opt)rte_mempool_free (t->ca_op_pool);rte_mempool_free (t->ca_sess_pool); -rte_mempool_free (t->ca_sess_priv_pool);rte_mempool_free (t->ca_asym_sess_pool); }
app/test-eventdev/test_perf_common.h +0 -1
@@ -70,7 +70,6 @@ struct test_perf { RTE_EVENT_TIMER_ADAPTER_NUM_MAX] __rte_cache_aligned;struct rte_mempool *ca_op_pool;struct rte_mempool *ca_sess_pool; -struct rte_mempool *ca_sess_priv_pool;struct rte_mempool *ca_asym_sess_pool; } __rte_cache_aligned;
app/test/test_cryptodev.c +86 -216
662 lines not on this page · raw message
app/test/test_cryptodev_blockcipher.c +5 -11
@@ -68,7 +68,6 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t,struct rte_mempool *mbuf_pool, struct rte_mempool *op_mpool, struct rte_mempool *sess_mpool, - struct rte_mempool *sess_priv_mpool, uint8_t dev_id, char *test_msg) {
@@ -514,11 +513,9 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t,*/ if (!(t->feature_mask &BLOCKCIPHER_TEST_FEATURE_SESSIONLESS ) && nb_iterates ==0 ) { - sess =rte_cryptodev_sym_session_create (sess_mpool); - - status =rte_cryptodev_sym_session_init (dev_id, sess, - init_xform, sess_priv_mpool); -if (status == -ENOTSUP ) { + sess =rte_cryptodev_sym_session_create (dev_id, init_xform, + sess_mpool); +if (sess ==NULL ) {snprintf (test_msg,BLOCKCIPHER_TEST_MSG_LEN ,"UNSUPPORTED" ); status =TEST_SKIPPED ; goto error_exit;
@@ -801,10 +798,8 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, error_exit:if (!(t->feature_mask &BLOCKCIPHER_TEST_FEATURE_SESSIONLESS )) { -if (sess) { -rte_cryptodev_sym_session_clear (dev_id, sess); -rte_cryptodev_sym_session_free (sess); - } +if (sess) +rte_cryptodev_sym_session_free (dev_id, sess);rte_free (cipher_xform);rte_free (auth_xform); }
@@ -829,7 +824,6 @@ blockcipher_test_case_run(const void *data) p_testsuite_params->mbuf_pool, p_testsuite_params->op_mpool, p_testsuite_params->session_mpool, - p_testsuite_params->session_priv_mpool, p_testsuite_params->valid_devs[0 ], test_msg);return status;
app/test/test_event_crypto_adapter.c +5 -30
@@ -157,7 +157,6 @@ struct event_crypto_adapter_test_params {struct rte_mempool *op_mpool;struct rte_mempool *asym_op_mpool;struct rte_mempool *session_mpool; -struct rte_mempool *session_priv_mpool;struct rte_mempool *asym_sess_mpool;struct rte_cryptodev_config *config;uint8_t crypto_event_port_id;
@@ -307,15 +306,10 @@ test_op_forward_mode(uint8_t session_less) sym_op = op->sym;if (!session_less) { - sess =rte_cryptodev_sym_session_create ( - params.session_mpool); + sess =rte_cryptodev_sym_session_create (TEST_CDEV_ID , + &cipher_xform, params.session_mpool);TEST_ASSERT_NOT_NULL (sess,"Session creation failed\n" ); -/* Create Crypto session*/ - ret =rte_cryptodev_sym_session_init (TEST_CDEV_ID , sess, - &cipher_xform, params.session_priv_mpool); -TEST_ASSERT_SUCCESS (ret,"Failed to init session\n" ); - ret =rte_event_crypto_adapter_caps_get (evdev,TEST_CDEV_ID , &cap);TEST_ASSERT_SUCCESS (ret,"Failed to get adapter capabilities\n" );
@@ -683,8 +677,8 @@ test_op_new_mode(uint8_t session_less) sym_op = op->sym;if (!session_less) { - sess =rte_cryptodev_sym_session_create ( - params.session_mpool); + sess =rte_cryptodev_sym_session_create (TEST_CDEV_ID , + &cipher_xform, params.session_mpool);TEST_ASSERT_NOT_NULL (sess,"Session creation failed\n" ); ret = rte_event_crypto_adapter_caps_get(evdev, TEST_CDEV_ID,
@@ -699,9 +693,6 @@ test_op_new_mode(uint8_t session_less)RTE_CRYPTO_OP_WITH_SESSION , &m_data, sizeof(m_data)); } - ret =rte_cryptodev_sym_session_init (TEST_CDEV_ID , sess, - &cipher_xform, params.session_priv_mpool); -TEST_ASSERT_SUCCESS (ret,"Failed to init session\n" );rte_crypto_op_attach_sym_session (op, sess); }else {
@@ -994,22 +985,12 @@ configure_cryptodev(void) params.session_mpool =rte_cryptodev_sym_session_pool_create ("CRYPTO_ADAPTER_SESSION_MP" , -MAX_NB_SESSIONS ,0 ,0 , +MAX_NB_SESSIONS , session_size,0 ,sizeof (union rte_event_crypto_metadata ),SOCKET_ID_ANY );TEST_ASSERT_NOT_NULL (params.session_mpool,"session mempool allocation failed\n" ); - params.session_priv_mpool =rte_mempool_create ( -"CRYPTO_AD_SESS_MP_PRIV" , -MAX_NB_SESSIONS , - session_size, -0 ,0 ,NULL ,NULL ,NULL , -NULL ,SOCKET_ID_ANY , -0 ); -TEST_ASSERT_NOT_NULL (params.session_priv_mpool, -"session mempool allocation failed\n" ); -rte_cryptodev_info_get (TEST_CDEV_ID , &info); while ((capability = &info.capabilities[i++])->op !=
@@ -1048,7 +1029,6 @@ configure_cryptodev(void) qp_conf.nb_descriptors =DEFAULT_NUM_OPS_INFLIGHT ; qp_conf.mp_session = params.session_mpool; - qp_conf.mp_session_private = params.session_priv_mpool; TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( TEST_CDEV_ID, TEST_CDEV_QP_ID, &qp_conf,
@@ -1410,11 +1390,6 @@ crypto_teardown(void)rte_mempool_free (params.session_mpool); params.session_mpool =NULL ; } -if (params.session_priv_mpool !=NULL ) { -rte_mempool_avail_count (params.session_priv_mpool); -rte_mempool_free (params.session_priv_mpool); - params.session_priv_mpool =NULL ; - }/* Free asym session mempool */ if (params.asym_sess_mpool !=NULL ) {
app/test/test_ipsec.c +7 -35
79 lines not on this page · raw message
drivers/crypto/armv8/rte_armv8_pmd.c +3 -18
@@ -521,34 +521,23 @@ get_session(struct armv8_crypto_qp *qp, struct rte_crypto_op *op)if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {/* get existing session */ if (likely (op->sym->session !=NULL )) { - sess = (struct armv8_crypto_session *) -get_sym_session_private_data ( - op->sym->session, - cryptodev_driver_id); + sess = op->sym->session->data; } }else {/* provide internal session */ -void *_sess =NULL ; -void *_sess_private_data =NULL ; +struct rte_cryptodev_sym_session *_sess =NULL ;if (rte_mempool_get (qp->sess_mp, (void **)&_sess))return NULL ; -if (rte_mempool_get (qp->sess_mp_priv, - (void **)&_sess_private_data)) -return NULL ; - - sess = (struct armv8_crypto_session *)_sess_private_data; + sess = (struct armv8_crypto_session *)_sess->data;if (unlikely (armv8_crypto_set_session_parameters (sess, op->sym->xform) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); -rte_mempool_put (qp->sess_mp_priv, _sess_private_data); sess =NULL ; } op->sym->session = (struct rte_cryptodev_sym_session *)_sess; -set_sym_session_private_data (op->sym->session, - cryptodev_driver_id, _sess_private_data); }if (unlikely (sess ==NULL ))
@@ -674,10 +663,6 @@ process_op(struct armv8_crypto_qp *qp, struct rte_crypto_op *op,/* Free session if a session-less crypto op */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sess,0 ,sizeof (struct armv8_crypto_session )); -memset (op->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - op->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sess);rte_mempool_put (qp->sess_mp, op->sym->session); op->sym->session =NULL ; }
drivers/crypto/armv8/rte_armv8_pmd_ops.c +5 -27
@@ -267,8 +267,7 @@ armv8_crypto_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int armv8_crypto_pmd_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;
@@ -278,43 +277,22 @@ armv8_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev,return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -CDEV_LOG_ERR ( -"Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = sess->data; ret =armv8_crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {ARMV8_CRYPTO_LOG_ERR ("failed configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/** Clear the memory of session so it doesn't leave key material behind */ static void -armv8_crypto_pmd_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -/* Zero out the whole structure */ -if (sess_priv) { -memset (sess_priv,0 ,sizeof (struct armv8_crypto_session )); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +armv8_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}struct rte_cryptodev_ops armv8_crypto_pmd_ops = { .dev_configure = armv8_crypto_pmd_config,
drivers/crypto/bcmfs/bcmfs_sym_session.c +7 -31
@@ -211,8 +211,7 @@ bcmfs_sym_get_session(struct rte_crypto_op *op) }else if (likely (op->sym->session !=NULL )) {/* get existing session */ sess = (struct bcmfs_sym_session *) -get_sym_session_private_data (op->sym->session, - cryptodev_bcmfs_driver_id); + op->sym->session->driver_priv_data; }if (sess ==NULL )
@@ -222,10 +221,9 @@ bcmfs_sym_get_session(struct rte_crypto_op *op) }int -bcmfs_sym_session_configure (struct rte_cryptodev *dev, +bcmfs_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;
@@ -235,45 +233,23 @@ bcmfs_sym_session_configure(struct rte_cryptodev *dev,return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -BCMFS_DP_LOG (ERR , -"Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = (void *)sess->driver_priv_data; ret =crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {BCMFS_DP_LOG (ERR ,"Failed configure session parameters" ); -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/* Clear the memory of session so it doesn't leave key material behind */ void -bcmfs_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -if (sess_priv) { -struct rte_mempool *sess_mp; - -memset (sess_priv,0 ,sizeof (struct bcmfs_sym_session )); - sess_mp =rte_mempool_from_obj (sess_priv); - -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +bcmfs_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}unsigned int bcmfs_sym_session_get_private_size(struct rte_cryptodev *dev __rte_unused)
drivers/crypto/bcmfs/bcmfs_sym_session.h +1 -2
@@ -93,8 +93,7 @@ bcmfs_process_crypto_op(struct rte_crypto_op *op,int bcmfs_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool); +struct rte_cryptodev_sym_session *sess); void bcmfs_sym_session_clear(struct rte_cryptodev *dev,
drivers/crypto/caam_jr/caam_jr.c +6 -22
@@ -1357,8 +1357,7 @@ caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp)switch (op->sess_type) {case RTE_CRYPTO_OP_WITH_SESSION : ses = (struct caam_jr_session *) -get_sym_session_private_data (op->sym->session, - cryptodev_driver_id); + op->sym->session->driver_priv_data;break ;case RTE_CRYPTO_OP_SECURITY_SESSION : ses = (struct caam_jr_session *)
@@ -1692,54 +1691,39 @@ caam_jr_set_session_parameters(struct rte_cryptodev *dev, }static int -caam_jr_sym_session_configure (struct rte_cryptodev *dev, +caam_jr_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;PMD_INIT_FUNC_TRACE (); - -if (rte_mempool_get (mempool, &sess_private_data)) { -CAAM_JR_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } - + sess_private_data = (void *)sess->driver_priv_data;memset (sess_private_data,0 ,sizeof (struct caam_jr_session )); ret =caam_jr_set_session_parameters (dev, xform, sess_private_data);if (ret !=0 ) {CAAM_JR_ERR ("failed to configure session parameters" );/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, sess_private_data); -return 0 ; }/* Clear the memory of session so it doesn't leave key material behind */ static void -caam_jr_sym_session_clear (struct rte_cryptodev *dev, +caam_jr_sym_session_clear(struct rte_cryptodev *dev __rte_unused,struct rte_cryptodev_sym_session *sess) { -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = (void *)sess->driver_priv_data;struct caam_jr_session *s = (struct caam_jr_session *)sess_priv;PMD_INIT_FUNC_TRACE ();if (sess_priv) { -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -rte_free (s->cipher_key.data);rte_free (s->auth_key.data); -memset (s,0 ,sizeof (struct caam_jr_session )); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); } }
drivers/crypto/ccp/ccp_crypto.c +16 -42
142 lines not on this page · raw message
drivers/crypto/ccp/ccp_pmd_ops.c +6 -26
@@ -727,7 +727,6 @@ ccp_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, } qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private;/* mempool for batch info */ qp->batch_mp = rte_mempool_create(
@@ -757,8 +756,7 @@ ccp_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int ccp_pmd_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) {int ret;void *sess_private_data;
@@ -769,40 +767,22 @@ ccp_pmd_sym_session_configure(struct rte_cryptodev *dev,return -ENOMEM ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -CCP_LOG_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = (void *)sess->driver_priv_data; + internals = (struct ccp_private *)dev->data->dev_private; ret =ccp_set_session_parameters (sess_private_data, xform, internals);if (ret !=0 ) {CCP_LOG_ERR ("failed configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data);return 0 ; }static void -ccp_pmd_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -if (sess_priv) { -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); - -rte_mempool_put (sess_mp, sess_priv); -memset (sess_priv,0 ,sizeof (struct ccp_session )); -set_sym_session_private_data (sess, index,NULL ); - } -} +ccp_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}struct rte_cryptodev_ops ccp_ops = { .dev_configure = ccp_pmd_config,
drivers/crypto/ccp/ccp_pmd_private.h +0 -2
@@ -78,8 +78,6 @@ struct ccp_qp {/**< Ring for placing process packets */ struct rte_mempool *sess_mp;/**< Session Mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session Private Data Mempool */ struct rte_mempool *batch_mp;/**< Session Mempool for batch info */ struct rte_cryptodev_stats qp_stats;
drivers/crypto/ccp/rte_ccp_pmd.c +8 -21
@@ -56,33 +56,23 @@ get_ccp_session(struct ccp_qp *qp, struct rte_crypto_op *op)if (unlikely (op->sym->session ==NULL ))return NULL ; - sess = (struct ccp_session *) -get_sym_session_private_data ( - op->sym->session, - ccp_cryptodev_driver_id); + sess = (void *)op->sym->session->driver_priv_data; }else if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -void *_sess; -void *_sess_private_data =NULL ; +struct rte_cryptodev_sym_session *_sess;struct ccp_private *internals; -if (rte_mempool_get (qp->sess_mp, &_sess)) -return NULL ; -if (rte_mempool_get (qp->sess_mp, (void **)&_sess_private_data)) +if (rte_mempool_get (qp->sess_mp, (void **)&_sess))return NULL ; - sess = (struct ccp_session *)_sess_private_data; + sess = (void *)_sess->driver_priv_data; internals = (struct ccp_private *)qp->dev->data->dev_private;if (unlikely (ccp_set_session_parameters (sess, op->sym->xform, internals) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); -rte_mempool_put (qp->sess_mp_priv, _sess_private_data); sess =NULL ; } - op->sym->session = (struct rte_cryptodev_sym_session *)_sess; -set_sym_session_private_data (op->sym->session, - ccp_cryptodev_driver_id, - _sess_private_data); + op->sym->session = _sess; }return sess;
@@ -161,13 +151,10 @@ ccp_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,for (i =0 ; i < nb_dequeued; i++)if (unlikely (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS )) { -struct ccp_session *sess = (struct ccp_session *) -get_sym_session_private_data ( - ops[i]->sym->session, - ccp_cryptodev_driver_id); +struct ccp_session *sess = + (void *)ops[i]->sym->session->driver_priv_data; -rte_mempool_put (qp->sess_mp_priv, - sess); +memset (sess,0 ,sizeof (*sess));rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ;
drivers/crypto/cnxk/cn10k_cryptodev_ops.c +14 -22
@@ -22,24 +22,23 @@static inline struct cnxk_se_sess *cn10k_cpt_sym_temp_sess_create (struct cnxk_cpt_qp *qp,struct rte_crypto_op *op) { -const int driver_id = cn10k_cryptodev_driver_id;struct rte_crypto_sym_op *sym_op = op->sym;struct rte_cryptodev_sym_session *sess;struct cnxk_se_sess *priv;int ret;/* Create temporary session */ - sess =rte_cryptodev_sym_session_create (qp->sess_mp); -if (sess ==NULL ) +if (rte_mempool_get (qp->sess_mp, (void **)&sess) <0 )return NULL ; - ret =sym_session_configure (qp->lf.roc_cpt, driver_id, sym_op->xform, - sess, qp->sess_mp_priv); -if (ret) + ret =sym_session_configure (qp->lf.roc_cpt, sym_op->xform, + sess); +if (ret) { +rte_mempool_put (qp->sess_mp, (void *)sess); goto sess_put; + } - priv =get_sym_session_private_data (sess, driver_id); - + priv = (void *)sess->driver_priv_data; sym_op->session = sess;return priv;
@@ -111,8 +110,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[],return 0 ; w7 = sec_sess->sa.inst.w7; }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { - sess =get_sym_session_private_data ( - sym_op->session, cn10k_cryptodev_driver_id); + sess = (void *)sym_op->session->driver_priv_data; ret =cpt_sym_inst_fill (qp, op, sess, infl_req, &inst[0 ]);if (unlikely (ret))
@@ -128,8 +126,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], ret =cpt_sym_inst_fill (qp, op, sess, infl_req, &inst[0 ]);if (unlikely (ret)) { -sym_session_clear (cn10k_cryptodev_driver_id, - op->sym->session); +sym_session_clear (op->sym->session);rte_mempool_put (qp->sess_mp, op->sym->session);return 0 ; }
@@ -296,8 +293,9 @@ cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, }else if (sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - sess, cn10k_cryptodev_driver_id); + priv = (void *)( + ((struct rte_cryptodev_sym_session *)sess)-> + driver_priv_data); priv->qp = qp; priv->cpt_inst_w2 = w2; }else
@@ -334,8 +332,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - op->sym->session, cn10k_cryptodev_driver_id); + priv = (void *)op->sym->session->driver_priv_data; *qp = priv->qp; *w2 = priv->cpt_inst_w2; }else {
@@ -543,7 +540,6 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, {const uint8_t uc_compcode = res->uc_compcode;const uint8_t compcode = res->compcode; -unsigned int sz; cop->status =RTE_CRYPTO_OP_STATUS_SUCCESS ;
@@ -620,11 +616,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, temp_sess_free:if (unlikely (cop->sess_type ==RTE_CRYPTO_OP_SESSIONLESS )) {if (cop->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) { -sym_session_clear (cn10k_cryptodev_driver_id, - cop->sym->session); - sz =rte_cryptodev_sym_get_existing_header_session_size ( - cop->sym->session); -memset (cop->sym->session,0 , sz); +sym_session_clear (cop->sym->session);rte_mempool_put (qp->sess_mp, cop->sym->session); cop->sym->session =NULL ; }
drivers/crypto/cnxk/cn9k_cryptodev_ops.c +10 -21
@@ -56,23 +56,20 @@ cn9k_cpt_sec_inst_fill(struct rte_crypto_op *op,static inline struct cnxk_se_sess *cn9k_cpt_sym_temp_sess_create (struct cnxk_cpt_qp *qp,struct rte_crypto_op *op) { -const int driver_id = cn9k_cryptodev_driver_id;struct rte_crypto_sym_op *sym_op = op->sym;struct rte_cryptodev_sym_session *sess;struct cnxk_se_sess *priv;int ret;/* Create temporary session */ - sess =rte_cryptodev_sym_session_create (qp->sess_mp); -if (sess ==NULL ) +if (rte_mempool_get (qp->sess_mp, (void **)&sess) <0 )return NULL ; - ret =sym_session_configure (qp->lf.roc_cpt, driver_id, sym_op->xform, - sess, qp->sess_mp_priv); + ret =sym_session_configure (qp->lf.roc_cpt, sym_op->xform, sess);if (ret) goto sess_put; - priv =get_sym_session_private_data (sess, driver_id); + priv = (void *)sess->driver_priv_data; sym_op->session = sess;
@@ -95,8 +92,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op,if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) { sym_op = op->sym; - sess =get_sym_session_private_data ( - sym_op->session, cn9k_cryptodev_driver_id); + sess = (void *)sym_op->session->driver_priv_data; ret =cpt_sym_inst_fill (qp, op, sess, infl_req, inst); inst->w7.u64 = sess->cpt_inst_w7; }else if (op->sess_type ==RTE_CRYPTO_OP_SECURITY_SESSION )
@@ -110,8 +106,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, ret =cpt_sym_inst_fill (qp, op, sess, infl_req, inst);if (unlikely (ret)) { -sym_session_clear (cn9k_cryptodev_driver_id, - op->sym->session); +sym_session_clear (op->sym->session);rte_mempool_put (qp->sess_mp, op->sym->session); } inst->w7.u64 = sess->cpt_inst_w7;
@@ -349,8 +344,9 @@ cn9k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, }else if (sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - sess, cn9k_cryptodev_driver_id); + priv = (void *)(( + (struct rte_cryptodev_sym_session *)sess)-> + driver_priv_data); priv->qp = qp; priv->cpt_inst_w2 = w2; }else
@@ -387,8 +383,7 @@ cn9k_ca_meta_info_extract(struct rte_crypto_op *op, }else if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {struct cnxk_se_sess *priv; - priv =get_sym_session_private_data ( - op->sym->session, cn9k_cryptodev_driver_id); + priv = (void *)op->sym->session->driver_priv_data; *qp = priv->qp; inst->w2.u64 = priv->cpt_inst_w2; }else {
@@ -583,8 +578,6 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop,struct cpt_inflight_req *infl_req, struct cpt_cn9k_res_s *res) { - unsigned int sz; - if (likely(res->compcode == CPT_COMP_GOOD)) { if (unlikely(res->uc_compcode)) { if (res->uc_compcode == ROC_SE_ERR_GC_ICV_MISCOMPARE)
@@ -645,11 +638,7 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, temp_sess_free:if (unlikely (cop->sess_type ==RTE_CRYPTO_OP_SESSIONLESS )) {if (cop->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) { -sym_session_clear (cn9k_cryptodev_driver_id, - cop->sym->session); - sz =rte_cryptodev_sym_get_existing_header_session_size ( - cop->sym->session); -memset (cop->sym->session,0 , sz); +sym_session_clear (cop->sym->session);rte_mempool_put (qp->sess_mp, cop->sym->session); cop->sym->session =NULL ; }
drivers/crypto/cnxk/cnxk_cryptodev_ops.c +11 -44
104 lines not on this page · raw message
drivers/crypto/cnxk/cnxk_cryptodev_ops.h +5 -9
@@ -78,8 +78,6 @@ struct cnxk_cpt_qp {/**< Crypto adapter related info */ struct rte_mempool *sess_mp;/**< Session mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session private data mempool */ }; int cnxk_cpt_dev_config(struct rte_cryptodev *dev,
@@ -104,18 +102,16 @@ unsigned int cnxk_cpt_sym_session_get_size(struct rte_cryptodev *dev);int cnxk_cpt_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *pool); +struct rte_cryptodev_sym_session *sess); -int sym_session_configure (struct roc_cpt *roc_cpt,int driver_id, +int sym_session_configure (struct roc_cpt *roc_cpt,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *pool); +struct rte_cryptodev_sym_session *sess);void cnxk_cpt_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess); +struct rte_cryptodev_sym_session *sess); -void sym_session_clear (int driver_id,struct rte_cryptodev_sym_session *sess); +void sym_session_clear (struct rte_cryptodev_sym_session *sess); unsigned int cnxk_ae_session_size_get(struct rte_cryptodev *dev __rte_unused);
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +7 -24
@@ -1379,8 +1379,7 @@ build_sec_fd(struct rte_crypto_op *op,dpaa2_sec_session *sess;if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa2_sec_session *)get_sym_session_private_data ( - op->sym->session, cryptodev_driver_id); + sess = (dpaa2_sec_session *)op->sym->session->driver_priv_data;#ifdef RTE_LIB_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa2_sec_session *)get_sec_session_private_data(
@@ -1678,8 +1677,7 @@ dpaa2_sec_dump(struct rte_crypto_op *op)struct rte_crypto_sym_op *sym_op;if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa2_sec_session *)get_sym_session_private_data ( - op->sym->session, cryptodev_driver_id); + sess = (dpaa2_sec_session *)op->sym->session->driver_priv_data;#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa2_sec_session *)get_sec_session_private_data(
@@ -3754,51 +3752,36 @@ dpaa2_sec_security_session_destroy(void *dev __rte_unused, }#endif static int -dpaa2_sec_sym_session_configure (struct rte_cryptodev *dev, +dpaa2_sec_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *sess_private_data; +void *sess_private_data = (void *)sess->driver_priv_data;int ret; -if (rte_mempool_get (mempool, &sess_private_data)) { -DPAA2_SEC_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } - ret =dpaa2_sec_set_session_parameters (xform, sess_private_data);if (ret !=0 ) {DPAA2_SEC_ERR ("Failed to configure session parameters" );/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/** Clear the memory of session so it doesn't leave key material behind */ static void -dpaa2_sec_sym_session_clear (struct rte_cryptodev *dev, +dpaa2_sec_sym_session_clear(struct rte_cryptodev *dev __rte_unused,struct rte_cryptodev_sym_session *sess) {PMD_INIT_FUNC_TRACE (); -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = (void *)sess->driver_priv_data;dpaa2_sec_session *s = (dpaa2_sec_session *)sess_priv;if (sess_priv) {rte_free (s->ctxt);rte_free (s->cipher_key.data);rte_free (s->auth_key.data); -memset (s,0 ,sizeof (dpaa2_sec_session)); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); } }
drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c +1 -2
@@ -1012,8 +1012,7 @@ dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess = (dpaa2_sec_session *)get_sec_session_private_data ( session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa2_sec_session *)get_sym_session_private_data ( - session_ctx.crypto_sess, cryptodev_driver_id); + sess = (void *)session_ctx.crypto_sess->driver_priv_data; else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa2_sec_raw_dequeue_burst;
drivers/crypto/dpaa_sec/dpaa_sec.c +7 -30
@@ -670,10 +670,7 @@ dpaa_sec_dump(struct dpaa_sec_op_ctx *ctx, struct dpaa_sec_qp *qp)struct qm_sg_entry sg[2 ];if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) - sess = (dpaa_sec_session *) -get_sym_session_private_data ( - op->sym->session, - dpaa_cryptodev_driver_id); + sess = (dpaa_sec_session *)op->sym->session->driver_priv_data;#ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa_sec_session *)
@@ -1927,10 +1924,8 @@ dpaa_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops,switch (op->sess_type) {case RTE_CRYPTO_OP_WITH_SESSION : - ses = (dpaa_sec_session *) -get_sym_session_private_data ( - op->sym->session, - dpaa_cryptodev_driver_id); + ses = (void *) + op->sym->session->driver_priv_data;break ;#ifdef RTE_LIB_SECURITY case RTE_CRYPTO_OP_SECURITY_SESSION :
@@ -2676,31 +2671,19 @@ dpaa_sec_set_session_parameters(struct rte_cryptodev *dev,static int dpaa_sec_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *sess_private_data; +void *sess_private_data = (void *)sess->driver_priv_data;int ret;PMD_INIT_FUNC_TRACE (); -if (rte_mempool_get (mempool, &sess_private_data)) { -DPAA_SEC_ERR ("Couldn't get object from session mempool" ); -return -ENOMEM ; - } - ret =dpaa_sec_set_session_parameters (dev, xform, sess_private_data);if (ret !=0 ) {DPAA_SEC_ERR ("failed to configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); - ret =dpaa_sec_prep_cdb (sess_private_data);if (ret) {DPAA_SEC_ERR ("Unable to prepare sec cdb" );
@@ -2714,7 +2697,6 @@ static inline void free_session_memory(struct rte_cryptodev *dev, dpaa_sec_session *s) { struct dpaa_sec_dev_private *qi = dev->data->dev_private; -struct rte_mempool *sess_mp =rte_mempool_from_obj ((void *)s);uint8_t i;for (i =0 ; i <MAX_DPAA_CORES ; i++) {
@@ -2724,7 +2706,6 @@ free_session_memory(struct rte_cryptodev *dev, dpaa_sec_session *s) s->qp[i] =NULL ; }free_session_data (s); -rte_mempool_put (sess_mp, (void *)s); }/** Clear the memory of session so it doesn't leave key material behind */
@@ -2733,14 +2714,10 @@ dpaa_sec_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) {PMD_INIT_FUNC_TRACE (); -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = (void *)sess->driver_priv_data;dpaa_sec_session *s = (dpaa_sec_session *)sess_priv; -if (sess_priv) { -free_session_memory (dev, s); -set_sym_session_private_data (sess, index,NULL ); - } +free_session_memory (dev, s); }#ifdef RTE_LIB_SECURITY
drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c +2 -2
@@ -1017,8 +1017,8 @@ dpaa_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess = (dpaa_sec_session *)get_sec_session_private_data ( session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa_sec_session *)get_sym_session_private_data ( - session_ctx.crypto_sess, dpaa_cryptodev_driver_id); + sess = (dpaa_sec_session *) + session_ctx.crypto_sess->driver_priv_data; else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa_sec_raw_dequeue_burst;
drivers/crypto/ipsec_mb/ipsec_mb_ops.c +6 -28
@@ -264,7 +264,6 @@ ipsec_mb_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, qp->pmd_type = internals->pmd_type; qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private; qp->ingress_queue =ipsec_mb_qp_create_processed_ops_ring (qp, qp_conf->nb_descriptors, socket_id);
@@ -312,9 +311,8 @@ ipsec_mb_sym_session_get_size(struct rte_cryptodev *dev)int ipsec_mb_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess,struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *sess_private_data;struct ipsec_mb_dev_private *internals = dev->data->dev_private;struct ipsec_mb_internals *pmd_data = &ipsec_mb_pmds[internals->pmd_type];
@@ -330,42 +328,22 @@ ipsec_mb_sym_session_configure(return -EINVAL ; } -if (rte_mempool_get (mempool, &sess_private_data)) { -IPSEC_MB_LOG (ERR ,"Couldn't get object from session mempool" ); -free_mb_mgr (mb_mgr); -return -ENOMEM ; - } - - ret = (*pmd_data->session_configure)(mb_mgr, sess_private_data, xform); + ret = (*pmd_data->session_configure)(mb_mgr, + (void *)sess->driver_priv_data, xform);if (ret !=0 ) {IPSEC_MB_LOG (ERR ,"failed configure session parameters" );/* Return session to mempool */ -rte_mempool_put (mempool, sess_private_data);free_mb_mgr (mb_mgr);return ret; } -set_sym_session_private_data (sess, dev->driver_id, sess_private_data); -free_mb_mgr (mb_mgr);return 0 ; }/** Clear the session memory */ void -ipsec_mb_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -/* Zero out the whole structure */ -if (sess_priv) { -memset (sess_priv,0 ,ipsec_mb_sym_session_get_size (dev)); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); - -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +ipsec_mb_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}
drivers/crypto/ipsec_mb/ipsec_mb_private.h +19 -22
85 lines not on this page · raw message
drivers/crypto/ipsec_mb/pmd_aesni_gcm.c +14 -34
90 lines not on this page · raw message
drivers/crypto/ipsec_mb/pmd_aesni_mb.c +3 -33
@@ -1710,8 +1710,6 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) {struct rte_crypto_op *op = (struct rte_crypto_op *)job->user_data;struct aesni_mb_session *sess =NULL ; -uint32_t driver_id =ipsec_mb_get_driver_id ( -IPSEC_MB_PMD_TYPE_AESNI_MB );#ifdef AESNI_MB_DOCSIS_SEC_ENABLED uint8_t is_docsis_sec =0 ;
@@ -1725,15 +1723,7 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) sess =get_sec_session_private_data (op->sym->sec_session); } else#endif - { - sess =get_sym_session_private_data (op->sym->session, - driver_id); - } - -if (unlikely (sess ==NULL )) { - op->status =RTE_CRYPTO_OP_STATUS_INVALID_SESSION ; -return op; - } + sess = (void *)op->sym->session->driver_priv_data;if (likely (op->status ==RTE_CRYPTO_OP_STATUS_NOT_PROCESSED )) {switch (job->status) {
@@ -1771,10 +1761,6 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job)/* Free session if a session-less crypto op */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sess,0 ,sizeof (struct aesni_mb_session )); -memset (op->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - op->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sess);rte_mempool_put (qp->sess_mp, op->sym->session); op->sym->session =NULL ; }
@@ -1962,16 +1948,6 @@ aesni_mb_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,return processed_jobs; } - -static inline void -ipsec_mb_fill_error_code (struct rte_crypto_sym_vec *vec,int32_t err) -{ -uint32_t i; - -for (i =0 ; i != vec->num; ++i) - vec->status[i] = err; -} -static inline int check_crypto_sgl (union rte_crypto_sym_ofs so,const struct rte_crypto_sgl *sgl) {
@@ -2028,7 +2004,7 @@ verify_sync_dgst(struct rte_crypto_sym_vec *vec, }static uint32_t -aesni_mb_process_bulk (struct rte_cryptodev *dev, +aesni_mb_process_bulk(struct rte_cryptodev *dev __rte_unused,struct rte_cryptodev_sym_session *sess,union rte_crypto_sym_ofs sofs,struct rte_crypto_sym_vec *vec) {
@@ -2037,15 +2013,9 @@ aesni_mb_process_bulk(struct rte_cryptodev *dev,void *buf;IMB_JOB *job;IMB_MGR *mb_mgr; -struct aesni_mb_session *s; +struct aesni_mb_session *s = (void *)sess->driver_priv_data;uint8_t tmp_dgst[vec->num][DIGEST_LENGTH_MAX ]; - s =get_sym_session_private_data (sess, dev->driver_id); -if (s ==NULL ) { -ipsec_mb_fill_error_code (vec,EINVAL ); -return 0 ; - } -/* get per-thread MB MGR, create one if needed */ mb_mgr =get_per_thread_mb_mgr ();if (unlikely (mb_mgr ==NULL ))
drivers/crypto/ipsec_mb/pmd_chacha_poly.c +0 -4
@@ -290,10 +290,6 @@ handle_completed_chacha20_poly1305_crypto_op(struct ipsec_mb_qp *qp,/* Free session if a session-less crypto op */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sess,0 ,sizeof (struct chacha20_poly1305_session )); -memset (op->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - op->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sess);rte_mempool_put (qp->sess_mp, op->sym->session); op->sym->session =NULL ; }
drivers/crypto/ipsec_mb/pmd_kasumi.c +3 -7
@@ -231,11 +231,6 @@ process_ops(struct rte_crypto_op **ops, struct kasumi_session *session,/* Free session if a session-less crypto op. */ if (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (session,0 ,sizeof (struct kasumi_session )); -memset ( - ops[i]->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - ops[i]->sym->session)); -rte_mempool_put (qp->sess_mp_priv, session);rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ; }
@@ -287,8 +282,9 @@ process_op_bit(struct rte_crypto_op *op, struct kasumi_session *session,/* Free session if a session-less crypto op. */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -memset (op->sym->session,0 ,sizeof (struct kasumi_session )); -rte_cryptodev_sym_session_free (op->sym->session); +memset (op->sym->session->driver_priv_data,0 , +sizeof (struct kasumi_session )); +rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ; }return processed_op;
drivers/crypto/ipsec_mb/pmd_snow3g.c +3 -6
@@ -362,10 +362,6 @@ process_ops(struct rte_crypto_op **ops, struct snow3g_session *session,/* Free session if a session-less crypto op. */ if (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (session,0 ,sizeof (struct snow3g_session )); -memset (ops[i]->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - ops[i]->sym->session)); -rte_mempool_put (qp->sess_mp_priv, session);rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ; }
@@ -417,8 +413,9 @@ process_op_bit(struct rte_crypto_op *op, struct snow3g_session *session,/* Free session if a session-less crypto op. */ if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) { -memset (op->sym->session,0 ,sizeof (struct snow3g_session )); -rte_cryptodev_sym_session_free (op->sym->session); +memset (op->sym->session->driver_priv_data,0 , +sizeof (struct snow3g_session )); +rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ; }
drivers/crypto/ipsec_mb/pmd_zuc.c +0 -4
@@ -239,10 +239,6 @@ process_ops(struct rte_crypto_op **ops, enum ipsec_mb_operation op_type,/* Free session if a session-less crypto op. */ if (ops[i]->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (sessions[i],0 ,sizeof (struct zuc_session )); -memset (ops[i]->sym->session,0 , -rte_cryptodev_sym_get_existing_header_session_size ( - ops[i]->sym->session)); -rte_mempool_put (qp->sess_mp_priv, sessions[i]);rte_mempool_put (qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session =NULL ; }
drivers/crypto/mlx5/mlx5_crypto.c +6 -20
@@ -171,14 +171,13 @@ mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int mlx5_crypto_sym_session_configure (struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *session, -struct rte_mempool *mp) +struct rte_cryptodev_sym_session *session) {struct mlx5_crypto_priv *priv = dev->data->dev_private; -struct mlx5_crypto_session *sess_private_data; +struct mlx5_crypto_session *sess_private_data = + (void *)session->driver_priv_data;struct rte_crypto_cipher_xform *cipher;uint8_t encryption_order; -int ret;if (unlikely (xform->next !=NULL )) {DRV_LOG (ERR ,"Xform next is not supported." );
@@ -189,17 +188,9 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev,DRV_LOG (ERR ,"Only AES-XTS algorithm is supported." );return -ENOTSUP ; } - ret =rte_mempool_get (mp, (void *)&sess_private_data); -if (ret !=0 ) { -DRV_LOG (ERR , -"Failed to get session %p private data from mempool." , - sess_private_data); -return -ENOMEM ; - } cipher = &xform->cipher; sess_private_data->dek =mlx5_crypto_dek_prepare (priv, cipher);if (sess_private_data->dek ==NULL ) { -rte_mempool_put (mp, sess_private_data);DRV_LOG (ERR ,"Failed to prepare dek." );return -ENOMEM ; }
@@ -239,8 +230,6 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev, sess_private_data->dek_id =rte_cpu_to_be_32 (sess_private_data->dek->obj->id &0xffffff ); -set_sym_session_private_data (session, dev->driver_id, - sess_private_data);DRV_LOG (DEBUG ,"Session %p was configured." , sess_private_data);return 0 ; }
@@ -250,16 +239,13 @@ mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) {struct mlx5_crypto_priv *priv = dev->data->dev_private; -struct mlx5_crypto_session *spriv =get_sym_session_private_data (sess, - dev->driver_id); +struct mlx5_crypto_session *spriv = (void *)sess->driver_priv_data;if (unlikely (spriv ==NULL )) {DRV_LOG (ERR ,"Failed to get session %p private data." , spriv);return ; }mlx5_crypto_dek_destroy (priv, spriv->dek); -set_sym_session_private_data (sess, dev->driver_id,NULL ); -rte_mempool_put (rte_mempool_from_obj (spriv), spriv);DRV_LOG (DEBUG ,"Session %p was cleared." , spriv); }
@@ -369,8 +355,8 @@ mlx5_crypto_wqe_set(struct mlx5_crypto_priv *priv,struct rte_crypto_op *op, struct mlx5_umr_wqe *umr) { - struct mlx5_crypto_session *sess = get_sym_session_private_data - (op->sym->session, mlx5_crypto_driver_id); + struct mlx5_crypto_session *sess = + (void *)op->sym->session->driver_priv_data;struct mlx5_wqe_cseg *cseg = &umr->ctr;struct mlx5_wqe_mkey_cseg *mkc = &umr->mkc;struct mlx5_wqe_dseg *klms = &umr->kseg[0 ];
drivers/crypto/mvsam/rte_mrvl_pmd.c +1 -7
@@ -597,13 +597,7 @@ mrvl_request_prepare_crp(struct sam_cio_op_params *request,return -EINVAL ; } - sess = (struct mrvl_crypto_session *)get_sym_session_private_data ( - op->sym->session, - cryptodev_driver_id); -if (unlikely (sess ==NULL )) { -MRVL_LOG (ERR ,"Session was not created for this device!" ); -return -EINVAL ; - } + sess = (struct mrvl_crypto_session *)op->sym->session->data; request->sa = sess->sam_sess; request->cookie = op;
drivers/crypto/mvsam/rte_mrvl_pmd_ops.c +3 -18
@@ -736,8 +736,7 @@ mrvl_crypto_pmd_sym_session_get_size(__rte_unused struct rte_cryptodev *dev)static int mrvl_crypto_pmd_sym_session_configure(__rte_unused struct rte_cryptodev *dev,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mp) +struct rte_cryptodev_sym_session *sess) {struct mrvl_crypto_session *mrvl_sess;void *sess_private_data;
@@ -748,23 +747,15 @@ mrvl_crypto_pmd_sym_session_configure(__rte_unused struct rte_cryptodev *dev,return -EINVAL ; } -if (rte_mempool_get (mp, &sess_private_data)) { -CDEV_LOG_ERR ("Couldn't get object from session mempool." ); -return -ENOMEM ; - } - + sess_private_data = sess->data;memset (sess_private_data,0 ,sizeof (struct mrvl_crypto_session )); ret =mrvl_crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {MRVL_LOG (ERR ,"Failed to configure session parameters!" ); - -/* Return session to mempool */ -rte_mempool_put (mp, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, sess_private_data); mrvl_sess = (struct mrvl_crypto_session *)sess_private_data; if (sam_session_create(&mrvl_sess->sam_sess_params,
@@ -791,8 +782,7 @@ mrvl_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) { -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); +void *sess_priv = sess->data;/* Zero out the whole structure */ if (sess_priv) {
@@ -803,11 +793,6 @@ mrvl_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev, sam_session_destroy(mrvl_sess->sam_sess) < 0) {MRVL_LOG (ERR ,"Error while destroying session!" ); } - -memset (mrvl_sess,0 ,sizeof (struct mrvl_crypto_session )); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); } }
drivers/crypto/nitrox/nitrox_sym.c +8 -31
@@ -530,24 +530,16 @@ configure_aead_ctx(struct rte_crypto_aead_xform *xform, }static int -nitrox_sym_dev_sess_configure (struct rte_cryptodev *cdev, +nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mempool) +struct rte_cryptodev_sym_session *sess) { -void *mp_obj; -struct nitrox_crypto_ctx *ctx; +struct nitrox_crypto_ctx *ctx = (void *)sess->driver_priv_data;struct rte_crypto_cipher_xform *cipher_xform =NULL ;struct rte_crypto_auth_xform *auth_xform =NULL ;struct rte_crypto_aead_xform *aead_xform =NULL ;int ret = -EINVAL ; -if (rte_mempool_get (mempool, &mp_obj)) { -NITROX_LOG (ERR ,"Couldn't allocate context\n" ); -return -ENOMEM ; - } - - ctx = mp_obj; ctx->nitrox_chain =get_crypto_chain_order (xform);switch (ctx->nitrox_chain) {case NITROX_CHAIN_CIPHER_ONLY :
@@ -585,38 +577,23 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev, goto err; } - ctx->iova =rte_mempool_virt2iova (ctx); -set_sym_session_private_data (sess, cdev->driver_id, ctx); + ctx->iova = sess->driver_priv_data_iova;return 0 ; err: -rte_mempool_put (mempool, mp_obj);return ret; }static void -nitrox_sym_dev_sess_clear (struct rte_cryptodev *cdev, -struct rte_cryptodev_sym_session *sess) -{ -struct nitrox_crypto_ctx *ctx =get_sym_session_private_data (sess, - cdev->driver_id); -struct rte_mempool *sess_mp; - -if (!ctx) -return ; - -memset (ctx,0 ,sizeof (*ctx)); - sess_mp =rte_mempool_from_obj (ctx); -set_sym_session_private_data (sess, cdev->driver_id,NULL ); -rte_mempool_put (sess_mp, ctx); -} +nitrox_sym_dev_sess_clear(struct rte_cryptodev *cdev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}static struct nitrox_crypto_ctx *get_crypto_ctx (struct rte_crypto_op *op) {if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {if (likely (op->sym->session)) -return get_sym_session_private_data (op->sym->session, - nitrox_sym_drv_id); +return (void *)op->sym->session->driver_priv_data; }return NULL ;
drivers/crypto/null/null_crypto_pmd.c +5 -14
@@ -58,7 +58,7 @@ process_op(const struct null_crypto_qp *qp, struct rte_crypto_op *op,if (op->sess_type ==RTE_CRYPTO_OP_SESSIONLESS ) {memset (op->sym->session,0 ,sizeof (struct null_crypto_session )); -rte_cryptodev_sym_session_free (op->sym->session); +rte_mempool_put (qp->sess_mp, (void *)op->sym->session); op->sym->session =NULL ; }
@@ -78,30 +78,21 @@ get_session(struct null_crypto_qp *qp, struct rte_crypto_op *op)if (op->sess_type ==RTE_CRYPTO_OP_WITH_SESSION ) {if (likely (sym_op->session !=NULL )) sess = (struct null_crypto_session *) -get_sym_session_private_data ( - sym_op->session, cryptodev_driver_id); + sym_op->session->driver_priv_data; }else { -void *_sess =NULL ; -void *_sess_private_data =NULL ; +struct rte_cryptodev_sym_session *_sess =NULL ;if (rte_mempool_get (qp->sess_mp, (void **)&_sess))return NULL ; -if (rte_mempool_get (qp->sess_mp_priv, - (void **)&_sess_private_data)) -return NULL ; - - sess = (struct null_crypto_session *)_sess_private_data; + sess = (struct null_crypto_session *)_sess->driver_priv_data;if (unlikely (null_crypto_set_session_parameters (sess, sym_op->xform) !=0 )) {rte_mempool_put (qp->sess_mp, _sess); -rte_mempool_put (qp->sess_mp_priv, _sess_private_data); sess =NULL ; } - sym_op->session = (struct rte_cryptodev_sym_session *)_sess; -set_sym_session_private_data (op->sym->session, - cryptodev_driver_id, _sess_private_data); + sym_op->session = _sess; }return sess;
drivers/crypto/null/null_crypto_pmd_ops.c +5 -28
@@ -233,7 +233,6 @@ null_crypto_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, } qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private;memset (&qp->qp_stats,0 ,sizeof (qp->qp_stats));
@@ -256,8 +255,7 @@ null_crypto_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)static int null_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused,struct rte_crypto_sym_xform *xform, -struct rte_cryptodev_sym_session *sess, -struct rte_mempool *mp) +struct rte_cryptodev_sym_session *sess) {void *sess_private_data;int ret;
@@ -267,43 +265,22 @@ null_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused,return -EINVAL ; } -if (rte_mempool_get (mp, &sess_private_data)) { -NULL_LOG (ERR , -"Couldn't get object from session mempool" ); -return -ENOMEM ; - } + sess_private_data = (void *)sess->driver_priv_data; ret =null_crypto_set_session_parameters (sess_private_data, xform);if (ret !=0 ) {NULL_LOG (ERR ,"failed configure session parameters" ); - -/* Return session to mempool */ -rte_mempool_put (mp, sess_private_data);return ret; } -set_sym_session_private_data (sess, dev->driver_id, - sess_private_data); -return 0 ; }/** Clear the memory of session so it doesn't leave key material behind */ static void -null_crypto_pmd_sym_session_clear (struct rte_cryptodev *dev, -struct rte_cryptodev_sym_session *sess) -{ -uint8_t index = dev->driver_id; -void *sess_priv =get_sym_session_private_data (sess, index); - -/* Zero out the whole structure */ -if (sess_priv) { -memset (sess_priv,0 ,sizeof (struct null_crypto_session )); -struct rte_mempool *sess_mp =rte_mempool_from_obj (sess_priv); -set_sym_session_private_data (sess, index,NULL ); -rte_mempool_put (sess_mp, sess_priv); - } -} +null_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{}static struct rte_cryptodev_ops pmd_ops = { .dev_configure = null_crypto_pmd_config,
drivers/crypto/null/null_crypto_pmd_private.h +0 -2
@@ -31,8 +31,6 @@ struct null_crypto_qp {/**< Ring for placing process packets */ struct rte_mempool *sess_mp;/**< Session Mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session Mempool */ struct rte_cryptodev_stats qp_stats;/**< Queue pair statistics */ } __rte_cache_aligned;
drivers/crypto/octeontx/otx_cryptodev_hw_access.h +0 -1
@@ -49,7 +49,6 @@ struct cpt_instance {uint32_t queue_id;uintptr_t rsvd;struct rte_mempool *sess_mp; -struct rte_mempool *sess_mp_priv;struct cpt_qp_meta_info meta_info;uint8_t ca_enabled; };
drivers/crypto/octeontx/otx_cryptodev_ops.c +17 -50
166 lines not on this page · raw message
drivers/crypto/openssl/openssl_pmd_private.h +0 -2
@@ -70,8 +70,6 @@ struct openssl_qp {/**< Ring for placing process packets */ struct rte_mempool *sess_mp;/**< Session Mempool */ -struct rte_mempool *sess_mp_priv; -/**< Session Private Data Mempool */ struct rte_cryptodev_stats stats;/**< Queue pair statistics */ uint8_t temp_digest[DIGEST_LENGTH_MAX ];
drivers/crypto/openssl/rte_openssl_pmd.c +6 -18
@@ -887,10 +887,8 @@ get_session(struct openssl_qp *qp, struct rte_crypto_op *op)if (op->type ==RTE_CRYPTO_OP_TYPE_SYMMETRIC ) {/* get existing session */ if (likely (op->sym->session !=NULL )) - sess = (struct openssl_session *) -get_sym_session_private_data ( - op->sym->session, - cryptodev_driver_id); + sess = (void *) + op->sym->session->driver_priv_data; }else {if (likely (op->asym->session !=NULL )) asym_sess = (struct openssl_asym_session *)
Lines of this file are not shown. View the raw message.
drivers/crypto/openssl/rte_openssl_pmd_ops.c +5 -24
Lines of this file are not shown. View the raw message.
drivers/crypto/scheduler/scheduler_pmd_ops.c +6 -32
Lines of this file are not shown. View the raw message.
drivers/crypto/virtio/virtio_cryptodev.c +7 -33
Lines of this file are not shown. View the raw message.
examples/fips_validation/fips_dev_self_test.c +5 -25
Lines of this file are not shown. View the raw message.
examples/l2fwd-crypto/main.c +6 -48
54 lines not on this page · raw message
lib/cryptodev/rte_cryptodev.c +97 -179
276 lines not on this page · raw message
lib/cryptodev/rte_cryptodev.h +33 -90
123 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.25.1
Part of the diff is not shown here. View the raw message.
PATCH v3 2/6 crypto/scheduler: use unified session
In reply to Akhil GoyalFrom: Fan Zhang <[email protected]>
This patch updates the scheduler PMD to use unified session data structure. Previously thanks to the private session array in cryptodev sym session there are no necessary change needed for scheduler PMD other than the way ops are enqueued/dequeued. The patch inherits the same design in the original session data structure to the scheduler PMD so the cryptodev sym session can be as a linear buffer for both session header and driver private data.
With the change there are inevitable extra cost on both memory (64 bytes per session per driver type) and cycle count (set the correct session for each cop based on the worker before enqueue, and retrieve the original session after dequeue).
--- drivers/crypto/scheduler/scheduler_failover.c | 19 ++- .../crypto/scheduler/scheduler_multicore.c | 17 +++ .../scheduler/scheduler_pkt_size_distr.c | 84 +++++++++--- drivers/crypto/scheduler/scheduler_pmd_ops.c | 107 +++++++++++++++- .../crypto/scheduler/scheduler_pmd_private.h | 120 +++++++++++++++++- .../crypto/scheduler/scheduler_roundrobin.c | 11 +- 6 files changed, 318 insertions(+), 40 deletions(-)
drivers/crypto/scheduler/scheduler_failover.c +12 -7
@@ -16,18 +16,19 @@struct fo_scheduler_qp_ctx {struct scheduler_worker primary_worker;struct scheduler_worker secondary_worker; +uint8_t primary_worker_index; +uint8_t secondary_worker_index;uint8_t deq_idx; }; static __rte_always_inline uint16_tfailover_worker_enqueue (struct scheduler_worker *worker, -struct rte_crypto_op **ops,uint16_t nb_ops) +struct rte_crypto_op **ops,uint16_t nb_ops,uint8_t index) { -uint16_t i, processed_ops; +uint16_t processed_ops; -for (i =0 ; i < nb_ops && i <4 ; i++) -rte_prefetch0 (ops[i]->sym->session); +scheduler_set_worker_session (ops, nb_ops, index); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, ops, nb_ops);
@@ -47,13 +48,14 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops)return 0 ; enqueued_ops =failover_worker_enqueue (&qp_ctx->primary_worker, - ops, nb_ops); + ops, nb_ops,PRIMARY_WORKER_IDX );if (enqueued_ops < nb_ops) enqueued_ops +=failover_worker_enqueue ( &qp_ctx->secondary_worker, &ops[enqueued_ops], - nb_ops - enqueued_ops); + nb_ops - enqueued_ops, +SECONDARY_WORKER_IDX );return enqueued_ops; }
@@ -94,7 +96,7 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) qp_ctx->deq_idx = (~qp_ctx->deq_idx) &WORKER_SWITCH_MASK ;if (nb_deq_ops == nb_ops) -return nb_deq_ops; + goto retrieve_session; worker = workers[qp_ctx->deq_idx];
@@ -104,6 +106,9 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) worker->nb_inflight_cops -= nb_deq_ops2; } +retrieve_session: +scheduler_retrieve_session (ops, nb_deq_ops + nb_deq_ops2); +return nb_deq_ops + nb_deq_ops2; }
drivers/crypto/scheduler/scheduler_multicore.c +17 -0
@@ -183,11 +183,19 @@ mc_scheduler_worker(struct rte_cryptodev *dev)while (!mc_ctx->stop_signal) {if (pending_enq_ops) { +scheduler_set_worker_session ( + &enq_ops[pending_enq_ops_idx], pending_enq_ops, + worker_idx); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, &enq_ops[pending_enq_ops_idx], pending_enq_ops); +if (processed_ops < pending_deq_ops) +scheduler_retrieve_session ( + &enq_ops[pending_enq_ops_idx + + processed_ops], + pending_deq_ops - processed_ops); pending_enq_ops -= processed_ops; pending_enq_ops_idx += processed_ops; inflight_ops += processed_ops;
@@ -195,9 +203,16 @@ mc_scheduler_worker(struct rte_cryptodev *dev) processed_ops =rte_ring_dequeue_burst (enq_ring, (void *)enq_ops,MC_SCHED_BUFFER_SIZE ,NULL );if (processed_ops) { +scheduler_set_worker_session (enq_ops, + processed_ops, worker_idx); pending_enq_ops_idx =rte_cryptodev_enqueue_burst ( worker->dev_id, worker->qp_id, enq_ops, processed_ops); +if (pending_enq_ops_idx < processed_ops) +scheduler_retrieve_session ( + enq_ops + pending_enq_ops_idx, + processed_ops - + pending_enq_ops_idx); pending_enq_ops = processed_ops - pending_enq_ops_idx; inflight_ops += pending_enq_ops_idx; }
@@ -214,6 +229,8 @@ mc_scheduler_worker(struct rte_cryptodev *dev) worker->dev_id, worker->qp_id, deq_ops, MC_SCHED_BUFFER_SIZE);if (processed_ops) { +scheduler_retrieve_session (deq_ops, + processed_ops); inflight_ops -= processed_ops;if (reordering_enabled) {uint16_t j;
drivers/crypto/scheduler/scheduler_pkt_size_distr.c +63 -21
149 lines not on this page · raw message
drivers/crypto/scheduler/scheduler_pmd_ops.c +101 -6
126 lines not on this page · raw message
drivers/crypto/scheduler/scheduler_pmd_private.h +119 -1
144 lines not on this page · raw message
drivers/crypto/scheduler/scheduler_roundrobin.c +6 -5
@@ -23,16 +23,17 @@ schedule_enqueue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) ((struct scheduler_qp_ctx *)qp)->private_qp_ctx;uint32_t worker_idx = rr_qp_ctx->last_enq_worker_idx;struct scheduler_worker *worker = &rr_qp_ctx->workers[worker_idx]; -uint16_t i, processed_ops; +uint16_t processed_ops;if (unlikely (nb_ops ==0 ))return 0 ; -for (i =0 ; i < nb_ops && i <4 ; i++) -rte_prefetch0 (ops[i]->sym->session); - +scheduler_set_worker_session (ops, nb_ops, worker_idx); processed_ops =rte_cryptodev_enqueue_burst (worker->dev_id, worker->qp_id, ops, nb_ops); +if (processed_ops < nb_ops) +scheduler_retrieve_session (ops + processed_ops, + nb_ops - processed_ops); worker->nb_inflight_cops += processed_ops;
@@ -86,7 +87,7 @@ schedule_dequeue(void *qp, struct rte_crypto_op **ops, uint16_t nb_ops) nb_deq_ops =rte_cryptodev_dequeue_burst (worker->dev_id, worker->qp_id, ops, nb_ops); - +scheduler_retrieve_session (ops, nb_deq_ops); last_worker_idx +=1 ; last_worker_idx %= rr_qp_ctx->nb_workers;
Signature · 1 line
2.25.1