compress/qat: im buffer too small - split op
v2 cover compress/qat: im buffer too small - split op
This patch implements a special way of buffer handling when internal QAT IM buffer is too small for Huffman dynamic compression operation. Instead of falling back to fixed compression, the operation is now split into multiple smaller dynamic compression requests (possible to execute on QAT) and their results are then combined and copied into the output buffer. This is not possible if any checksum calculation was requested - in such case the code falls back to fixed compression as before.
v2:
* various post-review small fixes
Adam Dybkowski (2):
compress/qat: im buffer too small - split op test/compress: im buffer too small - add unit tests
app/test/test_compressdev.c | 1251 ++++++++++++++++++++++-- doc/guides/compressdevs/qat_comp.rst | 3 - doc/guides/cryptodevs/qat.rst | 7 +- doc/guides/rel_notes/release_20_05.rst | 10 + drivers/common/qat/qat_qp.c | 223 ++++- drivers/common/qat/qat_qp.h | 3 + drivers/compress/qat/qat_comp.c | 474 ++++++++- drivers/compress/qat/qat_comp.h | 29 +- drivers/compress/qat/qat_comp_pmd.c | 27 +- 9 files changed, 1879 insertions(+), 148 deletions(-)
Signature · 1 line
2.17.1
PATCH v2 1/2 compress/qat: im buffer too small - split op
This patch implements a special way of buffer handling when internal QAT IM buffer is too small for Huffman dynamic compression operation. Instead of falling back to fixed compression, the operation is now split into multiple smaller dynamic compression requests (possible to execute on QAT) and their results are then combined and copied into the output buffer. This is not possible if any checksum calculation was requested - in such case the code falls back to fixed compression as before.
--- doc/guides/compressdevs/qat_comp.rst | 3 - doc/guides/cryptodevs/qat.rst | 7 +- doc/guides/rel_notes/release_20_05.rst | 10 + drivers/common/qat/qat_qp.c | 223 +++++++++++- drivers/common/qat/qat_qp.h | 3 + drivers/compress/qat/qat_comp.c | 474 +++++++++++++++++++++++-- drivers/compress/qat/qat_comp.h | 29 +- drivers/compress/qat/qat_comp_pmd.c | 27 +- 8 files changed, 702 insertions(+), 74 deletions(-)
doc/guides/compressdevs/qat_comp.rst +0 -3
@@ -42,9 +42,6 @@ Limitations from the RX queue must be done from one thread, but enqueues and dequeues may be done in different threads.) * No BSD support as BSD QAT kernel driver not available. -* When using Deflate dynamic huffman encoding for compression, the input size (op.src.length) - must be < CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE from the config file, - see :ref:`building_qat_config` for more details. * Stateful compression is not supported.
doc/guides/cryptodevs/qat.rst +5 -2
@@ -260,8 +260,11 @@ allocated while for GEN1 devices, 12 buffers are allocated, plus 1472 bytes over .. Note:: If the compressed output of a Deflate operation using Dynamic Huffman - Encoding is too big to fit in an intermediate buffer, then the - operation will fall back to fixed compression rather than failing the operation. + Encoding is too big to fit in an intermediate buffer, then the + operation will be split into smaller operations and their results will + be merged afterwards. + This is not possible if any checksum calculation was requested - in such + case the code falls back to fixed compression. To avoid this less performant case, applications should configure the intermediate buffer size to be larger than the expected input data size (compressed output size is usually unknown, so the only option is to make
doc/guides/rel_notes/release_20_05.rst +10 -0
@@ -81,6 +81,16 @@ New Features by making use of the event device capabilities. The event mode currently supports only inline IPsec protocol offload. +* **Added QAT intermediate buffer too small handling in QAT compression PMD.** + + Added a special way of buffer handling when internal QAT intermediate buffer + is too small for Huffman dynamic compression operation. Instead of falling + back to fixed compression, the operation is now split into multiple smaller + dynamic compression requests (possible to execute on QAT) and their results + are then combined and copied into the output buffer. This is not possible if + any checksum calculation was requested - in such case the code falls back to + fixed compression as before. + Removed Items -------------
drivers/common/qat/qat_qp.c +210 -13
263 lines not on this page · raw message
drivers/common/qat/qat_qp.h +3 -0
@@ -89,6 +89,9 @@ extern const struct qat_qp_hw_data qat_gen3_qps[][ADF_MAX_QPS_ON_ANY_SERVICE];uint16_t qat_enqueue_op_burst (void *qp,void **ops,uint16_t nb_ops); +uint16_t +qat_enqueue_comp_op_burst (void *qp,void **ops,uint16_t nb_ops); +uint16_t qat_dequeue_op_burst (void *qp,void **ops,uint16_t nb_ops);
drivers/compress/qat/qat_comp.c +437 -37
547 lines not on this page · raw message
drivers/compress/qat/qat_comp.h +28 -1
@@ -11,6 +11,7 @@#include <rte_compressdev_pmd.h> #include "qat_common.h" +#include "qat_qp.h" #include "icp_qat_hw.h" #include "icp_qat_fw_comp.h" #include "icp_qat_fw_la.h"
@@ -22,7 +23,7 @@#define ERR_CODE_QAT_COMP_WRONG_FW -99/* fallback to fixed compression threshold */ -#define QAT_FALLBACK_THLD ((uint32_t)(RTE_PMD_QAT_COMP_IM_BUFFER_SIZE / 1.1)) +#define QAT_FALLBACK_THLD ((uint32_t)(RTE_PMD_QAT_COMP_IM_BUFFER_SIZE / 1.3))#define QAT_MIN_OUT_BUF_SIZE 46
@@ -63,6 +64,24 @@ struct qat_comp_op_cookie {uint16_t dst_nb_elems;struct qat_sgl *qat_sgl_src_d;struct qat_sgl *qat_sgl_dst_d; +struct qat_qp *qp; +uint32_t cookie_index; + +/* QAT IM buffer too small handling: */ +uint8_t split_op; +uint8_t nb_children; + +/* used by the parent only */ +uint8_t nb_child_responses; +uint32_t total_consumed; +uint32_t total_produced; +const struct rte_memzone **dst_memzones; +struct rte_mbuf *dst_data; +uint32_t dst_data_offset; + +/* used by the child only */ +struct qat_comp_op_cookie *parent_cookie; +void *dest_buffer; };struct qat_comp_xform {
@@ -86,6 +105,14 @@ intqat_comp_build_request (void *in_op, uint8_t *out_msg, void *op_cookie, enum qat_device_gen qat_dev_gen __rte_unused); +int +qat_comp_build_multiple_requests (void *in_op,struct qat_qp *qp, +uint32_t parent_tail,int nb_descr); + +void +qat_comp_free_split_op_memzones (struct qat_comp_op_cookie *cookie, +unsigned int nb_children); +int qat_comp_process_response (void **op,uint8_t *resp,void *op_cookie,uint64_t *dequeue_err_count);
drivers/compress/qat/qat_comp_pmd.c +9 -18
@@ -146,6 +146,9 @@ qat_comp_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,struct qat_comp_op_cookie *cookie = qp->op_cookies[i]; + cookie->qp = qp; + cookie->cookie_index = i; + cookie->qat_sgl_src_d =rte_zmalloc_socket (NULL ,sizeof (struct qat_sgl ) + sizeof(struct qat_flat_buf) *
@@ -560,20 +563,6 @@ qat_comp_dev_info_get(struct rte_compressdev *dev, } } -static uint16_t -qat_comp_pmd_enqueue_op_burst (void *qp,struct rte_comp_op **ops, -uint16_t nb_ops) -{ -return qat_enqueue_op_burst (qp, (void **)ops, nb_ops); -} - -static uint16_t -qat_comp_pmd_dequeue_op_burst (void *qp,struct rte_comp_op **ops, -uint16_t nb_ops) -{ -return qat_dequeue_op_burst (qp, (void **)ops, nb_ops); -} - static uint16_t qat_comp_pmd_enq_deq_dummy_op_burst(void *qp __rte_unused, struct rte_comp_op **ops __rte_unused,
@@ -603,7 +592,7 @@ static struct rte_compressdev_ops compress_qat_dummy_ops = { };static uint16_t -qat_comp_pmd_dequeue_frst_op_burst (void *qp,struct rte_comp_op **ops, +qat_comp_pmd_dequeue_first_op_burst (void *qp,struct rte_comp_op **ops,uint16_t nb_ops) {uint16_t ret =qat_dequeue_op_burst (qp, (void **)ops, nb_ops);
@@ -623,7 +612,8 @@ qat_comp_pmd_dequeue_frst_op_burst(void *qp, struct rte_comp_op **ops, }else { tmp_qp->qat_dev->comp_dev->compressdev->dequeue_burst = - qat_comp_pmd_dequeue_op_burst; + (compressdev_dequeue_pkt_burst_t ) + qat_dequeue_op_burst; } }return ret;
@@ -698,8 +688,9 @@ qat_comp_dev_create(struct qat_pci_device *qat_pci_dev, compressdev->dev_ops = &compress_qat_ops; - compressdev->enqueue_burst = qat_comp_pmd_enqueue_op_burst; - compressdev->dequeue_burst = qat_comp_pmd_dequeue_frst_op_burst; + compressdev->enqueue_burst = (compressdev_enqueue_pkt_burst_t ) + qat_enqueue_comp_op_burst; + compressdev->dequeue_burst = qat_comp_pmd_dequeue_first_op_burst; compressdev->feature_flags =RTE_COMPDEV_FF_HW_ACCELERATED ;
Signature · 1 line
2.17.1
Quoted 18 lines
-----Original Message-----
From: Dybkowski, AdamX <[email protected]>
Sent: Wednesday, April 8, 2020 1:51 PM
To: [email protected]; Trahe, Fiona <[email protected]>; [email protected]
Cc: Dybkowski, AdamX <[email protected]>
Subject: [PATCH v2 1/2] compress/qat: im buffer too small - split opThis patch implements a special way of buffer handling when internal QAT IM buffer is too small for Huffman dynamic compression operation. Instead of falling back to fixed compression, the operation is now split into multiple smaller dynamic compression requests (possible to execute on QAT) and their results are then combined and copied into the output buffer. This is not possible if any checksum calculation was requested - in such case the code falls back to fixed compression as before.
Quoted 1 line
Signed-off-by Adam Dybkowski <[email protected]>
PATCH v2 2/2 test/compress: im buffer too small - add unit tests
In reply to Adam DybkowskiThis patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite.
--- app/test/test_compressdev.c | 1251 ++++++++++++++++++++++++++++++++--- 1 file changed, 1177 insertions(+), 74 deletions(-)
app/test/test_compressdev.c +1177 -74
@@ -1,10 +1,11 @@/* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2018 - 2019 Intel Corporation */ -#include <string.h> -#include <zlib.h> #include <math.h> +#include <stdio.h> #include <stdlib.h> +#include <string.h> +#include <zlib.h> #include <unistd.h> #include <rte_cycles.h>
@@ -18,6 +19,8 @@#include "test.h" #define DIV_CEIL (a, b) ((a) / (b) + ((a) % (b) != 0)) +#define ENDS_WITH (s, suffix) (strlen(s) >= strlen(suffix) && \ + !strcmp(suffix, s + strlen(s) - strlen(suffix)))#define DEFAULT_WINDOW_SIZE 15#define DEFAULT_MEM_LEVEL 8
@@ -30,6 +33,7 @@* due to the compress block headers */ #define COMPRESS_BUF_SIZE_RATIO 1.3 +#define COMPRESS_BUF_SIZE_RATIO_DISABLED 1.0#define COMPRESS_BUF_SIZE_RATIO_OVERFLOW 0.2#define NUM_LARGE_MBUFS 16#define SMALL_SEG_SIZE 256
@@ -52,6 +56,25 @@#define NUM_BIG_MBUFS 4#define BIG_DATA_TEST_SIZE (MAX_DATA_MBUF_SIZE * NUM_BIG_MBUFS / 2) +/* constants for "im buffer" tests start here */ +#define IM_NUM_BIG_MBUFS (512 + 1) +#define IM_BIG_DATA_TEST_SIZE (MAX_DATA_MBUF_SIZE * 2) +/* number of mbufs lower than number of inflight ops */ +#define IM_BUF_NUM_MBUFS 3 +/* above threshold (QAT_FALLBACK_THLD) and below max mbuf size */ +#define IM_BUF_DATA_TEST_SIZE_LB 59600 +/* data size smaller than the queue capacity */ +#define IM_BUF_DATA_TEST_SIZE_SGL (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS) +/* number of mbufs bigger than number of inflight ops */ +#define IM_BUF_NUM_MBUFS_OVER (NUM_MAX_INFLIGHT_OPS + 1) +/* data size bigger than the queue capacity */ +#define IM_BUF_DATA_TEST_SIZE_OVER (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS_OVER) +/* number of mid-size mbufs */ +#define IM_BUF_NUM_MBUFS_MID ((NUM_MAX_INFLIGHT_OPS / 3) + 1) +/* capacity of mid-size mbufs */ +#define IM_BUF_DATA_TEST_SIZE_MID (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS_MID) + +const char * huffman_type_strings[] = { [RTE_COMP_HUFFMAN_DEFAULT ] ="PMD default" ,
@@ -78,6 +101,11 @@ enum overflow_test {OVERFLOW_ENABLED }; +enum ratio_switch { +RATIO_DISABLED , +RATIO_ENABLED +}; +enum operation_type {OPERATION_COMPRESSION ,OPERATION_DECOMPRESSION
@@ -88,6 +116,7 @@ struct priv_op_data { };struct comp_testsuite_params { +const char *suite_name;struct rte_mempool *large_mbuf_pool;struct rte_mempool *small_mbuf_pool;struct rte_mempool *big_mbuf_pool;
@@ -123,6 +152,7 @@ struct test_data_params {const struct rte_memzone *uncompbuf_memzone;/* overflow test activation */ enum overflow_test overflow; +enum ratio_switch ratio; };struct test_private_arrays {
@@ -141,6 +171,26 @@ struct test_private_arrays {static struct comp_testsuite_params testsuite_params = {0 }; + +static uint8_t +is_qat_specific_testsuite (void ) +{ +return strstr (testsuite_params.suite_name,"QAT" ) !=NULL ; +} + +static unsigned int +get_num_big_mbufs (void ) +{ +return is_qat_specific_testsuite () ?IM_NUM_BIG_MBUFS :NUM_BIG_MBUFS ; +} + +static unsigned int +get_big_data_test_size (void ) +{ +return is_qat_specific_testsuite () ? +IM_BIG_DATA_TEST_SIZE :BIG_DATA_TEST_SIZE ; +} +static void testsuite_teardown (void ) {
@@ -212,7 +262,7 @@ testsuite_setup(void)/* Create mempool with big buffers for SGL testing */ ts_params->big_mbuf_pool =rte_pktmbuf_pool_create ("big_mbuf_pool" , -NUM_BIG_MBUFS +1 , +get_num_big_mbufs () +1 ,CACHE_SIZE ,0 ,MAX_MBUF_SEGMENT_SIZE ,rte_socket_id ());
@@ -316,6 +366,8 @@ test_compressdev_invalid_configuration(void) };struct rte_compressdev_info dev_info; +RTE_LOG (INFO ,USER1 ,"This is a negative test, errors are expected\n" ); +/* Invalid configuration with 0 queue pairs */ memcpy (&invalid_config, &valid_config,sizeof (struct rte_compressdev_config ));
@@ -691,7 +743,7 @@ prepare_sgl_bufs(const char *test_buf, struct rte_mbuf *head_buf,if (data_ptr !=NULL ) {/* Copy characters without NULL terminator */ -strncpy (buf_ptr, data_ptr, data_size); +memcpy (buf_ptr, data_ptr, data_size); data_ptr += data_size; } remaining_data -= data_size;
@@ -731,7 +783,7 @@ prepare_sgl_bufs(const char *test_buf, struct rte_mbuf *head_buf, }if (data_ptr !=NULL ) {/* Copy characters without NULL terminator */ -strncpy (buf_ptr, data_ptr, data_size); +memcpy (buf_ptr, data_ptr, data_size); data_ptr += data_size; } remaining_data -= data_size;
@@ -760,17 +812,20 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops, {uint16_t num_enqd, num_deqd, num_total_deqd;unsigned int deqd_retries =0 ; +int res =0 ;/* Enqueue and dequeue all operations */ num_enqd =rte_compressdev_enqueue_burst (0 ,0 , ops, num_bufs);if (num_enqd < num_bufs) {RTE_LOG (ERR ,USER1 ,"Some operations could not be enqueued\n" ); -return -1 ; + res =-1 ; } +/* dequeue ops even on error (same number of ops as was enqueued) */ + num_total_deqd =0 ; - do { +while (num_total_deqd < num_enqd) {/* * If retrying a dequeue call, wait for 10 ms to allow * enough time to the driver to process the operations
@@ -783,7 +838,8 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops,if (deqd_retries ==MAX_DEQD_RETRIES ) {RTE_LOG (ERR ,USER1 ,"Not all operations could be dequeued\n" ); -return -1 ; + res =-1 ; +break ; }usleep (DEQUEUE_WAIT_TIME ); }
@@ -792,9 +848,9 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops, num_total_deqd += num_deqd; deqd_retries++; - }while (num_total_deqd < num_enqd); + } -return 0 ; +return res; }/**
@@ -956,7 +1012,9 @@ test_mbufs_calculate_data_size(/* local variables: */ uint32_t data_size;struct priv_op_data *priv_data; -float ratio; +float ratio_val; +enum ratio_switch ratio = test_data->ratio; +uint8_t not_zlib_compr;/* true if zlib isn't current compression dev */ enum overflow_test overflow = test_data->overflow;
@@ -973,13 +1031,16 @@ test_mbufs_calculate_data_size( not_zlib_compr = (test_data->zlib_dir ==ZLIB_DECOMPRESS || test_data->zlib_dir ==ZLIB_NONE ); - ratio = (not_zlib_compr && + ratio_val = (ratio ==RATIO_ENABLED ) ? +COMPRESS_BUF_SIZE_RATIO : +COMPRESS_BUF_SIZE_RATIO_DISABLED ; + + ratio_val = (not_zlib_compr && (overflow ==OVERFLOW_ENABLED )) ?COMPRESS_BUF_SIZE_RATIO_OVERFLOW : -COMPRESS_BUF_SIZE_RATIO ; - - data_size =strlen (test_bufs[i]) * ratio; + ratio_val; + data_size =strlen (test_bufs[i]) * ratio_val; }else { priv_data = (struct priv_op_data *) (ops_processed[i] +1 );
@@ -1085,6 +1146,9 @@ test_setup_output_bufs( }else {for (i =0 ; i < num_bufs; i++) { +enum rte_comp_huffman comp_huffman = + ts_params->def_comp_xform->compress.deflate.huffman; +/* data size calculation */ data_size = test_mbufs_calculate_data_size( op_type,
@@ -1094,6 +1158,11 @@ test_setup_output_bufs( test_data, i); +if (comp_huffman !=RTE_COMP_HUFFMAN_DYNAMIC ) { +if (op_type ==OPERATION_DECOMPRESSION ) + data_size *=COMPRESS_BUF_SIZE_RATIO ; + } +/* data allocation */ if (buff_type ==SGL_BOTH || buff_type ==LB_TO_SGL ) { ret = prepare_sgl_bufs(NULL, current_bufs[i],
@@ -1192,6 +1261,11 @@ test_deflate_comp_run(const struct interim_data_params *int_data, ops[i]->src.length =rte_pktmbuf_pkt_len (uncomp_bufs[i]); ops[i]->dst.offset =0 ; +RTE_LOG (DEBUG ,USER1 , +"Uncompressed buffer length = %u compressed buffer length = %u" , +rte_pktmbuf_pkt_len (uncomp_bufs[i]), +rte_pktmbuf_pkt_len (comp_bufs[i])); +if (operation_type ==RTE_COMP_OP_STATELESS ) { ops[i]->flush_flag =RTE_COMP_FLUSH_FINAL ; }else {
@@ -1313,6 +1387,7 @@ test_deflate_comp_run(const struct interim_data_params *int_data,if (ret_status <0 )for (i =0 ; i < num_bufs; i++) {rte_comp_op_free (ops[i]); + ops[i] =NULL ; ops_processed[i] =NULL ; }
@@ -1431,7 +1506,7 @@ test_deflate_comp_finalize(const struct interim_data_params *int_data, }RTE_LOG (ERR ,USER1 , -"Some operations were not successful\n" ); +"Comp: Some operations were not successful\n" );return -1 ; } priv_data = (struct priv_op_data *)(ops_processed[i] +1 );
@@ -1490,6 +1565,7 @@ test_deflate_decomp_run(const struct interim_data_params *int_data,/* from test_priv_data: */ struct rte_mbuf **uncomp_bufs = test_priv_data->uncomp_bufs; +struct rte_mbuf **comp_bufs = test_priv_data->comp_bufs;struct rte_comp_op **ops = test_priv_data->ops;struct rte_comp_op **ops_processed = test_priv_data->ops_processed;void **priv_xforms = test_priv_data->priv_xforms;
@@ -1510,7 +1586,7 @@ test_deflate_decomp_run(const struct interim_data_params *int_data,/* Source buffer is the compressed data from the previous operations */ for (i =0 ; i < num_bufs; i++) { - ops[i]->m_src = ops_processed[i]->m_dst; + ops[i]->m_src = comp_bufs[i]; ops[i]->m_dst = uncomp_bufs[i]; ops[i]->src.offset =0 ;/*
@@ -1740,6 +1816,10 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE || ops_processed[i]->status == RTE_COMP_OP_STATUS_SUCCESS)) { + +RTE_LOG (DEBUG ,USER1 , +".............RECOVERABLE\n" ); +/* collect the output into all_decomp_data */ const void *ptr = rte_pktmbuf_read( ops_processed[i]->m_dst,
@@ -1777,7 +1857,6 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, ops[i]->src.length -= ops_processed[i]->consumed;/* repeat the operation */ -//goto next_step; return 2 ; }else {/* Compare the original stream with the */
@@ -1808,7 +1887,8 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, }else if (ops_processed[i]->status !=RTE_COMP_OP_STATUS_SUCCESS ) {RTE_LOG (ERR ,USER1 , -"Some operations were not successful\n" ); +"Decomp: Some operations were not successful, status = %u\n" , + ops_processed[i]->status);return -1 ; } priv_data = (struct priv_op_data *)(ops_processed[i] +1 );
@@ -1986,7 +2066,6 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, "Compress device does not support DEFLATE\n");return -1 ; } -//test_objects_init(&test_priv_data, num_bufs); /* Prepare the source mbufs with the data */ ret =test_setup_com_bufs (int_data, test_data, &test_priv_data);
@@ -1995,6 +2074,8 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, goto exit; } +RTE_LOG (DEBUG ,USER1 ,"<<< COMPRESSION >>>\n" ); +/* COMPRESSION */ /* Prepare output (destination) mbufs for compressed data */
@@ -2031,6 +2112,8 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data,/* DECOMPRESSION */ +RTE_LOG (DEBUG ,USER1 ,"<<< DECOMPRESSION >>>\n" ); +/* Prepare output (destination) mbufs for decompressed data */ ret = test_setup_output_bufs( OPERATION_DECOMPRESSION,
@@ -2096,7 +2179,6 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, priv_xforms[i] =NULL ; } } -for (i =0 ; i < num_bufs; i++) {rte_pktmbuf_free (uncomp_bufs[i]);rte_pktmbuf_free (comp_bufs[i]);
@@ -2152,7 +2234,8 @@ test_compressdev_deflate_stateless_fixed(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };for (i =0 ; i <RTE_DIM (compress_test_bufs); i++) {
@@ -2223,7 +2306,8 @@ test_compressdev_deflate_stateless_dynamic(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };for (i =0 ; i <RTE_DIM (compress_test_bufs); i++) {
@@ -2278,7 +2362,8 @@ test_compressdev_deflate_stateless_multi_op(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Compress with compressdev, decompress with Zlib */
@@ -2332,7 +2417,8 @@ test_compressdev_deflate_stateless_multi_level(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };for (i =0 ; i <RTE_DIM (compress_test_bufs); i++) {
@@ -2422,7 +2508,8 @@ test_compressdev_deflate_stateless_multi_xform(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Compress with compressdev, decompress with Zlib */
@@ -2471,7 +2558,8 @@ test_compressdev_deflate_stateless_sgl(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };for (i =0 ; i <RTE_DIM (compress_test_bufs); i++) {
@@ -2582,7 +2670,8 @@ test_compressdev_deflate_stateless_checksum(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Check if driver supports crc32 checksum and test */
@@ -2700,7 +2789,8 @@ test_compressdev_out_of_space_buffer(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 1, /* run out-of-space test */ .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Compress with compressdev, decompress with Zlib */ test_data.zlib_dir =ZLIB_DECOMPRESS ;
@@ -2742,7 +2832,7 @@ test_compressdev_deflate_stateless_dynamic_big(void)struct comp_testsuite_params *ts_params = &testsuite_params;uint16_t i =0 ;int ret; -int j; +unsigned int j;const struct rte_compressdev_capabilities *capab;char *test_buffer =NULL ;
@@ -2755,7 +2845,7 @@ test_compressdev_deflate_stateless_dynamic_big(void)if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 )return -ENOTSUP ; - test_buffer =rte_malloc (NULL ,BIG_DATA_TEST_SIZE ,0 ); + test_buffer =rte_malloc (NULL ,get_big_data_test_size (),0 );if (test_buffer ==NULL ) {RTE_LOG (ERR ,USER1 ,"Can't allocate buffer for big-data\n" );
@@ -2778,7 +2868,8 @@ test_compressdev_deflate_stateless_dynamic_big(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 1, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED }; ts_params->def_comp_xform->compress.deflate.huffman =
@@ -2786,9 +2877,9 @@ test_compressdev_deflate_stateless_dynamic_big(void)/* fill the buffer with data based on rand. data */ srand (BIG_DATA_TEST_SIZE ); -for (j =0 ; j <BIG_DATA_TEST_SIZE -1 ; ++j) +for (j =0 ; j <get_big_data_test_size () -1 ; ++j) test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; - test_buffer[BIG_DATA_TEST_SIZE -1 ] =0 ; + test_buffer[get_big_data_test_size () -1 ] =0 ;/* Compress with compressdev, decompress with Zlib */ test_data.zlib_dir =ZLIB_DECOMPRESS ;
@@ -2843,7 +2934,8 @@ test_compressdev_deflate_stateful_decomp(void) .big_data = 0, .decompress_output_block_size = 2000, .decompress_steps_max = 4, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Compress with Zlib, decompress with compressdev */
@@ -2926,7 +3018,8 @@ test_compressdev_deflate_stateful_decomp_checksum(void) .big_data = 0, .decompress_output_block_size = 2000, .decompress_steps_max = 4, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Check if driver supports crc32 checksum and test */
@@ -3139,7 +3232,8 @@ test_compressdev_deflate_stateless_fixed_oos_recoverable(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_ENABLED + .overflow = OVERFLOW_ENABLED, + .ratio = RATIO_ENABLED };for (i =0 ; i <RTE_DIM (compress_test_bufs); i++) {
@@ -3176,47 +3270,1056 @@ test_compressdev_deflate_stateless_fixed_oos_recoverable(void)return ret; } -static struct unit_test_suite compressdev_testsuite = { - .suite_name ="compressdev unit test suite" , - .setup = testsuite_setup, - .teardown = testsuite_teardown, - .unit_test_cases = { -TEST_CASE_ST (NULL ,NULL , - test_compressdev_invalid_configuration), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_fixed), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_dynamic), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_dynamic_big), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_op), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_level), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_xform), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_sgl), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_checksum), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_out_of_space_buffer), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateful_decomp), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateful_decomp_checksum), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_external_mbufs), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_fixed_oos_recoverable), -TEST_CASES_END ()/**< NULL terminate unit test array */ +static int +test_compressdev_deflate_im_buffers_LB_1op (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_LB ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for 'im buffer' test\n" ); +return TEST_FAILED ; } -}; + +struct interim_data_params int_data = { + (const char *const *)&test_buffer, +1 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , +/* must be LB to SGL, +* input LB buffer reaches its maximum, +* if ratio 1.3 than another mbuf must be +* created and attached +*/ + .buff_type =LB_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_LB ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_LB -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +}static int -test_compressdev (void ) +test_compressdev_deflate_im_buffers_LB_2ops_first (void ) { -return unit_test_suite_runner (&compressdev_testsuite); +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_LB ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for 'im buffer' test\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = test_buffer; + test_buffers[1 ] = compress_test_bufs[0 ]; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =LB_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_LB ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_LB -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_2ops_second (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_LB ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for 'im buffer' test\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =LB_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_LB ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_LB -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_3ops (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[3 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_LB ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for 'im buffer' test\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + test_buffers[2 ] = compress_test_bufs[1 ]; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +3 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =LB_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_LB ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_LB -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_4ops (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[4 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_LB ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for 'im buffer' test\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + test_buffers[2 ] = compress_test_bufs[1 ]; + test_buffers[3 ] = test_buffer; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +4 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =LB_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_LB ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_LB -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_1op (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_SGL ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + +struct interim_data_params int_data = { + (const char *const *)&test_buffer, +1 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_SGL ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_SGL -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_2ops_first (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_SGL ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = test_buffer; + test_buffers[1 ] = compress_test_bufs[0 ]; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_SGL ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_SGL -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_2ops_second (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_SGL ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_SGL ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_SGL -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_3ops (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[3 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_SGL ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + test_buffers[2 ] = compress_test_bufs[1 ]; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +3 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_SGL ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_SGL -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_4ops (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[4 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_SGL ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + test_buffers[2 ] = compress_test_bufs[1 ]; + test_buffers[3 ] = test_buffer; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +4 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_SGL ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_SGL -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_over_1op (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; + +RTE_LOG (INFO ,USER1 ,"This is a negative test, errors are expected\n" ); + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_OVER ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + +struct interim_data_params int_data = { + (const char *const *)&test_buffer, +1 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_OVER ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_OVER -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_SUCCESS ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); + +return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_over_2ops_first (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + +RTE_LOG (INFO ,USER1 ,"This is a negative test, errors are expected\n" ); + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_OVER ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = test_buffer; + test_buffers[1 ] = compress_test_bufs[0 ]; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_OVER ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_OVER -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_SUCCESS ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_over_2ops_second (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + +RTE_LOG (INFO ,USER1 ,"This is a negative test, errors are expected\n" ); + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_OVER ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_OVER ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_OVER -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_SUCCESS ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static struct unit_test_suite compressdev_testsuite = { + .suite_name ="compressdev unit test suite" , + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { +TEST_CASE_ST (NULL ,NULL , + test_compressdev_invalid_configuration), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_fixed), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_dynamic), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_dynamic_big), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_op), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_level), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_xform), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_sgl), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_checksum), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_out_of_space_buffer), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateful_decomp), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateful_decomp_checksum), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_external_mbufs), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_fixed_oos_recoverable), +TEST_CASES_END ()/**< NULL terminate unit test array */ + } +}; + +/**********************************/ + +static struct unit_test_suite compressdev_testsuite_qat = { + .suite_name ="compressdev unit test suite for QAT PMD" , + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { +/* Positive test cases for IM buffer handling verification */ + +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_1op), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_2ops_first), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_2ops_second), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_3ops), + +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_4ops), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_1op), + +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_2ops_first), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_2ops_second), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_3ops), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_4ops), + +/* Negative test cases for IM buffer handling verification */ + +/* For this test huge mempool is necessary. +* It tests one case: +* only one op containing big amount of data, so that +* number of requested descriptors higher than number +* of available descriptors (128) +*/ +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_1op), + +/* For this test huge mempool is necessary. +* 2 ops. First op contains big amount of data: +* number of requested descriptors higher than number +* of available descriptors (128), the second op is +* relatively small. In this case both ops are rejected +*/ +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_2ops_first), + +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_2ops_second), + +TEST_CASES_END ()/**< NULL terminate unit test array */ + } +}; + +static int +test_compressdev (void ) +{ + testsuite_params.suite_name = compressdev_testsuite.suite_name; +return unit_test_suite_runner (&compressdev_testsuite); +} + +static int +test_compressdev_qat_specific (void ) +{ +/* All tests are run on device 0. Check if it's QAT PMD. */ +if (rte_compressdev_count () ==0 || + !ENDS_WITH (rte_compressdev_name_get (0 ),"qat_comp" )) { +RTE_LOG (ERR ,USER1 ,"QAT PMD must be loaded. Check if " +"CONFIG_RTE_LIBRTE_PMD_QAT is enabled " +"in config file to run this testsuite.\n" ); +return TEST_FAILED ; + } + + testsuite_params.suite_name = compressdev_testsuite_qat.suite_name; +return unit_test_suite_runner (&compressdev_testsuite_qat); }REGISTER_TEST_COMMAND (compressdev_autotest, test_compressdev); +REGISTER_TEST_COMMAND (compressdev_qat_specific, test_compressdev_qat_specific);
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
Quoted 9 lines
-----Original Message-----
From: Dybkowski, AdamX <[email protected]>
Sent: Wednesday, April 8, 2020 1:51 PM
To: [email protected]; Trahe, Fiona <[email protected]>; [email protected]
Cc: Dybkowski, AdamX <[email protected]>
Subject: [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
Quoted 6 lines
This patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite.
Signed-off-by Adam Dybkowski <[email protected]>
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
In reply to Adam DybkowskiHi Fiona/Adam,
Quoted 7 lines
This patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite.
Signed-off-by Adam Dybkowski <[email protected]>---
Why do we need to have separate testsuite for QAT?
Can't we have a single one and based on capability of the driver, Determine which tests need to be skipped in case they are not supported. This would create a mess in the longer run just like cryptodev.
Please fix this, we cannot take this patch as is.
Quoted 131 lines
app/test/test_compressdev.c +30
-0+ +static struct unit_test_suite compressdev_testsuite = { + .suite_name ="compressdev unit test suite" , + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { +TEST_CASE_ST (NULL ,NULL , + test_compressdev_invalid_configuration), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_fixed), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_dynamic), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_dynamic_big), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_op), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_level), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_xform), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_sgl), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_checksum), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_out_of_space_buffer), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateful_decomp), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, +
test_compressdev_deflate_stateful_decomp_checksum),app/test/test_compressdev.c +23
-0+TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_external_mbufs), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_fixed_oos_recoverable), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +/**********************************/ + +static struct unit_test_suite compressdev_testsuite_qat = { + .suite_name ="compressdev unit test suite for QAT PMD" , + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { +/* Positive test cases for IM buffer handling verification */ + +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_1op), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_2ops_first), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, +
test_compressdev_deflate_im_buffers_LB_2ops_second),app/test/test_compressdev.c +12
-0+TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_3ops), + +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_4ops), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_1op), + +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_2ops_first), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, +
test_compressdev_deflate_im_buffers_SGL_2ops_second),app/test/test_compressdev.c +24
-0+TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_3ops), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_4ops), + +/* Negative test cases for IM buffer handling verification */ + +/* For this test huge mempool is necessary. +* It tests one case: +* only one op containing big amount of data, so that +* number of requested descriptors higher than number +* of available descriptors (128) +*/ +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_1op), + +/* For this test huge mempool is necessary. +* 2 ops. First op contains big amount of data: +* number of requested descriptors higher than number +* of available descriptors (128), the second op is +* relatively small. In this case both ops are rejected +*/ + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, +
test_compressdev_deflate_im_buffers_SGL_over_2ops_first),app/test/test_compressdev.c +3
-0+ + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, +
test_compressdev_deflate_im_buffers_SGL_over_2ops_second),app/test/test_compressdev.c +18
-0+ + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static int +test_compressdev (void ) +{ + testsuite_params.suite_name = compressdev_testsuite.suite_name; +return unit_test_suite_runner (&compressdev_testsuite); +} + +static int +test_compressdev_qat_specific(void) +{ + /* All tests are run on device 0. Check if it's QAT PMD. */ + if (rte_compressdev_count() == 0 || + !ENDS_WITH(rte_compressdev_name_get(0),"
qat_comp")) {app/test/test_compressdev.c +9
-0+RTE_LOG (ERR ,USER1 ,"QAT PMD must be loaded. Check if " +"CONFIG_RTE_LIBRTE_PMD_QAT is enabled " +"in config file to run this testsuite.\n" ); +return TEST_FAILED ; + } + + testsuite_params.suite_name = compressdev_testsuite_qat.suite_name; +return unit_test_suite_runner (&compressdev_testsuite_qat); }REGISTER_TEST_COMMAND (compressdev_autotest, test_compressdev); +REGISTER_TEST_COMMAND(compressdev_qat_specific,
test_compressdev_qat_specific);
--
2.17.1
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
Hi Akhil,
Quoted 18 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Wednesday, April 15, 2020 7:36 PM
To: Dybkowski, AdamX <[email protected]>; [email protected]; Trahe, Fiona <[email protected]>
Cc: Shally Verma <[email protected]>
Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit testsHi Fiona/Adam,
Quoted 7 lines
This patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite.
Signed-off-by Adam Dybkowski <[email protected]>---
Quoted 6 lines
Why do we need to have separate testsuite for QAT?
Can't we have a single one and based on capability of the driver, Determine which tests need to be skipped in case they are not supported. This would create a mess in the longer run just like cryptodev.Please fix this, we cannot take this patch as is.
[Fiona] Yes, I understand your concern and we considered including in the main suite. However these tests are not based on something that can be
checked in capabilities. They are tests to hone in on a specific corner case
based on a QAT limitation in its intermediate buffer size. So some of the
tests are to validate that the recent changes we made in the PMD correctly
work around that limitation, but other tests are negative and expected to fail
as provoking a corner-case that still exists. Other devices would probably not fail the same tests. This was why we felt it better to have a qat-specific suite.
We would expect any future tests to go into the main suite, so don't think this should cause a mess in the future.
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
Hi Fiona,
Quoted 21 lines
Hi Akhil,
Quoted 11 lines
Hi Fiona/Adam,
Quoted 7 lines
This patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite.
Signed-off-by Adam Dybkowski <[email protected]>---
Quoted 6 lines
Why do we need to have separate testsuite for QAT?
Can't we have a single one and based on capability of the driver, Determine which tests need to be skipped in case they are not supported. This would create a mess in the longer run just like cryptodev.Please fix this, we cannot take this patch as is.
Quoted 9 lines
[Fiona] Yes, I understand your concern and we considered including in the main suite.
However these tests are not based on something that can be
checked in capabilities. They are tests to hone in on a specific corner case based on a QAT limitation in its intermediate buffer size. So some of the tests are to validate that the recent changes we made in the PMD correctly work around that limitation, but other tests are negative and expected to fail as provoking a corner-case that still exists. Other devices would probably not fail the same tests.
Does that mean that all PMDs will pass with the newly added testcase which is for A corner case in QAT. If that is the case what is the issue in adding that in the main Test suite. It will get passed in all PMDs, isn't it? Am I missing something?
I believe we should not have PMD specific test suites, rather it should be based on Capabilities to identify the cases which should be run for that particular PMD.
Quoted 6 lines
This was why we felt it better to have a qat-specific suite.
We would expect any future tests to go into the main suite, so don't think this should cause a
mess in the future.
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
Quoted 42 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Thursday, April 16, 2020 11:25 AM
To: Trahe, Fiona <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]
Cc: Shally Verma <[email protected]>
Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit testsHi Fiona,
Quoted 21 lines
Hi Akhil,
Quoted 11 lines
Hi Fiona/Adam,
Quoted 7 lines
This patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite.
Signed-off-by Adam Dybkowski <[email protected]>---
Quoted 6 lines
Why do we need to have separate testsuite for QAT?
Can't we have a single one and based on capability of the driver, Determine which tests need to be skipped in case they are not supported. This would create a mess in the longer run just like cryptodev.Please fix this, we cannot take this patch as is.
Quoted 9 lines
[Fiona] Yes, I understand your concern and we considered including in the main suite.
However these tests are not based on something that can be
checked in capabilities. They are tests to hone in on a specific corner case based on a QAT limitation in its intermediate buffer size. So some of the tests are to validate that the recent changes we made in the PMD correctly work around that limitation, but other tests are negative and expected to fail as provoking a corner-case that still exists. Other devices would probably not fail the same tests.
Quoted 6 lines
Does that mean that all PMDs will pass with the newly added testcase which is for A corner case in QAT. If that is the case what is the issue in adding that in the main Test suite. It will get passed in all PMDs, isn't it? Am I missing something?
I believe we should not have PMD specific test suites, rather it should be based on Capabilities to identify the cases which should be run for that particular PMD.
[Fiona] yes, several of the cases should pass on all PMDs.
So we could move those into the main suite.
But what to do about the negative tests?
Example: If a very large data buffer is passed to QAT to compress with dyn compression, it will get split in the PMD into many smaller requests to the hardware. However if the number of requests is bigger than can fit on the qp then this will never succeed. The test validates that the PMD behaves appropriately in this expected error case. That same case would probably not have an error on another device. Maybe we should just leave out such negative tests, but I find them useful as they validate the known behaviour. The buffer size used in the test is based on the known size QAT can handle and the corner case in which QAT will return an error.
I see 4 options to handle this:
1. Leave out those tests
2. Use a qat-specific test suite only for negative cases which are constructed based on specific qat internal meta-data.
3. Include the negative tests in the main suite, but only run them on QAT (by checking driver type)
4. include the negative tests in the main suite, run them on all, expecting a FAIL from QAT and a PASS from other devices.
My preference is for 2.
But up to you.
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
Quoted 60 lines · Fiona
Quoted 42 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Thursday, April 16, 2020 11:25 AM
To: Trahe, Fiona <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]
Cc: Shally Verma <[email protected]>
Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit testsHi Fiona,
Quoted 21 lines
Hi Akhil,
Quoted 11 lines
Hi Fiona/Adam,
Quoted 7 lines
This patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite.
Signed-off-by Adam Dybkowski <[email protected]>---
Quoted 6 lines
Why do we need to have separate testsuite for QAT?
Can't we have a single one and based on capability of the driver, Determine which tests need to be skipped in case they are not supported. This would create a mess in the longer run just like cryptodev.Please fix this, we cannot take this patch as is.
Quoted 9 lines
[Fiona] Yes, I understand your concern and we considered including in the main suite.
However these tests are not based on something that can be
checked in capabilities. They are tests to hone in on a specific corner case based on a QAT limitation in its intermediate buffer size. So some of the tests are to validate that the recent changes we made in the PMD correctly work around that limitation, but other tests are negative and expected to fail as provoking a corner-case that still exists. Other devices would probably not fail the same tests.Quoted 6 lines
Does that mean that all PMDs will pass with the newly added testcase which is for A corner case in QAT. If that is the case what is the issue in adding that in the main Test suite. It will get passed in all PMDs, isn't it? Am I missing something?
I believe we should not have PMD specific test suites, rather it should be based on Capabilities to identify the cases which should be run for that particular PMD.
[Fiona] yes, several of the cases should pass on all PMDs.
So we could move those into the main suite.
But what to do about the negative tests?
Example: If a very large data buffer is passed to QAT to compress with dyn compression, it will get split in the PMD into many smaller requests to the hardware. However if the number of requests is bigger than can fit on the qp then this will never succeed. The test validates that the PMD behaves appropriately in this expected error case. That same case would probably not have an error on another device. Maybe we should just leave out such negative tests, but I find them useful as they validate the known behaviour. The buffer size used in the test is based on the known size QAT can handle and the corner case in which QAT will return an error.
Quoted 9 lines
I see 4 options to handle this:
1. Leave out those tests
2. Use a qat-specific test suite only for negative cases which are constructed based on specific qat internal meta-data.
3. Include the negative tests in the main suite, but only run them on QAT (by checking driver type)
4. include the negative tests in the main suite, run them on all, expecting a FAIL from QAT and a PASS from other devices.My preference is for 2.
But up to you.
While not something for this release, perhaps in future cryptodev could implement a "selftest()" callback API like rawdev does [1], which allows drivers to implement their own specific test cases too.
[1] http://doc.dpdk.org/api-20.02/rte__rawdev_8h.html#a776edaa7060fc6a9d66e00f84132e140
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
Hi Bruce,
Quoted 5 lines
While not something for this release, perhaps in future cryptodev could implement a "
selftest()" callback API like rawdev does [1], which allows drivers to implement their own specific test cases too.[1] http://doc.dpdk.org/api-20.02/rte__rawdev_8h.html#a776edaa7060fc6a9d66e00f84132e140
[Fiona] Thanks for the suggestion.
Yes, that's probably a good way to handle useful, but device-specific test cases, and would make sense for this particular case. We'll consider it the next time such a case arises.
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
In reply to Trahe, FionaQuoted 68 lines
Quoted 14 lines
Quoted 14 lines
Hi Akhil,
Quoted 11 lines
Hi Fiona/Adam,
Quoted 7 lines
This patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite.
Signed-off-by Adam Dybkowski <[email protected]>---
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Why do we need to have separate testsuite for QAT?
Can't we have a single one and based on capability of the driver, Determine which tests need to be skipped in case they are not supported. This would create a mess in the longer run just like cryptodev.Please fix this, we cannot take this patch as is.
[Fiona] Yes, I understand your concern and we considered including in the
main
Quoted 7 lines
Quoted 7 lines
suite.
However these tests are not based on something that can be checked in capabilities. They are tests to hone in on a specific corner case based on a QAT limitation in its intermediate buffer size. So some of the tests are to validate that the recent changes we made in the PMD correctly work around that limitation, but other tests are negative and expected to fail as provoking a corner-case that still exists. Other devices would probably notfail
Quoted 3 lines
Quoted 1 line
the same tests.
Does that mean that all PMDs will pass with the newly added testcase which is
for
Quoted 1 line
A corner case in QAT. If that is the case what is the issue in adding that in the
main
Quoted 3 lines
Test suite. It will get passed in all PMDs, isn't it? Am I missing something?
I believe we should not have PMD specific test suites, rather it should be based
on
Quoted 1 line
Capabilities to identify the cases which should be run for that particular PMD.
[Fiona] yes, several of the cases should pass on all PMDs.
So we could move those into the main suite.
But what to do about the negative tests?
Example: If a very large data buffer is passed to QAT to compress with dyn compression, it will get
split in the PMD into many smaller requests to the hardware. However if the number
of requests is bigger than can fit on the qp then this will never succeed. The test validates that the PMD behaves appropriately in this expected error case. That same
case would probably not have an error on another device. Maybe we should just leave out
such negative tests, but I find them useful as they validate the known behaviour. The buffer size used in the test is based on the known size QAT can handle and the
corner case in which QAT will return an error.I see 4 options to handle this:
1. Leave out those tests
2. Use a qat-specific test suite only for negative cases which are constructed based on specific qat internal meta-data.
3. Include the negative tests in the main suite, but only run them on QAT (by checking driver type)
4. include the negative tests in the main suite, run them on all, expecting a FAIL from QAT and a PASS from other devices.
Quoted 3 lines
My preference is for 2.
But up to you.
I would say 4 is better. And why do you say negative cases will fail on QAT and pass on all other. The test cases are to test the library APIs which are same for all the PMDs and the behavior should Be same.
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
Hi Akhil,
Quoted 76 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Thursday, April 16, 2020 3:38 PM
To: Trahe, Fiona <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]
Cc: Shally Verma <[email protected]>
Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit testsQuoted 68 lines
Quoted 14 lines
Quoted 14 lines
Hi Akhil,
Quoted 11 lines
Hi Fiona/Adam,
Quoted 7 lines
This patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite.
Signed-off-by Adam Dybkowski <[email protected]>---
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Why do we need to have separate testsuite for QAT?
Can't we have a single one and based on capability of the driver, Determine which tests need to be skipped in case they are not supported. This would create a mess in the longer run just like cryptodev.Please fix this, we cannot take this patch as is.
[Fiona] Yes, I understand your concern and we considered including in the
main
Quoted 7 lines
Quoted 7 lines
suite.
However these tests are not based on something that can be checked in capabilities. They are tests to hone in on a specific corner case based on a QAT limitation in its intermediate buffer size. So some of the tests are to validate that the recent changes we made in the PMD correctly work around that limitation, but other tests are negative and expected to fail as provoking a corner-case that still exists. Other devices would probably notfail
Quoted 3 lines
Quoted 1 line
the same tests.
Does that mean that all PMDs will pass with the newly added testcase which is
for
Quoted 1 line
A corner case in QAT. If that is the case what is the issue in adding that in the
main
Quoted 3 lines
Test suite. It will get passed in all PMDs, isn't it? Am I missing something?
I believe we should not have PMD specific test suites, rather it should be based
on
Quoted 1 line
Capabilities to identify the cases which should be run for that particular PMD.
[Fiona] yes, several of the cases should pass on all PMDs.
So we could move those into the main suite.
But what to do about the negative tests?
Example: If a very large data buffer is passed to QAT to compress with dyn compression, it will get
split in the PMD into many smaller requests to the hardware. However if the number
of requests is bigger than can fit on the qp then this will never succeed. The test validates that the PMD behaves appropriately in this expected error case. That same
case would probably not have an error on another device. Maybe we should just leave out
such negative tests, but I find them useful as they validate the known behaviour. The buffer size used in the test is based on the known size QAT can handle and the
corner case in which QAT will return an error.I see 4 options to handle this:
1. Leave out those tests
2. Use a qat-specific test suite only for negative cases which are constructed based on specific qat internal meta-data.
3. Include the negative tests in the main suite, but only run them on QAT (by checking driver type)
4. include the negative tests in the main suite, run them on all, expecting a FAIL from QAT and a PASS from other devices.
Quoted 6 lines
Quoted 3 lines
My preference is for 2.
But up to you.I would say 4 is better. And why do you say negative cases will fail on QAT and pass on all other. The test cases are to test the library APIs which are same for all the PMDs and the behavior should Be same.
[Fiona] I've explained above why QAT fails, sorry if it isn't clear.
Any device can have errors - it's not an API or capability issue, it's a device limitation in a very unlikely corner case. So 4 is ok? i.e. if there is conditional code in the UT expecting different result depending on PMD type? If not, we'll revert to 1 and leave out those tests.
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
Quoted 84 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Hi Fiona/Adam,
Quoted 5 lines
This patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Signed-off-by Adam Dybkowski <[email protected]>---
Why do we need to have separate testsuite for QAT? Can't we have a single one and based on capability of the driver, Determine which tests need to be skipped in case they are not
supported.
Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
This would create a mess in the longer run just like cryptodev.
Please fix this, we cannot take this patch as is.
[Fiona] Yes, I understand your concern and we considered including in the
main
Quoted 5 lines
Quoted 5 lines
suite.
However these tests are not based on something that can be checked in capabilities. They are tests to hone in on a specific corner case based on a QAT limitation in its intermediate buffer size. So some of the tests are to validate that the recent changes we made in the PMDcorrectly
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
work around that limitation, but other tests are negative and expected to
fail
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
as provoking a corner-case that still exists. Other devices would probably
not
Quoted 4 lines
Quoted 4 lines
fail
Quoted 3 lines
Quoted 1 line
the same tests.
Does that mean that all PMDs will pass with the newly added testcase
which is
Quoted 2 lines
Quoted 2 lines
for
Quoted 1 line
A corner case in QAT. If that is the case what is the issue in adding that in
the
Quoted 4 lines
Quoted 4 lines
main
Quoted 3 lines
Test suite. It will get passed in all PMDs, isn't it? Am I missing something?
I believe we should not have PMD specific test suites, rather it should be
based
Quoted 2 lines
Quoted 2 lines
on
Quoted 1 line
Capabilities to identify the cases which should be run for that particular
PMD.
Quoted 8 lines
Quoted 8 lines
[Fiona] yes, several of the cases should pass on all PMDs.
So we could move those into the main suite.
But what to do about the negative tests?
Example: If a very large data buffer is passed to QAT to compress with dyn compression, it will get
split in the PMD into many smaller requests to the hardware. However if the number
of requests is bigger than can fit on the qp then this will never succeed. Thetest
Quoted 1 line
Quoted 1 line
validates that the PMD behaves appropriately in this expected error case.
That
Quoted 2 lines
Quoted 2 lines
same
case would probably not have an error on another device. Maybe we shouldjust
Quoted 2 lines
Quoted 2 lines
leave out
such negative tests, but I find them useful as they validate the knownbehaviour.
Quoted 1 line
Quoted 1 line
The buffer size used in the test is based on the known size QAT can handle
and
Quoted 10 lines
Quoted 10 lines
the
corner case in which QAT will return an error.I see 4 options to handle this:
1. Leave out those tests
2. Use a qat-specific test suite only for negative cases which are constructed based on specific qat internal meta-data.
3. Include the negative tests in the main suite, but only run them on QAT (by checking driver type)
4. include the negative tests in the main suite, run them on all, expecting aFAIL
Quoted 6 lines
Quoted 5 lines
from QAT and a PASS from other devices.
My preference is for 2.
But up to you.I would say 4 is better. And why do you say negative cases will fail on QAT and
pass on all other.
Quoted 1 line
The test cases are to test the library APIs which are same for all the PMDs and
Quoted 8 lines
the behavior should
Quoted 1 line
Be same.
[Fiona] I've explained above why QAT fails, sorry if it isn't clear. Any device can have errors - it's not an API or capability issue, it's a device limitation in a very unlikely corner case.
So 4 is ok? i.e. if there is conditional code in the UT expecting different result depending on PMD type?
If not, we'll revert to 1 and leave out those tests.
I am still not convinced how different PMDs will behave differently for a particular case. Even if QAT/any PMD has a corner case, the test case will fail in that case.
You mean you want to make that case pass if the corner case has hit because you have
A known issue reported for that case and you don't want to highlight that in the test summary? I am not sure if that is a good thing to do.
If the case is failing, then it should report as failed even if you have a defined known issue for that.
We don't need to add any checks for PMD types.
Regards,
Akhil
Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
Hi Akhil,
Quoted 11 lines
-----Original Message-----
From: Akhil Goyal <[email protected]>
Sent: Friday, April 17, 2020 4:39 PM
To: Trahe, Fiona <[email protected]>; Dybkowski, AdamX <[email protected]>; [email protected]
Cc: Shally Verma <[email protected]>
Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit testsQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Hi Fiona/Adam,
Quoted 8 lines
I am still not convinced how different PMDs will behave differently for a particular case. Even if QAT/any PMD has a corner case, the test case will fail in that case.
You mean you want to make that case pass if the corner case has hit because you have
A known issue reported for that case and you don't want to highlight that in the test summary? I am not sure if that is a good thing to do.
If the case is failing, then it should report as failed even if you have a defined known issue for that.We don't need to add any checks for PMD types.
[Fiona] ok, we've reworked the tests so all are common.
v3 cover compress/qat: im buffer too small - split op
In reply to Adam DybkowskiThis patch implements a special way of buffer handling when internal QAT IM buffer is too small for Huffman dynamic compression operation. Instead of falling back to fixed compression, the operation is now split into multiple smaller dynamic compression requests (possible to execute on QAT) and their results are then combined and copied into the output buffer. This is not possible if any checksum calculation was requested - in such case the code falls back to fixed compression as before.
v2:
* various post-review small fixes
v3:
* refactor and simplify unit tests, removing the need of adding a new test suite separate for QAT-specific tests
Adam Dybkowski (2):
compress/qat: im buffer too small - split op test/compress: im buffer too small - add unit tests
app/test/test_compressdev.c | 1185 ++++++++++++++++++++++-- doc/guides/compressdevs/qat_comp.rst | 3 - doc/guides/cryptodevs/qat.rst | 7 +- doc/guides/rel_notes/release_20_05.rst | 10 + drivers/common/qat/qat_qp.c | 223 ++++- drivers/common/qat/qat_qp.h | 3 + drivers/compress/qat/qat_comp.c | 474 +++++++++- drivers/compress/qat/qat_comp.h | 29 +- drivers/compress/qat/qat_comp_pmd.c | 27 +- 9 files changed, 1820 insertions(+), 141 deletions(-)
Signature · 1 line
2.17.1
PATCH v3 1/2 compress/qat: im buffer too small - split op
This patch implements a special way of buffer handling when internal QAT IM buffer is too small for Huffman dynamic compression operation. Instead of falling back to fixed compression, the operation is now split into multiple smaller dynamic compression requests (possible to execute on QAT) and their results are then combined and copied into the output buffer. This is not possible if any checksum calculation was requested - in such case the code falls back to fixed compression as before.
--- doc/guides/compressdevs/qat_comp.rst | 3 - doc/guides/cryptodevs/qat.rst | 7 +- doc/guides/rel_notes/release_20_05.rst | 10 + drivers/common/qat/qat_qp.c | 223 +++++++++++- drivers/common/qat/qat_qp.h | 3 + drivers/compress/qat/qat_comp.c | 474 +++++++++++++++++++++++-- drivers/compress/qat/qat_comp.h | 29 +- drivers/compress/qat/qat_comp_pmd.c | 27 +- 8 files changed, 702 insertions(+), 74 deletions(-)
doc/guides/compressdevs/qat_comp.rst +0 -3
@@ -42,9 +42,6 @@ Limitations from the RX queue must be done from one thread, but enqueues and dequeues may be done in different threads.) * No BSD support as BSD QAT kernel driver not available. -* When using Deflate dynamic huffman encoding for compression, the input size (op.src.length) - must be < CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE from the config file, - see :ref:`building_qat_config` for more details. * Stateful compression is not supported.
doc/guides/cryptodevs/qat.rst +5 -2
@@ -260,8 +260,11 @@ allocated while for GEN1 devices, 12 buffers are allocated, plus 1472 bytes over .. Note:: If the compressed output of a Deflate operation using Dynamic Huffman - Encoding is too big to fit in an intermediate buffer, then the - operation will fall back to fixed compression rather than failing the operation. + Encoding is too big to fit in an intermediate buffer, then the + operation will be split into smaller operations and their results will + be merged afterwards. + This is not possible if any checksum calculation was requested - in such + case the code falls back to fixed compression. To avoid this less performant case, applications should configure the intermediate buffer size to be larger than the expected input data size (compressed output size is usually unknown, so the only option is to make
doc/guides/rel_notes/release_20_05.rst +10 -0
@@ -81,6 +81,16 @@ New Features by making use of the event device capabilities. The event mode currently supports only inline IPsec protocol offload. +* **Added QAT intermediate buffer too small handling in QAT compression PMD.** + + Added a special way of buffer handling when internal QAT intermediate buffer + is too small for Huffman dynamic compression operation. Instead of falling + back to fixed compression, the operation is now split into multiple smaller + dynamic compression requests (possible to execute on QAT) and their results + are then combined and copied into the output buffer. This is not possible if + any checksum calculation was requested - in such case the code falls back to + fixed compression as before. + Removed Items -------------
drivers/common/qat/qat_qp.c +210 -13
263 lines not on this page · raw message
drivers/common/qat/qat_qp.h +3 -0
@@ -89,6 +89,9 @@ extern const struct qat_qp_hw_data qat_gen3_qps[][ADF_MAX_QPS_ON_ANY_SERVICE];uint16_t qat_enqueue_op_burst (void *qp,void **ops,uint16_t nb_ops); +uint16_t +qat_enqueue_comp_op_burst (void *qp,void **ops,uint16_t nb_ops); +uint16_t qat_dequeue_op_burst (void *qp,void **ops,uint16_t nb_ops);
drivers/compress/qat/qat_comp.c +437 -37
547 lines not on this page · raw message
drivers/compress/qat/qat_comp.h +28 -1
@@ -11,6 +11,7 @@#include <rte_compressdev_pmd.h> #include "qat_common.h" +#include "qat_qp.h" #include "icp_qat_hw.h" #include "icp_qat_fw_comp.h" #include "icp_qat_fw_la.h"
@@ -22,7 +23,7 @@#define ERR_CODE_QAT_COMP_WRONG_FW -99/* fallback to fixed compression threshold */ -#define QAT_FALLBACK_THLD ((uint32_t)(RTE_PMD_QAT_COMP_IM_BUFFER_SIZE / 1.1)) +#define QAT_FALLBACK_THLD ((uint32_t)(RTE_PMD_QAT_COMP_IM_BUFFER_SIZE / 1.3))#define QAT_MIN_OUT_BUF_SIZE 46
@@ -63,6 +64,24 @@ struct qat_comp_op_cookie {uint16_t dst_nb_elems;struct qat_sgl *qat_sgl_src_d;struct qat_sgl *qat_sgl_dst_d; +struct qat_qp *qp; +uint32_t cookie_index; + +/* QAT IM buffer too small handling: */ +uint8_t split_op; +uint8_t nb_children; + +/* used by the parent only */ +uint8_t nb_child_responses; +uint32_t total_consumed; +uint32_t total_produced; +const struct rte_memzone **dst_memzones; +struct rte_mbuf *dst_data; +uint32_t dst_data_offset; + +/* used by the child only */ +struct qat_comp_op_cookie *parent_cookie; +void *dest_buffer; };struct qat_comp_xform {
@@ -86,6 +105,14 @@ intqat_comp_build_request (void *in_op, uint8_t *out_msg, void *op_cookie, enum qat_device_gen qat_dev_gen __rte_unused); +int +qat_comp_build_multiple_requests (void *in_op,struct qat_qp *qp, +uint32_t parent_tail,int nb_descr); + +void +qat_comp_free_split_op_memzones (struct qat_comp_op_cookie *cookie, +unsigned int nb_children); +int qat_comp_process_response (void **op,uint8_t *resp,void *op_cookie,uint64_t *dequeue_err_count);
drivers/compress/qat/qat_comp_pmd.c +9 -18
@@ -146,6 +146,9 @@ qat_comp_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,struct qat_comp_op_cookie *cookie = qp->op_cookies[i]; + cookie->qp = qp; + cookie->cookie_index = i; + cookie->qat_sgl_src_d =rte_zmalloc_socket (NULL ,sizeof (struct qat_sgl ) + sizeof(struct qat_flat_buf) *
@@ -560,20 +563,6 @@ qat_comp_dev_info_get(struct rte_compressdev *dev, } } -static uint16_t -qat_comp_pmd_enqueue_op_burst (void *qp,struct rte_comp_op **ops, -uint16_t nb_ops) -{ -return qat_enqueue_op_burst (qp, (void **)ops, nb_ops); -} - -static uint16_t -qat_comp_pmd_dequeue_op_burst (void *qp,struct rte_comp_op **ops, -uint16_t nb_ops) -{ -return qat_dequeue_op_burst (qp, (void **)ops, nb_ops); -} - static uint16_t qat_comp_pmd_enq_deq_dummy_op_burst(void *qp __rte_unused, struct rte_comp_op **ops __rte_unused,
@@ -603,7 +592,7 @@ static struct rte_compressdev_ops compress_qat_dummy_ops = { };static uint16_t -qat_comp_pmd_dequeue_frst_op_burst (void *qp,struct rte_comp_op **ops, +qat_comp_pmd_dequeue_first_op_burst (void *qp,struct rte_comp_op **ops,uint16_t nb_ops) {uint16_t ret =qat_dequeue_op_burst (qp, (void **)ops, nb_ops);
@@ -623,7 +612,8 @@ qat_comp_pmd_dequeue_frst_op_burst(void *qp, struct rte_comp_op **ops, }else { tmp_qp->qat_dev->comp_dev->compressdev->dequeue_burst = - qat_comp_pmd_dequeue_op_burst; + (compressdev_dequeue_pkt_burst_t ) + qat_dequeue_op_burst; } }return ret;
@@ -698,8 +688,9 @@ qat_comp_dev_create(struct qat_pci_device *qat_pci_dev, compressdev->dev_ops = &compress_qat_ops; - compressdev->enqueue_burst = qat_comp_pmd_enqueue_op_burst; - compressdev->dequeue_burst = qat_comp_pmd_dequeue_frst_op_burst; + compressdev->enqueue_burst = (compressdev_enqueue_pkt_burst_t ) + qat_enqueue_comp_op_burst; + compressdev->dequeue_burst = qat_comp_pmd_dequeue_first_op_burst; compressdev->feature_flags =RTE_COMPDEV_FF_HW_ACCELERATED ;
Signature · 1 line
2.17.1
PATCH v3 2/2 test/compress: im buffer too small - add unit tests
In reply to Adam DybkowskiThis patch adds new tests for verification of the "internal QAT IM buffer too small" case handling. These unit tests aren't specific to the QAT PMD only - they pass or skip on other PMDs like ISAL and ZLIB (depending on particular PMD capabilities).
--- app/test/test_compressdev.c | 1185 +++++++++++++++++++++++++++++++++-- 1 file changed, 1118 insertions(+), 67 deletions(-)
app/test/test_compressdev.c +1118 -67
@@ -6,6 +6,7 @@#include <math.h> #include <stdlib.h> #include <unistd.h> +#include <stdio.h> #include <rte_cycles.h> #include <rte_malloc.h>
@@ -30,6 +31,7 @@* due to the compress block headers */ #define COMPRESS_BUF_SIZE_RATIO 1.3 +#define COMPRESS_BUF_SIZE_RATIO_DISABLED 1.0#define COMPRESS_BUF_SIZE_RATIO_OVERFLOW 0.2#define NUM_LARGE_MBUFS 16#define SMALL_SEG_SIZE 256
@@ -49,8 +51,26 @@#define MAX_MBUF_SEGMENT_SIZE 65535#define MAX_DATA_MBUF_SIZE (MAX_MBUF_SEGMENT_SIZE - RTE_PKTMBUF_HEADROOM) -#define NUM_BIG_MBUFS 4 -#define BIG_DATA_TEST_SIZE (MAX_DATA_MBUF_SIZE * NUM_BIG_MBUFS / 2) +#define NUM_BIG_MBUFS (512 + 1) +#define BIG_DATA_TEST_SIZE (MAX_DATA_MBUF_SIZE * 2) + +/* constants for "im buffer" tests start here */ + +/* number of mbufs lower than number of inflight ops */ +#define IM_BUF_NUM_MBUFS 3 +/* above threshold (QAT_FALLBACK_THLD) and below max mbuf size */ +#define IM_BUF_DATA_TEST_SIZE_LB 59600 +/* data size smaller than the queue capacity */ +#define IM_BUF_DATA_TEST_SIZE_SGL (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS) +/* number of mbufs bigger than number of inflight ops */ +#define IM_BUF_NUM_MBUFS_OVER (NUM_MAX_INFLIGHT_OPS + 1) +/* data size bigger than the queue capacity */ +#define IM_BUF_DATA_TEST_SIZE_OVER (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS_OVER) +/* number of mid-size mbufs */ +#define IM_BUF_NUM_MBUFS_MID ((NUM_MAX_INFLIGHT_OPS / 3) + 1) +/* capacity of mid-size mbufs */ +#define IM_BUF_DATA_TEST_SIZE_MID (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS_MID) +const char * huffman_type_strings[] = {
@@ -78,6 +98,11 @@ enum overflow_test {OVERFLOW_ENABLED }; +enum ratio_switch { +RATIO_DISABLED , +RATIO_ENABLED +}; +enum operation_type {OPERATION_COMPRESSION ,OPERATION_DECOMPRESSION
@@ -123,6 +148,7 @@ struct test_data_params {const struct rte_memzone *uncompbuf_memzone;/* overflow test activation */ enum overflow_test overflow; +enum ratio_switch ratio; };struct test_private_arrays {
@@ -141,6 +167,7 @@ struct test_private_arrays {static struct comp_testsuite_params testsuite_params = {0 }; +static void testsuite_teardown (void ) {
@@ -316,6 +343,8 @@ test_compressdev_invalid_configuration(void) };struct rte_compressdev_info dev_info; +RTE_LOG (INFO ,USER1 ,"This is a negative test, errors are expected\n" ); +/* Invalid configuration with 0 queue pairs */ memcpy (&invalid_config, &valid_config,sizeof (struct rte_compressdev_config ));
@@ -691,7 +720,7 @@ prepare_sgl_bufs(const char *test_buf, struct rte_mbuf *head_buf,if (data_ptr !=NULL ) {/* Copy characters without NULL terminator */ -strncpy (buf_ptr, data_ptr, data_size); +memcpy (buf_ptr, data_ptr, data_size); data_ptr += data_size; } remaining_data -= data_size;
@@ -731,7 +760,7 @@ prepare_sgl_bufs(const char *test_buf, struct rte_mbuf *head_buf, }if (data_ptr !=NULL ) {/* Copy characters without NULL terminator */ -strncpy (buf_ptr, data_ptr, data_size); +memcpy (buf_ptr, data_ptr, data_size); data_ptr += data_size; } remaining_data -= data_size;
@@ -760,17 +789,20 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops, {uint16_t num_enqd, num_deqd, num_total_deqd;unsigned int deqd_retries =0 ; +int res =0 ;/* Enqueue and dequeue all operations */ num_enqd =rte_compressdev_enqueue_burst (0 ,0 , ops, num_bufs);if (num_enqd < num_bufs) {RTE_LOG (ERR ,USER1 ,"Some operations could not be enqueued\n" ); -return -1 ; + res =-1 ; } +/* dequeue ops even on error (same number of ops as was enqueued) */ + num_total_deqd =0 ; - do { +while (num_total_deqd < num_enqd) {/* * If retrying a dequeue call, wait for 10 ms to allow * enough time to the driver to process the operations
@@ -783,7 +815,8 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops,if (deqd_retries ==MAX_DEQD_RETRIES ) {RTE_LOG (ERR ,USER1 ,"Not all operations could be dequeued\n" ); -return -1 ; + res =-1 ; +break ; }usleep (DEQUEUE_WAIT_TIME ); }
@@ -792,9 +825,9 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops, num_total_deqd += num_deqd; deqd_retries++; - }while (num_total_deqd < num_enqd); + } -return 0 ; +return res; }/**
@@ -956,7 +989,9 @@ test_mbufs_calculate_data_size(/* local variables: */ uint32_t data_size;struct priv_op_data *priv_data; -float ratio; +float ratio_val; +enum ratio_switch ratio = test_data->ratio; +uint8_t not_zlib_compr;/* true if zlib isn't current compression dev */ enum overflow_test overflow = test_data->overflow;
@@ -973,13 +1008,16 @@ test_mbufs_calculate_data_size( not_zlib_compr = (test_data->zlib_dir ==ZLIB_DECOMPRESS || test_data->zlib_dir ==ZLIB_NONE ); - ratio = (not_zlib_compr && + ratio_val = (ratio ==RATIO_ENABLED ) ? +COMPRESS_BUF_SIZE_RATIO : +COMPRESS_BUF_SIZE_RATIO_DISABLED ; + + ratio_val = (not_zlib_compr && (overflow ==OVERFLOW_ENABLED )) ?COMPRESS_BUF_SIZE_RATIO_OVERFLOW : -COMPRESS_BUF_SIZE_RATIO ; - - data_size =strlen (test_bufs[i]) * ratio; + ratio_val; + data_size =strlen (test_bufs[i]) * ratio_val; }else { priv_data = (struct priv_op_data *) (ops_processed[i] +1 );
@@ -1085,6 +1123,9 @@ test_setup_output_bufs( }else {for (i =0 ; i < num_bufs; i++) { +enum rte_comp_huffman comp_huffman = + ts_params->def_comp_xform->compress.deflate.huffman; +/* data size calculation */ data_size = test_mbufs_calculate_data_size( op_type,
@@ -1094,6 +1135,11 @@ test_setup_output_bufs( test_data, i); +if (comp_huffman !=RTE_COMP_HUFFMAN_DYNAMIC ) { +if (op_type ==OPERATION_DECOMPRESSION ) + data_size *=COMPRESS_BUF_SIZE_RATIO ; + } +/* data allocation */ if (buff_type ==SGL_BOTH || buff_type ==LB_TO_SGL ) { ret = prepare_sgl_bufs(NULL, current_bufs[i],
@@ -1192,6 +1238,11 @@ test_deflate_comp_run(const struct interim_data_params *int_data, ops[i]->src.length =rte_pktmbuf_pkt_len (uncomp_bufs[i]); ops[i]->dst.offset =0 ; +RTE_LOG (DEBUG ,USER1 , +"Uncompressed buffer length = %u compressed buffer length = %u" , +rte_pktmbuf_pkt_len (uncomp_bufs[i]), +rte_pktmbuf_pkt_len (comp_bufs[i])); +if (operation_type ==RTE_COMP_OP_STATELESS ) { ops[i]->flush_flag =RTE_COMP_FLUSH_FINAL ; }else {
@@ -1313,6 +1364,7 @@ test_deflate_comp_run(const struct interim_data_params *int_data,if (ret_status <0 )for (i =0 ; i < num_bufs; i++) {rte_comp_op_free (ops[i]); + ops[i] =NULL ; ops_processed[i] =NULL ; }
@@ -1431,7 +1483,7 @@ test_deflate_comp_finalize(const struct interim_data_params *int_data, }RTE_LOG (ERR ,USER1 , -"Some operations were not successful\n" ); +"Comp: Some operations were not successful\n" );return -1 ; } priv_data = (struct priv_op_data *)(ops_processed[i] +1 );
@@ -1490,6 +1542,7 @@ test_deflate_decomp_run(const struct interim_data_params *int_data,/* from test_priv_data: */ struct rte_mbuf **uncomp_bufs = test_priv_data->uncomp_bufs; +struct rte_mbuf **comp_bufs = test_priv_data->comp_bufs;struct rte_comp_op **ops = test_priv_data->ops;struct rte_comp_op **ops_processed = test_priv_data->ops_processed;void **priv_xforms = test_priv_data->priv_xforms;
@@ -1510,7 +1563,7 @@ test_deflate_decomp_run(const struct interim_data_params *int_data,/* Source buffer is the compressed data from the previous operations */ for (i =0 ; i < num_bufs; i++) { - ops[i]->m_src = ops_processed[i]->m_dst; + ops[i]->m_src = comp_bufs[i]; ops[i]->m_dst = uncomp_bufs[i]; ops[i]->src.offset =0 ;/*
@@ -1740,6 +1793,10 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE || ops_processed[i]->status == RTE_COMP_OP_STATUS_SUCCESS)) { + +RTE_LOG (DEBUG ,USER1 , +".............RECOVERABLE\n" ); +/* collect the output into all_decomp_data */ const void *ptr = rte_pktmbuf_read( ops_processed[i]->m_dst,
@@ -1777,7 +1834,6 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, ops[i]->src.length -= ops_processed[i]->consumed;/* repeat the operation */ -//goto next_step; return 2 ; }else {/* Compare the original stream with the */
@@ -1808,7 +1864,8 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, }else if (ops_processed[i]->status !=RTE_COMP_OP_STATUS_SUCCESS ) {RTE_LOG (ERR ,USER1 , -"Some operations were not successful\n" ); +"Decomp: Some operations were not successful, status = %u\n" , + ops_processed[i]->status);return -1 ; } priv_data = (struct priv_op_data *)(ops_processed[i] +1 );
@@ -1986,7 +2043,6 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, "Compress device does not support DEFLATE\n");return -1 ; } -//test_objects_init(&test_priv_data, num_bufs); /* Prepare the source mbufs with the data */ ret =test_setup_com_bufs (int_data, test_data, &test_priv_data);
@@ -1995,6 +2051,8 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, goto exit; } +RTE_LOG (DEBUG ,USER1 ,"<<< COMPRESSION >>>\n" ); +/* COMPRESSION */ /* Prepare output (destination) mbufs for compressed data */
@@ -2031,6 +2089,8 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data,/* DECOMPRESSION */ +RTE_LOG (DEBUG ,USER1 ,"<<< DECOMPRESSION >>>\n" ); +/* Prepare output (destination) mbufs for decompressed data */ ret = test_setup_output_bufs( OPERATION_DECOMPRESSION,
@@ -2096,7 +2156,6 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, priv_xforms[i] =NULL ; } } -for (i =0 ; i < num_bufs; i++) {rte_pktmbuf_free (uncomp_bufs[i]);rte_pktmbuf_free (comp_bufs[i]);
@@ -2152,7 +2211,8 @@ test_compressdev_deflate_stateless_fixed(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };for (i =0 ; i <RTE_DIM (compress_test_bufs); i++) {
@@ -2223,7 +2283,8 @@ test_compressdev_deflate_stateless_dynamic(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };for (i =0 ; i <RTE_DIM (compress_test_bufs); i++) {
@@ -2278,7 +2339,8 @@ test_compressdev_deflate_stateless_multi_op(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Compress with compressdev, decompress with Zlib */
@@ -2332,7 +2394,8 @@ test_compressdev_deflate_stateless_multi_level(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };for (i =0 ; i <RTE_DIM (compress_test_bufs); i++) {
@@ -2422,7 +2485,8 @@ test_compressdev_deflate_stateless_multi_xform(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Compress with compressdev, decompress with Zlib */
@@ -2471,7 +2535,8 @@ test_compressdev_deflate_stateless_sgl(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };for (i =0 ; i <RTE_DIM (compress_test_bufs); i++) {
@@ -2582,7 +2647,8 @@ test_compressdev_deflate_stateless_checksum(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Check if driver supports crc32 checksum and test */
@@ -2700,7 +2766,8 @@ test_compressdev_out_of_space_buffer(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 1, /* run out-of-space test */ .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Compress with compressdev, decompress with Zlib */ test_data.zlib_dir =ZLIB_DECOMPRESS ;
@@ -2742,7 +2809,7 @@ test_compressdev_deflate_stateless_dynamic_big(void)struct comp_testsuite_params *ts_params = &testsuite_params;uint16_t i =0 ;int ret; -int j; +unsigned int j;const struct rte_compressdev_capabilities *capab;char *test_buffer =NULL ;
@@ -2778,7 +2845,8 @@ test_compressdev_deflate_stateless_dynamic_big(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 1, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED }; ts_params->def_comp_xform->compress.deflate.huffman =
@@ -2788,7 +2856,7 @@ test_compressdev_deflate_stateless_dynamic_big(void)srand (BIG_DATA_TEST_SIZE );for (j =0 ; j <BIG_DATA_TEST_SIZE -1 ; ++j) test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; - test_buffer[BIG_DATA_TEST_SIZE -1 ] =0 ; + test_buffer[BIG_DATA_TEST_SIZE -1 ] =0 ;/* Compress with compressdev, decompress with Zlib */ test_data.zlib_dir =ZLIB_DECOMPRESS ;
@@ -2843,7 +2911,8 @@ test_compressdev_deflate_stateful_decomp(void) .big_data = 0, .decompress_output_block_size = 2000, .decompress_steps_max = 4, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Compress with Zlib, decompress with compressdev */
@@ -2926,7 +2995,8 @@ test_compressdev_deflate_stateful_decomp_checksum(void) .big_data = 0, .decompress_output_block_size = 2000, .decompress_steps_max = 4, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED };/* Check if driver supports crc32 checksum and test */
@@ -3139,7 +3209,8 @@ test_compressdev_deflate_stateless_fixed_oos_recoverable(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_ENABLED + .overflow = OVERFLOW_ENABLED, + .ratio = RATIO_ENABLED };for (i =0 ; i <RTE_DIM (compress_test_bufs); i++) {
@@ -3176,39 +3247,1019 @@ test_compressdev_deflate_stateless_fixed_oos_recoverable(void)return ret; } -static struct unit_test_suite compressdev_testsuite = { - .suite_name ="compressdev unit test suite" , - .setup = testsuite_setup, - .teardown = testsuite_teardown, - .unit_test_cases = { -TEST_CASE_ST (NULL ,NULL , - test_compressdev_invalid_configuration), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_fixed), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_dynamic), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_dynamic_big), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_op), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_level), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_xform), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_sgl), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_checksum), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_out_of_space_buffer), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateful_decomp), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateful_decomp_checksum), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_external_mbufs), -TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_fixed_oos_recoverable), +static int +test_compressdev_deflate_im_buffers_LB_1op (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_LB ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for 'im buffer' test\n" ); +return TEST_FAILED ; + } + +struct interim_data_params int_data = { + (const char *const *)&test_buffer, +1 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , +/* must be LB to SGL, +* input LB buffer reaches its maximum, +* if ratio 1.3 than another mbuf must be +* created and attached +*/ + .buff_type =LB_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_LB ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_LB -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_2ops_first (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_LB ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for 'im buffer' test\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = test_buffer; + test_buffers[1 ] = compress_test_bufs[0 ]; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =LB_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_LB ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_LB -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_2ops_second (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_LB ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for 'im buffer' test\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =LB_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_LB ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_LB -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_3ops (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[3 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_LB ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for 'im buffer' test\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + test_buffers[2 ] = compress_test_bufs[1 ]; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +3 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =LB_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_LB ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_LB -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_4ops (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[4 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_LB ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for 'im buffer' test\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + test_buffers[2 ] = compress_test_bufs[1 ]; + test_buffers[3 ] = test_buffer; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +4 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =LB_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_LB ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_LB -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_1op (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_SGL ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + +struct interim_data_params int_data = { + (const char *const *)&test_buffer, +1 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_SGL ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_SGL -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_2ops_first (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_SGL ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = test_buffer; + test_buffers[1 ] = compress_test_bufs[0 ]; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_SGL ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_SGL -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_2ops_second (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_SGL ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_SGL ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_SGL -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_3ops (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[3 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_SGL ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + test_buffers[2 ] = compress_test_bufs[1 ]; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +3 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_SGL ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_SGL -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_4ops (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[4 ]; + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_SGL ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + test_buffers[2 ] = compress_test_bufs[1 ]; + test_buffers[3 ] = test_buffer; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +4 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_SGL ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_SGL -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_FAILED ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_over_1op (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; + +RTE_LOG (INFO ,USER1 ,"This is a negative test, errors are expected\n" ); + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_OVER ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + +struct interim_data_params int_data = { + (const char *const *)&test_buffer, +1 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_OVER ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_OVER -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_SUCCESS ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); + +return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_over_2ops_first (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + +RTE_LOG (INFO ,USER1 ,"This is a negative test, errors are expected\n" ); + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_OVER ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = test_buffer; + test_buffers[1 ] = compress_test_bufs[0 ]; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_OVER ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_OVER -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_SUCCESS ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_over_2ops_second (void ) +{ +struct comp_testsuite_params *ts_params = &testsuite_params; +uint16_t i =0 ; +int ret =TEST_SUCCESS ; +int j; +const struct rte_compressdev_capabilities *capab; +char *test_buffer =NULL ; +const char *test_buffers[2 ]; + +RTE_LOG (INFO ,USER1 ,"This is a negative test, errors are expected\n" ); + + capab =rte_compressdev_capability_get (0 ,RTE_COMP_ALGO_DEFLATE ); +TEST_ASSERT (capab !=NULL ,"Failed to retrieve device capabilities" ); + +if ((capab->comp_feature_flags &RTE_COMP_FF_HUFFMAN_DYNAMIC ) ==0 ) +return -ENOTSUP ; + +if ((capab->comp_feature_flags &RTE_COMP_FF_OOP_SGL_IN_SGL_OUT ) ==0 ) +return -ENOTSUP ; + + test_buffer =rte_malloc (NULL ,IM_BUF_DATA_TEST_SIZE_OVER ,0 ); +if (test_buffer ==NULL ) { +RTE_LOG (ERR ,USER1 , +"Can't allocate buffer for big-data\n" ); +return TEST_FAILED ; + } + + test_buffers[0 ] = compress_test_bufs[0 ]; + test_buffers[1 ] = test_buffer; + +struct interim_data_params int_data = { + (const char *const *)test_buffers, +2 , + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, +1 + }; + +struct test_data_params test_data = { + .compress_state =RTE_COMP_OP_STATELESS , + .decompress_state =RTE_COMP_OP_STATELESS , + .buff_type =SGL_BOTH , + .zlib_dir =ZLIB_NONE , + .out_of_space =0 , + .big_data =1 , + .overflow =OVERFLOW_DISABLED , + .ratio =RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DYNAMIC ; + +/* fill the buffer with data based on rand. data */ +srand (IM_BUF_DATA_TEST_SIZE_OVER ); +for (j =0 ; j <IM_BUF_DATA_TEST_SIZE_OVER -1 ; ++j) + test_buffer[j] = (uint8_t )(rand () % ((uint8_t )-1 )) |1 ; + +/* Compress with compressdev, decompress with compressdev */ +if (test_deflate_comp_decomp (&int_data, &test_data) <0 ) { + ret =TEST_SUCCESS ; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = +RTE_COMP_HUFFMAN_DEFAULT ; +rte_free (test_buffer); +return ret; +} + +static struct unit_test_suite compressdev_testsuite = { + .suite_name ="compressdev unit test suite" , + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { +TEST_CASE_ST (NULL ,NULL , + test_compressdev_invalid_configuration), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_fixed), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_dynamic), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_dynamic_big), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_op), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_level), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_xform), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_sgl), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_checksum), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_out_of_space_buffer), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateful_decomp), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateful_decomp_checksum), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_external_mbufs), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_fixed_oos_recoverable), + +/* Positive test cases for IM buffer handling verification */ +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_1op), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_2ops_first), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_2ops_second), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_3ops), + +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_4ops), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_1op), + +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_2ops_first), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_2ops_second), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_3ops), +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_4ops), + +/* Negative test cases for IM buffer handling verification */ + +/* For this test huge mempool is necessary. +* It tests one case: +* only one op containing big amount of data, so that +* number of requested descriptors higher than number +* of available descriptors (128) +*/ +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_1op), + +/* For this test huge mempool is necessary. +* 2 ops. First op contains big amount of data: +* number of requested descriptors higher than number +* of available descriptors (128), the second op is +* relatively small. In this case both ops are rejected +*/ +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_2ops_first), + +TEST_CASE_ST (generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_2ops_second), +TEST_CASES_END ()/**< NULL terminate unit test array */ } };
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH v3 2/2] test/compress: im buffer too small - add unit tests
Quoted 9 lines
-----Original Message-----
From: Dybkowski, AdamX <[email protected]>
Sent: Friday, April 17, 2020 4:44 PM
To: [email protected]; Trahe, Fiona <[email protected]>; [email protected]
Cc: Dybkowski, AdamX <[email protected]>
Subject: [PATCH v3 2/2] test/compress: im buffer too small - add unit tests
Quoted 6 lines
This patch adds new tests for verification of the "internal QAT IM buffer too small" case handling. These unit tests aren't specific to the QAT PMD only - they pass or skip on other PMDs like ISAL and ZLIB (depending on particular PMD capabilities).
Signed-off-by Adam Dybkowski <[email protected]>
Re: [dpdk-dev] [PATCH v3 2/2] test/compress: im buffer too small - add unit tests
Quoted 7 lines
Quoted 6 lines
This patch adds new tests for verification of the "internal QAT IM buffer too small" case handling. These unit tests aren't specific to the QAT PMD only - they pass or skip on other PMDs like ISAL and ZLIB (depending on particular PMD capabilities).
Signed-off-by Adam Dybkowski <[email protected]>Acked-by Fiona Trahe <[email protected]>
Updated patch title
compress/qat: support IM buffer too small operation test/compress: add cases for IM buffer too small
Series applied to dpdk-next-crypto
Thanks.