Skip to contentSkip to conversation

compressdev: add LZ4 support

PATCH Superseded Michael Baum 3 participants 31 messages Jan 9, 2023 – Feb 16, 2023

lore
Share & tools
lore
#0 lore raw

cover compressdev: add LZ4 support

Add support for LZ4 compression algorithm for both API and app.

In addition, enable app to test de/comp only. This option already exists in both app doc and arguments but isn't implemented inside the application.

LZ4: https://github.com/lz4/lz4
RFC: https://patchwork.dpdk.org/project/dpdk/patch/[email protected]/

Depends-on: patch-26344 ("compressdev: fix end of comp PMD list macro conflict")

Michael Baum (4):
compressdev: add LZ4 algorithm support app/test-compress-perf: allow test single compress operation app/test-compress-perf: add algo option app/test-compress-perf: add LZ4 support

 app/test-compress-perf/comp_perf_options.h    |   9 +-
 .../comp_perf_options_parse.c                 |  83 +++++++++++-
 .../comp_perf_test_common.c                   | 126 +++++++++++++-----
 .../comp_perf_test_cyclecount.c               |  82 +++++++-----
 .../comp_perf_test_throughput.c               |  75 +++++++----
 .../comp_perf_test_verify.c                   |  75 +++++++----
 app/test-compress-perf/main.c                 |  63 +++++++--
 doc/guides/compressdevs/features/default.ini  |   4 +
 doc/guides/rel_notes/release_23_03.rst        |  16 +++
 doc/guides/tools/comp_perf.rst                |   5 +
 lib/compressdev/rte_comp.c                    |   6 +
 lib/compressdev/rte_comp.h                    |  51 ++++++-
 12 files changed, 451 insertions(+), 144 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/4 compressdev: add LZ4 algorithm support

Add support for LZ4 algorithm:
- Add Lz4 param structure to XFORM structures.
- Add capabilities flags for LZ4 params.
- Add xxHash-32 checksum and capabilities flag.

---
 doc/guides/compressdevs/features/default.ini |  4 ++
 doc/guides/rel_notes/release_23_03.rst       |  7 +++
 lib/compressdev/rte_comp.c                   |  6 +++
 lib/compressdev/rte_comp.h                   | 51 +++++++++++++++++++-
 4 files changed, 66 insertions(+), 2 deletions(-)
doc/guides/compressdevs/features/default.ini +4 -0
index e1419ee8db..208b8591cf 100644
@@ -20,8 +20,12 @@ OOP SGL In LB  Out     =
 OOP LB  In SGL Out     =
 Deflate                =
 LZS                    =
+LZ4                    =
 Adler32                =
 Crc32                  =
 Adler32&Crc32          =
+xxHash32               =
 Fixed                  =
 Dynamic                =
+LZ4 Block Checksum     =
+LZ4 Block Independence =
doc/guides/rel_notes/release_23_03.rst +7 -0
index b8c5b68d6c..3bcf638544 100644
@@ -55,6 +55,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added LZ4 algorithm in Compressdev Library.**
+
+  Added new compression algorithm, including:
+
+  * Added support for ``RTE_COMP_ALGO_LZ4``.
+  * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
+
 
 Removed Items
 -------------
lib/compressdev/rte_comp.c +6 -0
index 320c6dab92..2ad05dbf01 100644
@@ -39,6 +39,12 @@ rte_comp_get_feature_name(uint64_t flag)
 		return "HUFFMAN_FIXED";
 	case RTE_COMP_FF_HUFFMAN_DYNAMIC:
 		return "HUFFMAN_DYNAMIC";
+	case RTE_COMP_FF_XXHASH32_CHECKSUM:
+		return "XXHASH32_CHECKSUM";
+	case RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE:
+		return "LZ4_BLOCK_INDEPENDENCE";
+	case RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM:
+		return "LZ4_BLOCK_WITH_CHECKSUM";
 	default:
 		return NULL;
 	}
lib/compressdev/rte_comp.h +49 -2

95 lines not on this page · raw message

Signature · 1 line
2.25.1
#2 lore raw

RE: [EXT] [PATCH 1/4] compressdev: add LZ4 algorithm support

Quoted 16 lines
lib/compressdev/rte_comp.h +16 -0
+/**
+ * Block checksum flag.
+ * If this flag is set, each data block will be followed by a 4-bytes checksum,
+ * calculated by using the xxHash-32 algorithm on the raw (compressed) data
+ * block. The intention is to detect data corruption (storage or transmission
+ * errors) immediately, before decoding. Block checksum usage is optional.
+ */
+#define RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM (1 << 4)
+
+/**
+ * Block Independence flag.
+ * If this flag is set to 1, blocks are independent.
+ * If this flag is set to 0, each block depends on previous ones (up to LZ4
+ * window size, which is 64 KB). In such case, it is necessary to decode all
+ * blocks in sequence.
+ * Block dependency improves compression ratio, especially for small blocks.
Quoted 5 lines

On
+ * the other hand, it makes random access or multi-threaded decoding impossible.
+ */
+#define RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE (1 << 5)

Why did you start with 4th and 5th bit of the flags? Why not first two bits?

++ more people for review.

#3 lore raw

RE: [EXT] [PATCH 1/4] compressdev: add LZ4 algorithm support

Quoted 16 lines · Akhil Goyal
Quoted 16 lines
lib/compressdev/rte_comp.h +15 -0
+/**
+ * Block checksum flag.
+ * If this flag is set, each data block will be followed by a 4-bytes checksum,
+ * calculated by using the xxHash-32 algorithm on the raw (compressed) data
+ * block. The intention is to detect data corruption (storage or transmission
+ * errors) immediately, before decoding. Block checksum usage is optional.
+ */
+#define RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM (1 << 4)
+
+/**
+ * Block Independence flag.
+ * If this flag is set to 1, blocks are independent.
+ * If this flag is set to 0, each block depends on previous ones (up to LZ4
+ * window size, which is 64 KB). In such case, it is necessary to decode all
+ * blocks in sequence.
Quoted 8 lines
Quoted 6 lines

+ * Block dependency improves compression ratio, especially for small blocks. On
+ * the other hand, it makes random access or multi-threaded decoding impossible.
+ */
+#define RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE (1 << 5)

Why did you start with 4th and 5th bit of the flags? Why not first two bits?

I didn't choose the values by myself, I took them from LZ4 standard:
https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#frame-descriptor

Quoted 2 lines

++ more people for review.

#4 lore raw

RE: [EXT] [PATCH 1/4] compressdev: add LZ4 algorithm support

Quoted 38 lines

-----Original Message-----
From: Michael Baum <[email protected]>
Sent: Tuesday, January 31, 2023 2:07 AM
To: Akhil Goyal <[email protected]>; [email protected]; Mahipal Challa <[email protected]>; Fan Zhang <[email protected]>; Ashish Gupta <[email protected]>
Cc: Matan Azrad <[email protected]>; Fiona Trahe <[email protected]>; NBU-Contact-Thomas Monjalon (EXTERNAL) <[email protected]>
Subject: RE: [EXT] [PATCH 1/4] compressdev: add LZ4 algorithm support

Quoted 5 lines · Akhil Goyal
Quoted 5 lines

+/**
+ * Block checksum flag.
+ * If this flag is set, each data block will be followed by a 4-bytes checksum,
+ * calculated by using the xxHash-32 algorithm on the raw (compressed)

data

Quoted 4 lines
Quoted 4 lines

+ * block. The intention is to detect data corruption (storage or transmission
+ * errors) immediately, before decoding. Block checksum usage is optional.
+ */
+#define RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM (1 << 4)

Quoted 8 lines

lib/compressdev/rte_comp.h enum rte_comp_op_type {

Quoted 8 lines
lib/compressdev/rte_comp.h +8 -0
+
+/**
+ * Block Independence flag.
+ * If this flag is set to 1, blocks are independent.
+ * If this flag is set to 0, each block depends on previous ones (up to LZ4
+ * window size, which is 64 KB). In such case, it is necessary to decode all
+ * blocks in sequence.
+ * Block dependency improves compression ratio, especially for small

blocks.

Quoted 6 lines
Quoted 5 lines

On
+ * the other hand, it makes random access or multi-threaded decoding impossible.
+ */
+#define RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE (1 << 5)

Quoted 4 lines
Quoted 1 line

Why did you start with 4th and 5th bit of the flags? Why not first two bits?

I didn't choose the values by myself, I took them from LZ4 standard:
https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#frame-descriptor

Ok, Are we defining and making the frame descriptor inside library? Also, why all the bits not added?
I believe these are just flags which PMD would use to make the frame descriptors. If so, we should start with 0th bit and let the PMD handle bit shifting.

#5 lore raw

PATCH 2/4 app/test-compress-perf: allow test single compress operation

In reply to Michael Baum

Part of the application options is to test only compress and only decompress but actually the application ignores this user option and tries to test always both compress and decompress.

Allow testing only compress and only decompress.

---
 app/test-compress-perf/comp_perf_options.h    |   6 +-
 .../comp_perf_options_parse.c                 |   4 +-
 .../comp_perf_test_common.c                   | 126 +++++++++++++-----
 .../comp_perf_test_cyclecount.c               |  71 ++++++----
 .../comp_perf_test_throughput.c               |  64 +++++----
 .../comp_perf_test_verify.c                   |  64 +++++----
 app/test-compress-perf/main.c                 |   8 ++
 doc/guides/rel_notes/release_23_03.rst        |   4 +
 8 files changed, 230 insertions(+), 117 deletions(-)
app/test-compress-perf/comp_perf_options.h +3 -3
index 57dd146330..d00b299247 100644
@@ -32,9 +32,9 @@ enum cperf_test_type {
 };
 
 enum comp_operation {
-	COMPRESS_ONLY,
-	DECOMPRESS_ONLY,
-	COMPRESS_DECOMPRESS
+	COMPRESS = (1 << 0),
+	DECOMPRESS = (1 << 1),
+	COMPRESS_DECOMPRESS = (COMPRESS | DECOMPRESS),
 };
 
 struct range_list {
app/test-compress-perf/comp_perf_options_parse.c +2 -2
index 019eddb7bd..7a992bf43e 100644
@@ -446,11 +446,11 @@ parse_op_type(struct comp_test_data *test_data, const char *arg)
 	struct name_id_map optype_namemap[] = {
 		{
 			"comp",
-			COMPRESS_ONLY
+			COMPRESS
 		},
 		{
 			"decomp",
-			DECOMPRESS_ONLY
+			DECOMPRESS
 		},
 		{
 			"comp_and_decomp",
app/test-compress-perf/comp_perf_test_common.c +91 -35

222 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_cyclecount.c +44 -27

89 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_throughput.c +38 -26

79 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_verify.c +40 -24

105 lines not on this page · raw message

app/test-compress-perf/main.c +8 -0
index 41b8edc2bd..bbb4c7917b 100644
@@ -254,6 +254,14 @@ comp_perf_dump_input_data(struct comp_test_data *test_data)
 		goto end;
 	}
 
+	if (!(test_data->test_op & COMPRESS) &&
+	    test_data->input_data_sz >
+	    (size_t) test_data->seg_sz * (size_t) test_data->max_sgl_segs) {
+		RTE_LOG(ERR, USER1,
+			"Size of input must be less than total segments\n");
+		goto end;
+	}
+
 	test_data->input_data = rte_zmalloc_socket(NULL,
 				test_data->input_data_sz, 0, rte_socket_id());
 
doc/guides/rel_notes/release_23_03.rst +4 -0
index 3bcf638544..1f879da6b8 100644
@@ -62,6 +62,10 @@ New Features
   * Added support for ``RTE_COMP_ALGO_LZ4``.
   * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
 
+* **Allowed test single compress operation in test-compress-perf.**
+
+  Enable the application options for testing only compress and only decompress.
+
 
 Removed Items
 -------------
Signature · 1 line
2.25.1
#6 lore raw

PATCH 3/4 app/test-compress-perf: add algo option

In reply to Michael Baum

Add a command line option algo to select the compress algorithm supported by the compress API: null (DMA), deflate and lzs.

Default for deflate.

---
 app/test-compress-perf/comp_perf_options.h    |  2 +
 .../comp_perf_options_parse.c                 | 40 ++++++++++++++++++-
 .../comp_perf_test_cyclecount.c               |  4 +-
 .../comp_perf_test_throughput.c               |  4 +-
 .../comp_perf_test_verify.c                   |  4 +-
 app/test-compress-perf/main.c                 | 38 +++++++++++-------
 doc/guides/rel_notes/release_23_03.rst        |  4 ++
 doc/guides/tools/comp_perf.rst                |  2 +
 8 files changed, 76 insertions(+), 22 deletions(-)
app/test-compress-perf/comp_perf_options.h +2 -0
index d00b299247..5e5227a700 100644
@@ -63,6 +63,8 @@ struct comp_test_data {
 
 	enum rte_comp_huffman huffman_enc;
 	enum comp_operation test_op;
+	enum rte_comp_algorithm test_algo;
+
 	int window_sz;
 	struct range_list level_lst;
 	uint8_t level;
app/test-compress-perf/comp_perf_options_parse.c +38 -2
index 7a992bf43e..97ddff87ef 100644
@@ -25,6 +25,7 @@
 #define CPERF_MAX_SGL_SEGS	("max-num-sgl-segs")
 #define CPERF_NUM_ITER		("num-iter")
 #define CPERF_OPTYPE		("operation")
+#define CPERF_ALGO		("algo")
 #define CPERF_HUFFMAN_ENC	("huffman-enc")
 #define CPERF_LEVEL		("compress-level")
 #define CPERF_WINDOW_SIZE	("window-sz")
@@ -56,6 +57,8 @@ usage(char *progname)
 		"		compressed/decompressed (default: 10000)\n"
 		" --operation [comp/decomp/comp_and_decomp]: perform test on\n"
 		"		compression, decompression or both operations\n"
+		" --algo [null/deflate/lzs]: perform test on algorithm\n"
+		"		null(DMA), deflate or lzs (default: deflate)\n"
 		" --huffman-enc [fixed/dynamic/default]: Huffman encoding\n"
 		"		(default: dynamic)\n"
 		" --compress-level N: compression level, which could be a single value, list or range\n"
@@ -470,6 +473,36 @@ parse_op_type(struct comp_test_data *test_data, const char *arg)
 	return 0;
 }
 
+static int
+parse_algo(struct comp_test_data *test_data, const char *arg)
+{
+	struct name_id_map algo_namemap[] = {
+		{
+			"null",
+			RTE_COMP_ALGO_NULL
+		},
+		{
+			"deflate",
+			RTE_COMP_ALGO_DEFLATE
+		},
+		{
+			"lzs",
+			RTE_COMP_ALGO_LZS
+		}
+	};
+
+	int id = get_str_key_id_mapping(algo_namemap,
+			RTE_DIM(algo_namemap), arg);
+	if (id < 0) {
+		RTE_LOG(ERR, USER1, "Invalid algorithm specified\n");
+		return -1;
+	}
+
+	test_data->test_algo = (enum rte_comp_algorithm)id;
+
+	return 0;
+}
+
 static int
 parse_huffman_enc(struct comp_test_data *test_data, const char *arg)
 {
@@ -491,7 +524,7 @@ parse_huffman_enc(struct comp_test_data *test_data, const char *arg)
 	int id = get_str_key_id_mapping(huffman_namemap,
 			RTE_DIM(huffman_namemap), arg);
 	if (id < 0) {
-		RTE_LOG(ERR, USER1, "Invalid Huffmane encoding specified\n");
+		RTE_LOG(ERR, USER1, "Invalid Huffman encoding specified\n");
 		return -1;
 	}
 
@@ -507,7 +540,7 @@ parse_level(struct comp_test_data *test_data, const char *arg)
 
 	/*
 	 * Try parsing the argument as a range, if it fails,
-	 * arse it as a list
+	 * parse it as a list
 	 */
 	if (parse_range(arg, &test_data->level_lst.min,
 			&test_data->level_lst.max,
@@ -572,6 +605,7 @@ static struct option lgopts[] = {
 	{ CPERF_MAX_SGL_SEGS, required_argument, 0, 0},
 	{ CPERF_NUM_ITER, required_argument, 0, 0 },
 	{ CPERF_OPTYPE,	required_argument, 0, 0 },
+	{ CPERF_ALGO, required_argument, 0, 0 },
 	{ CPERF_HUFFMAN_ENC, required_argument, 0, 0 },
 	{ CPERF_LEVEL, required_argument, 0, 0 },
 	{ CPERF_WINDOW_SIZE, required_argument, 0, 0 },
@@ -594,6 +628,7 @@ comp_perf_opts_parse_long(int opt_idx, struct comp_test_data *test_data)
 		{ CPERF_MAX_SGL_SEGS,	parse_max_num_sgl_segs },
 		{ CPERF_NUM_ITER,	parse_num_iter },
 		{ CPERF_OPTYPE,		parse_op_type },
+		{ CPERF_ALGO,		parse_algo },
 		{ CPERF_HUFFMAN_ENC,	parse_huffman_enc },
 		{ CPERF_LEVEL,		parse_level },
 		{ CPERF_WINDOW_SIZE,	parse_window_sz },
@@ -649,6 +684,7 @@ comp_perf_options_default(struct comp_test_data *test_data)
 	test_data->num_iter = 10000;
 	test_data->huffman_enc = RTE_COMP_HUFFMAN_DYNAMIC;
 	test_data->test_op = COMPRESS_DECOMPRESS;
+	test_data->test_algo = RTE_COMP_ALGO_DEFLATE;
 	test_data->window_sz = -1;
 	test_data->level_lst.min = RTE_COMP_LEVEL_MIN;
 	test_data->level_lst.max = RTE_COMP_LEVEL_MAX;
app/test-compress-perf/comp_perf_test_cyclecount.c +2 -2
index ce6c4d7605..81c3d30038 100644
@@ -193,7 +193,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
@@ -208,7 +208,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test-compress-perf/comp_perf_test_throughput.c +2 -2
index c9f8237626..2545ee9925 100644
@@ -84,7 +84,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
@@ -99,7 +99,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test-compress-perf/comp_perf_test_verify.c +2 -2
index 7d6b6abecd..88f4f41851 100644
@@ -87,7 +87,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
@@ -104,7 +104,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test-compress-perf/main.c +24 -14
index bbb4c7917b..d049527ba1 100644
@@ -57,29 +57,39 @@ comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id)
 {
 	const struct rte_compressdev_capabilities *cap;
 
-	cap = rte_compressdev_capability_get(cdev_id,
-					     RTE_COMP_ALGO_DEFLATE);
+	cap = rte_compressdev_capability_get(cdev_id, test_data->test_algo);
 
 	if (cap == NULL) {
 		RTE_LOG(ERR, USER1,
-			"Compress device does not support DEFLATE\n");
+			"Compress device does not support %u algorithm\n",
+			test_data->test_algo);
 		return -1;
 	}
 
 	uint64_t comp_flags = cap->comp_feature_flags;
 
-	/* Huffman encoding */
-	if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED &&
-			(comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0) {
-		RTE_LOG(ERR, USER1,
-			"Compress device does not supported Fixed Huffman\n");
-		return -1;
-	}
+	/* Algorithm type */
+	switch (test_data->test_algo) {
+	case RTE_COMP_ALGO_DEFLATE:
+		/* Huffman encoding */
+		if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED &&
+		    (comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not supported Fixed Huffman\n");
+			return -1;
+		}
 
-	if (test_data->huffman_enc == RTE_COMP_HUFFMAN_DYNAMIC &&
-			(comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) {
-		RTE_LOG(ERR, USER1,
-			"Compress device does not supported Dynamic Huffman\n");
+		if (test_data->huffman_enc == RTE_COMP_HUFFMAN_DYNAMIC &&
+		    (comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not supported Dynamic Huffman\n");
+			return -1;
+		}
+		break;
+	case RTE_COMP_ALGO_LZS:
+	case RTE_COMP_ALGO_NULL:
+		break;
+	default:
 		return -1;
 	}
 
doc/guides/rel_notes/release_23_03.rst +4 -0
index 1f879da6b8..107d4d7603 100644
@@ -66,6 +66,10 @@ New Features
 
   Enable the application options for testing only compress and only decompress.
 
+* **Added algo option in test-compress-perf.**
+
+  Added support for testing other algorithms except for DEFLAT.
+
 
 Removed Items
 -------------
doc/guides/tools/comp_perf.rst +2 -0
index 9d2f4dbe4a..cf9035cc23 100644
@@ -84,6 +84,8 @@ Application Options
 
  ``--operation [comp/decomp/comp_and_decomp]``: perform test on compression, decompression or both operations
 
+ ``--algo [null/deflate/lzs]`` : perform test on algorithm null(DMA), Deflate or lzs (default: Deflate)
+
  ``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default: dynamic)
 
  ``--compress-level N``: compression level, which could be a single value, list or range (default: range between 1 and 9)
Signature · 1 line
2.25.1
#7 lore raw

PATCH 4/4 app/test-compress-perf: add LZ4 support

In reply to Michael Baum

Add support for LZ4 algorithm and add a new parameter for it.

---
 app/test-compress-perf/comp_perf_options.h    |  1 +
 .../comp_perf_options_parse.c                 | 43 ++++++++++++++++++-
 .../comp_perf_test_cyclecount.c               |  7 ++-
 .../comp_perf_test_throughput.c               |  7 ++-
 .../comp_perf_test_verify.c                   |  7 ++-
 app/test-compress-perf/main.c                 | 17 ++++++++
 doc/guides/rel_notes/release_23_03.rst        |  1 +
 doc/guides/tools/comp_perf.rst                |  5 ++-
 8 files changed, 82 insertions(+), 6 deletions(-)
app/test-compress-perf/comp_perf_options.h +1 -0
index 5e5227a700..828a7309d8 100644
@@ -61,6 +61,7 @@ struct comp_test_data {
 	uint16_t max_sgl_segs;
 	uint32_t total_segs;
 
+	uint8_t lz4_flags;
 	enum rte_comp_huffman huffman_enc;
 	enum comp_operation test_op;
 	enum rte_comp_algorithm test_algo;
app/test-compress-perf/comp_perf_options_parse.c +41 -2

93 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_cyclecount.c +6 -1
index 81c3d30038..4d336ec8d6 100644
@@ -194,13 +194,16 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
 				.algo = test_data->test_algo,
-				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->decomp_bufs;
 		output_bufs = mem->comp_bufs;
 		out_seg_sz = test_data->out_seg_sz;
@@ -214,6 +217,8 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->comp_bufs;
 		output_bufs = mem->decomp_bufs;
 		out_seg_sz = test_data->seg_sz;
app/test-compress-perf/comp_perf_test_throughput.c +6 -1
index 2545ee9925..1f7072d223 100644
@@ -85,13 +85,16 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
 				.algo = test_data->test_algo,
-				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->decomp_bufs;
 		output_bufs = mem->comp_bufs;
 		out_seg_sz = test_data->out_seg_sz;
@@ -105,6 +108,8 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->comp_bufs;
 		output_bufs = mem->decomp_bufs;
 		out_seg_sz = test_data->seg_sz;
app/test-compress-perf/comp_perf_test_verify.c +6 -1
index 88f4f41851..7bd18073cf 100644
@@ -88,13 +88,16 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
 				.algo = test_data->test_algo,
-				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		output_data_ptr = ctx->mem.compressed_data;
 		output_data_sz = &ctx->comp_data_sz;
 		input_bufs = mem->decomp_bufs;
@@ -110,6 +113,8 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		output_data_ptr = ctx->mem.decompressed_data;
 		output_data_sz = &ctx->decomp_data_sz;
 		input_bufs = mem->comp_bufs;
app/test-compress-perf/main.c +17 -0
index d049527ba1..fa366123ed 100644
@@ -86,6 +86,23 @@ comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id)
 			return -1;
 		}
 		break;
+	case RTE_COMP_ALGO_LZ4:
+		/* LZ4 flags */
+		if ((test_data->lz4_flags & RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM) &&
+		    (comp_flags & RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not support LZ4 block with checksum\n");
+			return -1;
+		}
+
+		if ((test_data->lz4_flags &
+		     RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE) &&
+		    (comp_flags & RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not support LZ4 independent blocks\n");
+			return -1;
+		}
+		break;
 	case RTE_COMP_ALGO_LZS:
 	case RTE_COMP_ALGO_NULL:
 		break;
doc/guides/rel_notes/release_23_03.rst +1 -0
index 107d4d7603..7ae012ae33 100644
@@ -61,6 +61,7 @@ New Features
 
   * Added support for ``RTE_COMP_ALGO_LZ4``.
   * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
+  * Added support for ``lz4`` in test-compress-perf algo options.
 
 * **Allowed test single compress operation in test-compress-perf.**
 
doc/guides/tools/comp_perf.rst +4 -1
index cf9035cc23..5eb05a095c 100644
@@ -84,10 +84,13 @@ Application Options
 
  ``--operation [comp/decomp/comp_and_decomp]``: perform test on compression, decompression or both operations
 
- ``--algo [null/deflate/lzs]`` : perform test on algorithm null(DMA), Deflate or lzs (default: Deflate)
+ ``--algo [null/deflate/lzs/lz4]`` : perform test on algorithm null(DMA), Deflate, lzs or lz4 (default: Deflate)
 
  ``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default: dynamic)
 
+ ``--lz4-flags N``: flags to for LZ4 parameters,
+ see `LZ4 Frame Descriptor <https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#frame-descriptor>`_ (default: no flags)
+
  ``--compress-level N``: compression level, which could be a single value, list or range (default: range between 1 and 9)
 
  ``--window-sz N``: base two log value of compression window size (default: max supported by PMD)
Signature · 1 line
2.25.1
#8 lore raw

v2 cover compressdev: add LZ4 support

In reply to Michael Baum

Add support for LZ4 compression algorithm for both API and app.

In addition, enable app to test de/comp only. This option already exists in both app doc and arguments but isn't implemented inside the application.

LZ4: https://github.com/lz4/lz4
RFC:
https://patchwork.dpdk.org/project/dpdk/patch/[email protected]/

v2:
- Add macros for all LZ4 flags in the standard.
- Rebase.

Depends-on: series-26738 ("compressdev: fix end of list enums conflict")

Michael Baum (4):
compressdev: add LZ4 algorithm support app/test-compress-perf: allow test single compress operation app/test-compress-perf: add algo option app/test-compress-perf: add LZ4 support

 app/test-compress-perf/comp_perf_options.h    |   9 +-
 .../comp_perf_options_parse.c                 |  83 +++++++++++-
 .../comp_perf_test_common.c                   | 126 +++++++++++++-----
 .../comp_perf_test_cyclecount.c               |  82 +++++++-----
 .../comp_perf_test_throughput.c               |  75 +++++++----
 .../comp_perf_test_verify.c                   |  75 +++++++----
 app/test-compress-perf/main.c                 |  63 +++++++--
 doc/guides/compressdevs/features/default.ini  |   7 +
 doc/guides/rel_notes/release_23_03.rst        |  16 +++
 doc/guides/tools/comp_perf.rst                |   5 +
 lib/compressdev/rte_comp.c                    |  12 ++
 lib/compressdev/rte_comp.h                    |  79 ++++++++++-
 12 files changed, 488 insertions(+), 144 deletions(-)
Signature · 1 line
2.25.1
#9 lore raw

PATCH v2 1/4 compressdev: add LZ4 algorithm support

Add support for LZ4 algorithm:
- Add Lz4 param structure to XFORM structures.
- Add capabilities flags for LZ4 params.
- Add xxHash-32 checksum and capabilities flag.

---
 doc/guides/compressdevs/features/default.ini |  7 ++
 doc/guides/rel_notes/release_23_03.rst       |  7 ++
 lib/compressdev/rte_comp.c                   | 12 +++
 lib/compressdev/rte_comp.h                   | 79 +++++++++++++++++++-
 4 files changed, 103 insertions(+), 2 deletions(-)
doc/guides/compressdevs/features/default.ini +7 -0
index e1419ee8db..2f178c5efd 100644
@@ -20,8 +20,15 @@ OOP SGL In LB  Out     =
 OOP LB  In SGL Out     =
 Deflate                =
 LZS                    =
+LZ4                    =
 Adler32                =
 Crc32                  =
 Adler32&Crc32          =
+xxHash32               =
 Fixed                  =
 Dynamic                =
+LZ4 Dictionary ID      =
+LZ4 Content Checksum   =
+LZ4 Content Size       =
+LZ4 Block Checksum     =
+LZ4 Block Independence =
doc/guides/rel_notes/release_23_03.rst +7 -0
index 73f5d94e14..2722296180 100644
@@ -78,6 +78,13 @@ New Features
     ``rte_event_dev_config::nb_single_link_event_port_queues`` parameter
     required for eth_rx, eth_tx, crypto and timer eventdev adapters.
 
+* **Added LZ4 algorithm in Compressdev Library.**
+
+  Added new compression algorithm, including:
+
+  * Added support for ``RTE_COMP_ALGO_LZ4``.
+  * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
+
 
 Removed Items
 -------------
lib/compressdev/rte_comp.c +12 -0
index 320c6dab92..f060c68557 100644
@@ -39,6 +39,18 @@ rte_comp_get_feature_name(uint64_t flag)
 		return "HUFFMAN_FIXED";
 	case RTE_COMP_FF_HUFFMAN_DYNAMIC:
 		return "HUFFMAN_DYNAMIC";
+	case RTE_COMP_FF_XXHASH32_CHECKSUM:
+		return "XXHASH32_CHECKSUM";
+	case RTE_COMP_FF_LZ4_DICT_ID:
+		return "LZ4_DICT_ID";
+	case RTE_COMP_FF_LZ4_CONTENT_WITH_CHECKSUM:
+		return "LZ4_CONTENT_WITH_CHECKSUM";
+	case RTE_COMP_FF_LZ4_CONTENT_SIZE:
+		return "LZ4_CONTENT_SIZE";
+	case RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE:
+		return "LZ4_BLOCK_INDEPENDENCE";
+	case RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM:
+		return "LZ4_BLOCK_WITH_CHECKSUM";
 	default:
 		return NULL;
 	}
lib/compressdev/rte_comp.h +77 -2

123 lines not on this page · raw message

Signature · 1 line
2.25.1
#10 lore raw

RE: [EXT] [PATCH v2 1/4] compressdev: add LZ4 algorithm support

Quoted 7 lines

Add support for LZ4 algorithm:
- Add Lz4 param structure to XFORM structures.
- Add capabilities flags for LZ4 params.
- Add xxHash-32 checksum and capabilities flag.

---

Please review the series, or else it will be merged in RC1.

#11 lore raw

PATCH v2 2/4 app/test-compress-perf: allow test single compress operation

In reply to Michael Baum

Part of the application options is to test only compress and only decompress but actually the application ignores this user option and tries to test always both compress and decompress.

Allow testing only compress and only decompress.

---
 app/test-compress-perf/comp_perf_options.h    |   6 +-
 .../comp_perf_options_parse.c                 |   4 +-
 .../comp_perf_test_common.c                   | 126 +++++++++++++-----
 .../comp_perf_test_cyclecount.c               |  71 ++++++----
 .../comp_perf_test_throughput.c               |  64 +++++----
 .../comp_perf_test_verify.c                   |  64 +++++----
 app/test-compress-perf/main.c                 |   8 ++
 doc/guides/rel_notes/release_23_03.rst        |   4 +
 8 files changed, 230 insertions(+), 117 deletions(-)
app/test-compress-perf/comp_perf_options.h +3 -3
index 57dd146330..d00b299247 100644
@@ -32,9 +32,9 @@ enum cperf_test_type {
 };
 
 enum comp_operation {
-	COMPRESS_ONLY,
-	DECOMPRESS_ONLY,
-	COMPRESS_DECOMPRESS
+	COMPRESS = (1 << 0),
+	DECOMPRESS = (1 << 1),
+	COMPRESS_DECOMPRESS = (COMPRESS | DECOMPRESS),
 };
 
 struct range_list {
app/test-compress-perf/comp_perf_options_parse.c +2 -2
index 019eddb7bd..7a992bf43e 100644
@@ -446,11 +446,11 @@ parse_op_type(struct comp_test_data *test_data, const char *arg)
 	struct name_id_map optype_namemap[] = {
 		{
 			"comp",
-			COMPRESS_ONLY
+			COMPRESS
 		},
 		{
 			"decomp",
-			DECOMPRESS_ONLY
+			DECOMPRESS
 		},
 		{
 			"comp_and_decomp",
app/test-compress-perf/comp_perf_test_common.c +91 -35

222 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_cyclecount.c +44 -27

89 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_throughput.c +38 -26

79 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_verify.c +40 -24

105 lines not on this page · raw message

app/test-compress-perf/main.c +8 -0
index 41b8edc2bd..bbb4c7917b 100644
@@ -254,6 +254,14 @@ comp_perf_dump_input_data(struct comp_test_data *test_data)
 		goto end;
 	}
 
+	if (!(test_data->test_op & COMPRESS) &&
+	    test_data->input_data_sz >
+	    (size_t) test_data->seg_sz * (size_t) test_data->max_sgl_segs) {
+		RTE_LOG(ERR, USER1,
+			"Size of input must be less than total segments\n");
+		goto end;
+	}
+
 	test_data->input_data = rte_zmalloc_socket(NULL,
 				test_data->input_data_sz, 0, rte_socket_id());
 
doc/guides/rel_notes/release_23_03.rst +4 -0
index 2722296180..15fa8e812b 100644
@@ -85,6 +85,10 @@ New Features
   * Added support for ``RTE_COMP_ALGO_LZ4``.
   * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
 
+* **Allowed test single compress operation in test-compress-perf.**
+
+  Enable the application options for testing only compress and only decompress.
+
 
 Removed Items
 -------------
Signature · 1 line
2.25.1
#12 lore raw

PATCH v2 3/4 app/test-compress-perf: add algo option

In reply to Michael Baum

Add a command line option algo to select the compress algorithm supported by the compress API: null (DMA), deflate and lzs.

Default for deflate.

---
 app/test-compress-perf/comp_perf_options.h    |  2 +
 .../comp_perf_options_parse.c                 | 40 ++++++++++++++++++-
 .../comp_perf_test_cyclecount.c               |  4 +-
 .../comp_perf_test_throughput.c               |  4 +-
 .../comp_perf_test_verify.c                   |  4 +-
 app/test-compress-perf/main.c                 | 38 +++++++++++-------
 doc/guides/rel_notes/release_23_03.rst        |  4 ++
 doc/guides/tools/comp_perf.rst                |  2 +
 8 files changed, 76 insertions(+), 22 deletions(-)
app/test-compress-perf/comp_perf_options.h +2 -0
index d00b299247..5e5227a700 100644
@@ -63,6 +63,8 @@ struct comp_test_data {
 
 	enum rte_comp_huffman huffman_enc;
 	enum comp_operation test_op;
+	enum rte_comp_algorithm test_algo;
+
 	int window_sz;
 	struct range_list level_lst;
 	uint8_t level;
app/test-compress-perf/comp_perf_options_parse.c +38 -2
index 7a992bf43e..97ddff87ef 100644
@@ -25,6 +25,7 @@
 #define CPERF_MAX_SGL_SEGS	("max-num-sgl-segs")
 #define CPERF_NUM_ITER		("num-iter")
 #define CPERF_OPTYPE		("operation")
+#define CPERF_ALGO		("algo")
 #define CPERF_HUFFMAN_ENC	("huffman-enc")
 #define CPERF_LEVEL		("compress-level")
 #define CPERF_WINDOW_SIZE	("window-sz")
@@ -56,6 +57,8 @@ usage(char *progname)
 		"		compressed/decompressed (default: 10000)\n"
 		" --operation [comp/decomp/comp_and_decomp]: perform test on\n"
 		"		compression, decompression or both operations\n"
+		" --algo [null/deflate/lzs]: perform test on algorithm\n"
+		"		null(DMA), deflate or lzs (default: deflate)\n"
 		" --huffman-enc [fixed/dynamic/default]: Huffman encoding\n"
 		"		(default: dynamic)\n"
 		" --compress-level N: compression level, which could be a single value, list or range\n"
@@ -470,6 +473,36 @@ parse_op_type(struct comp_test_data *test_data, const char *arg)
 	return 0;
 }
 
+static int
+parse_algo(struct comp_test_data *test_data, const char *arg)
+{
+	struct name_id_map algo_namemap[] = {
+		{
+			"null",
+			RTE_COMP_ALGO_NULL
+		},
+		{
+			"deflate",
+			RTE_COMP_ALGO_DEFLATE
+		},
+		{
+			"lzs",
+			RTE_COMP_ALGO_LZS
+		}
+	};
+
+	int id = get_str_key_id_mapping(algo_namemap,
+			RTE_DIM(algo_namemap), arg);
+	if (id < 0) {
+		RTE_LOG(ERR, USER1, "Invalid algorithm specified\n");
+		return -1;
+	}
+
+	test_data->test_algo = (enum rte_comp_algorithm)id;
+
+	return 0;
+}
+
 static int
 parse_huffman_enc(struct comp_test_data *test_data, const char *arg)
 {
@@ -491,7 +524,7 @@ parse_huffman_enc(struct comp_test_data *test_data, const char *arg)
 	int id = get_str_key_id_mapping(huffman_namemap,
 			RTE_DIM(huffman_namemap), arg);
 	if (id < 0) {
-		RTE_LOG(ERR, USER1, "Invalid Huffmane encoding specified\n");
+		RTE_LOG(ERR, USER1, "Invalid Huffman encoding specified\n");
 		return -1;
 	}
 
@@ -507,7 +540,7 @@ parse_level(struct comp_test_data *test_data, const char *arg)
 
 	/*
 	 * Try parsing the argument as a range, if it fails,
-	 * arse it as a list
+	 * parse it as a list
 	 */
 	if (parse_range(arg, &test_data->level_lst.min,
 			&test_data->level_lst.max,
@@ -572,6 +605,7 @@ static struct option lgopts[] = {
 	{ CPERF_MAX_SGL_SEGS, required_argument, 0, 0},
 	{ CPERF_NUM_ITER, required_argument, 0, 0 },
 	{ CPERF_OPTYPE,	required_argument, 0, 0 },
+	{ CPERF_ALGO, required_argument, 0, 0 },
 	{ CPERF_HUFFMAN_ENC, required_argument, 0, 0 },
 	{ CPERF_LEVEL, required_argument, 0, 0 },
 	{ CPERF_WINDOW_SIZE, required_argument, 0, 0 },
@@ -594,6 +628,7 @@ comp_perf_opts_parse_long(int opt_idx, struct comp_test_data *test_data)
 		{ CPERF_MAX_SGL_SEGS,	parse_max_num_sgl_segs },
 		{ CPERF_NUM_ITER,	parse_num_iter },
 		{ CPERF_OPTYPE,		parse_op_type },
+		{ CPERF_ALGO,		parse_algo },
 		{ CPERF_HUFFMAN_ENC,	parse_huffman_enc },
 		{ CPERF_LEVEL,		parse_level },
 		{ CPERF_WINDOW_SIZE,	parse_window_sz },
@@ -649,6 +684,7 @@ comp_perf_options_default(struct comp_test_data *test_data)
 	test_data->num_iter = 10000;
 	test_data->huffman_enc = RTE_COMP_HUFFMAN_DYNAMIC;
 	test_data->test_op = COMPRESS_DECOMPRESS;
+	test_data->test_algo = RTE_COMP_ALGO_DEFLATE;
 	test_data->window_sz = -1;
 	test_data->level_lst.min = RTE_COMP_LEVEL_MIN;
 	test_data->level_lst.max = RTE_COMP_LEVEL_MAX;
app/test-compress-perf/comp_perf_test_cyclecount.c +2 -2
index ce6c4d7605..81c3d30038 100644
@@ -193,7 +193,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
@@ -208,7 +208,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test-compress-perf/comp_perf_test_throughput.c +2 -2
index c9f8237626..2545ee9925 100644
@@ -84,7 +84,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
@@ -99,7 +99,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test-compress-perf/comp_perf_test_verify.c +2 -2
index 7d6b6abecd..88f4f41851 100644
@@ -87,7 +87,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
@@ -104,7 +104,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test-compress-perf/main.c +24 -14
index bbb4c7917b..d049527ba1 100644
@@ -57,29 +57,39 @@ comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id)
 {
 	const struct rte_compressdev_capabilities *cap;
 
-	cap = rte_compressdev_capability_get(cdev_id,
-					     RTE_COMP_ALGO_DEFLATE);
+	cap = rte_compressdev_capability_get(cdev_id, test_data->test_algo);
 
 	if (cap == NULL) {
 		RTE_LOG(ERR, USER1,
-			"Compress device does not support DEFLATE\n");
+			"Compress device does not support %u algorithm\n",
+			test_data->test_algo);
 		return -1;
 	}
 
 	uint64_t comp_flags = cap->comp_feature_flags;
 
-	/* Huffman encoding */
-	if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED &&
-			(comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0) {
-		RTE_LOG(ERR, USER1,
-			"Compress device does not supported Fixed Huffman\n");
-		return -1;
-	}
+	/* Algorithm type */
+	switch (test_data->test_algo) {
+	case RTE_COMP_ALGO_DEFLATE:
+		/* Huffman encoding */
+		if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED &&
+		    (comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not supported Fixed Huffman\n");
+			return -1;
+		}
 
-	if (test_data->huffman_enc == RTE_COMP_HUFFMAN_DYNAMIC &&
-			(comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) {
-		RTE_LOG(ERR, USER1,
-			"Compress device does not supported Dynamic Huffman\n");
+		if (test_data->huffman_enc == RTE_COMP_HUFFMAN_DYNAMIC &&
+		    (comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not supported Dynamic Huffman\n");
+			return -1;
+		}
+		break;
+	case RTE_COMP_ALGO_LZS:
+	case RTE_COMP_ALGO_NULL:
+		break;
+	default:
 		return -1;
 	}
 
doc/guides/rel_notes/release_23_03.rst +4 -0
index 15fa8e812b..da345ded48 100644
@@ -89,6 +89,10 @@ New Features
 
   Enable the application options for testing only compress and only decompress.
 
+* **Added algo option in test-compress-perf.**
+
+  Added support for testing other algorithms except for DEFLAT.
+
 
 Removed Items
 -------------
doc/guides/tools/comp_perf.rst +2 -0
index 9d2f4dbe4a..cf9035cc23 100644
@@ -84,6 +84,8 @@ Application Options
 
  ``--operation [comp/decomp/comp_and_decomp]``: perform test on compression, decompression or both operations
 
+ ``--algo [null/deflate/lzs]`` : perform test on algorithm null(DMA), Deflate or lzs (default: Deflate)
+
  ``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default: dynamic)
 
  ``--compress-level N``: compression level, which could be a single value, list or range (default: range between 1 and 9)
Signature · 1 line
2.25.1
#13 lore raw

PATCH v2 4/4 app/test-compress-perf: add LZ4 support

In reply to Michael Baum

Add support for LZ4 algorithm and add a new parameter for it.

---
 app/test-compress-perf/comp_perf_options.h    |  1 +
 .../comp_perf_options_parse.c                 | 43 ++++++++++++++++++-
 .../comp_perf_test_cyclecount.c               |  7 ++-
 .../comp_perf_test_throughput.c               |  7 ++-
 .../comp_perf_test_verify.c                   |  7 ++-
 app/test-compress-perf/main.c                 | 17 ++++++++
 doc/guides/rel_notes/release_23_03.rst        |  1 +
 doc/guides/tools/comp_perf.rst                |  5 ++-
 8 files changed, 82 insertions(+), 6 deletions(-)
app/test-compress-perf/comp_perf_options.h +1 -0
index 5e5227a700..828a7309d8 100644
@@ -61,6 +61,7 @@ struct comp_test_data {
 	uint16_t max_sgl_segs;
 	uint32_t total_segs;
 
+	uint8_t lz4_flags;
 	enum rte_comp_huffman huffman_enc;
 	enum comp_operation test_op;
 	enum rte_comp_algorithm test_algo;
app/test-compress-perf/comp_perf_options_parse.c +41 -2

93 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_cyclecount.c +6 -1
index 81c3d30038..4d336ec8d6 100644
@@ -194,13 +194,16 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
 				.algo = test_data->test_algo,
-				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->decomp_bufs;
 		output_bufs = mem->comp_bufs;
 		out_seg_sz = test_data->out_seg_sz;
@@ -214,6 +217,8 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->comp_bufs;
 		output_bufs = mem->decomp_bufs;
 		out_seg_sz = test_data->seg_sz;
app/test-compress-perf/comp_perf_test_throughput.c +6 -1
index 2545ee9925..1f7072d223 100644
@@ -85,13 +85,16 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
 				.algo = test_data->test_algo,
-				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->decomp_bufs;
 		output_bufs = mem->comp_bufs;
 		out_seg_sz = test_data->out_seg_sz;
@@ -105,6 +108,8 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->comp_bufs;
 		output_bufs = mem->decomp_bufs;
 		out_seg_sz = test_data->seg_sz;
app/test-compress-perf/comp_perf_test_verify.c +6 -1
index 88f4f41851..7bd18073cf 100644
@@ -88,13 +88,16 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
 				.algo = test_data->test_algo,
-				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		output_data_ptr = ctx->mem.compressed_data;
 		output_data_sz = &ctx->comp_data_sz;
 		input_bufs = mem->decomp_bufs;
@@ -110,6 +113,8 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		output_data_ptr = ctx->mem.decompressed_data;
 		output_data_sz = &ctx->decomp_data_sz;
 		input_bufs = mem->comp_bufs;
app/test-compress-perf/main.c +17 -0
index d049527ba1..fa366123ed 100644
@@ -86,6 +86,23 @@ comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id)
 			return -1;
 		}
 		break;
+	case RTE_COMP_ALGO_LZ4:
+		/* LZ4 flags */
+		if ((test_data->lz4_flags & RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM) &&
+		    (comp_flags & RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not support LZ4 block with checksum\n");
+			return -1;
+		}
+
+		if ((test_data->lz4_flags &
+		     RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE) &&
+		    (comp_flags & RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not support LZ4 independent blocks\n");
+			return -1;
+		}
+		break;
 	case RTE_COMP_ALGO_LZS:
 	case RTE_COMP_ALGO_NULL:
 		break;
doc/guides/rel_notes/release_23_03.rst +1 -0
index da345ded48..aedc5767ff 100644
@@ -84,6 +84,7 @@ New Features
 
   * Added support for ``RTE_COMP_ALGO_LZ4``.
   * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
+  * Added support for ``lz4`` in test-compress-perf algo options.
 
 * **Allowed test single compress operation in test-compress-perf.**
 
doc/guides/tools/comp_perf.rst +4 -1
index cf9035cc23..5eb05a095c 100644
@@ -84,10 +84,13 @@ Application Options
 
  ``--operation [comp/decomp/comp_and_decomp]``: perform test on compression, decompression or both operations
 
- ``--algo [null/deflate/lzs]`` : perform test on algorithm null(DMA), Deflate or lzs (default: Deflate)
+ ``--algo [null/deflate/lzs/lz4]`` : perform test on algorithm null(DMA), Deflate, lzs or lz4 (default: Deflate)
 
  ``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default: dynamic)
 
+ ``--lz4-flags N``: flags to for LZ4 parameters,
+ see `LZ4 Frame Descriptor <https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#frame-descriptor>`_ (default: no flags)
+
  ``--compress-level N``: compression level, which could be a single value, list or range (default: range between 1 and 9)
 
  ``--window-sz N``: base two log value of compression window size (default: max supported by PMD)
Signature · 1 line
2.25.1
#14 lore raw

v3 cover compressdev: add LZ4 support

In reply to Michael Baum

Add support for LZ4 compression algorithm for both API and app.

In addition, enable app to test de/comp only. This option already exists in both app doc and arguments but isn't implemented inside the application.

LZ4: https://github.com/lz4/lz4
RFC:
https://patchwork.dpdk.org/project/dpdk/patch/[email protected]/

v2:
- Add macros for all LZ4 flags in the standard.
- Rebase.

v3:
- Rebase.

Michael Baum (4):
compressdev: add LZ4 algorithm support app/test-compress-perf: allow test single compress operation app/test-compress-perf: add algo option app/test-compress-perf: add LZ4 support

 app/test-compress-perf/comp_perf_options.h    |   9 +-
 .../comp_perf_options_parse.c                 |  83 +++++++++++-
 .../comp_perf_test_common.c                   | 126 +++++++++++++-----
 .../comp_perf_test_cyclecount.c               |  82 +++++++-----
 .../comp_perf_test_throughput.c               |  75 +++++++----
 .../comp_perf_test_verify.c                   |  75 +++++++----
 app/test-compress-perf/main.c                 |  63 +++++++--
 doc/guides/compressdevs/features/default.ini  |   7 +
 doc/guides/rel_notes/release_23_03.rst        |  16 +++
 doc/guides/tools/comp_perf.rst                |   5 +
 lib/compressdev/rte_comp.c                    |  12 ++
 lib/compressdev/rte_comp.h                    |  79 ++++++++++-
 12 files changed, 488 insertions(+), 144 deletions(-)
Signature · 1 line
2.25.1
#15 lore raw

PATCH v3 1/4 compressdev: add LZ4 algorithm support

Add support for LZ4 algorithm:
- Add Lz4 param structure to XFORM structures.
- Add capabilities flags for LZ4 params.
- Add xxHash-32 checksum and capabilities flag.

---
 doc/guides/compressdevs/features/default.ini |  7 ++
 doc/guides/rel_notes/release_23_03.rst       |  7 ++
 lib/compressdev/rte_comp.c                   | 12 +++
 lib/compressdev/rte_comp.h                   | 79 +++++++++++++++++++-
 4 files changed, 103 insertions(+), 2 deletions(-)
doc/guides/compressdevs/features/default.ini +7 -0
index e1419ee8db..2f178c5efd 100644
@@ -20,8 +20,15 @@ OOP SGL In LB  Out     =
 OOP LB  In SGL Out     =
 Deflate                =
 LZS                    =
+LZ4                    =
 Adler32                =
 Crc32                  =
 Adler32&Crc32          =
+xxHash32               =
 Fixed                  =
 Dynamic                =
+LZ4 Dictionary ID      =
+LZ4 Content Checksum   =
+LZ4 Content Size       =
+LZ4 Block Checksum     =
+LZ4 Block Independence =
doc/guides/rel_notes/release_23_03.rst +7 -0
index 7527c6d57f..790c2a5eef 100644
@@ -105,6 +105,13 @@ New Features
   * Added support to capture packets at each graph node with packet metadata and
     node name.
 
+* **Added LZ4 algorithm in Compressdev Library.**
+
+  Added new compression algorithm, including:
+
+  * Added support for ``RTE_COMP_ALGO_LZ4``.
+  * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
+
 
 Removed Items
 -------------
lib/compressdev/rte_comp.c +12 -0
index 320c6dab92..f060c68557 100644
@@ -39,6 +39,18 @@ rte_comp_get_feature_name(uint64_t flag)
 		return "HUFFMAN_FIXED";
 	case RTE_COMP_FF_HUFFMAN_DYNAMIC:
 		return "HUFFMAN_DYNAMIC";
+	case RTE_COMP_FF_XXHASH32_CHECKSUM:
+		return "XXHASH32_CHECKSUM";
+	case RTE_COMP_FF_LZ4_DICT_ID:
+		return "LZ4_DICT_ID";
+	case RTE_COMP_FF_LZ4_CONTENT_WITH_CHECKSUM:
+		return "LZ4_CONTENT_WITH_CHECKSUM";
+	case RTE_COMP_FF_LZ4_CONTENT_SIZE:
+		return "LZ4_CONTENT_SIZE";
+	case RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE:
+		return "LZ4_BLOCK_INDEPENDENCE";
+	case RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM:
+		return "LZ4_BLOCK_WITH_CHECKSUM";
 	default:
 		return NULL;
 	}
lib/compressdev/rte_comp.h +77 -2

123 lines not on this page · raw message

Signature · 1 line
2.25.1
#16 lore raw

PATCH v3 2/4 app/test-compress-perf: allow test single compress operation

In reply to Michael Baum

Part of the application options is to test only compress and only decompress but actually the application ignores this user option and tries to test always both compress and decompress.

Allow testing only compress and only decompress.

---
 app/test-compress-perf/comp_perf_options.h    |   6 +-
 .../comp_perf_options_parse.c                 |   4 +-
 .../comp_perf_test_common.c                   | 126 +++++++++++++-----
 .../comp_perf_test_cyclecount.c               |  71 ++++++----
 .../comp_perf_test_throughput.c               |  64 +++++----
 .../comp_perf_test_verify.c                   |  64 +++++----
 app/test-compress-perf/main.c                 |   8 ++
 doc/guides/rel_notes/release_23_03.rst        |   4 +
 8 files changed, 230 insertions(+), 117 deletions(-)
app/test-compress-perf/comp_perf_options.h +3 -3
index 57dd146330..d00b299247 100644
@@ -32,9 +32,9 @@ enum cperf_test_type {
 };
 
 enum comp_operation {
-	COMPRESS_ONLY,
-	DECOMPRESS_ONLY,
-	COMPRESS_DECOMPRESS
+	COMPRESS = (1 << 0),
+	DECOMPRESS = (1 << 1),
+	COMPRESS_DECOMPRESS = (COMPRESS | DECOMPRESS),
 };
 
 struct range_list {
app/test-compress-perf/comp_perf_options_parse.c +2 -2
index 019eddb7bd..7a992bf43e 100644
@@ -446,11 +446,11 @@ parse_op_type(struct comp_test_data *test_data, const char *arg)
 	struct name_id_map optype_namemap[] = {
 		{
 			"comp",
-			COMPRESS_ONLY
+			COMPRESS
 		},
 		{
 			"decomp",
-			DECOMPRESS_ONLY
+			DECOMPRESS
 		},
 		{
 			"comp_and_decomp",
app/test-compress-perf/comp_perf_test_common.c +91 -35

222 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_cyclecount.c +44 -27

89 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_throughput.c +38 -26

79 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_verify.c +40 -24

105 lines not on this page · raw message

app/test-compress-perf/main.c +8 -0
index 41b8edc2bd..bbb4c7917b 100644
@@ -254,6 +254,14 @@ comp_perf_dump_input_data(struct comp_test_data *test_data)
 		goto end;
 	}
 
+	if (!(test_data->test_op & COMPRESS) &&
+	    test_data->input_data_sz >
+	    (size_t) test_data->seg_sz * (size_t) test_data->max_sgl_segs) {
+		RTE_LOG(ERR, USER1,
+			"Size of input must be less than total segments\n");
+		goto end;
+	}
+
 	test_data->input_data = rte_zmalloc_socket(NULL,
 				test_data->input_data_sz, 0, rte_socket_id());
 
doc/guides/rel_notes/release_23_03.rst +4 -0
index 790c2a5eef..b2a8c921b9 100644
@@ -112,6 +112,10 @@ New Features
   * Added support for ``RTE_COMP_ALGO_LZ4``.
   * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
 
+* **Allowed test single compress operation in test-compress-perf.**
+
+  Enable the application options for testing only compress and only decompress.
+
 
 Removed Items
 -------------
Signature · 1 line
2.25.1
#17 lore raw

RE: [EXT] [PATCH v3 2/4] app/test-compress-perf: allow test single compress operation

Quoted 4 lines

Part of the application options is to test only compress and only decompress but actually the application ignores this user option and tries to test always both compress and decompress.

From the description it seems to be a bug in previous implementation. Can we back port it? And update the patch title accordingly? Also, release note update is not required for this fix.

Quoted 5 lines

Allow testing only compress and only decompress.

#18 lore raw

RE: [EXT] [PATCH v3 2/4] app/test-compress-perf: allow test single compress operation

Quoted 3 lines · Akhil Goyal

From the description it seems to be a bug in previous implementation. Can we back port it? And update the patch title accordingly?

I'll update and send a new version.

Quoted 1 line

Also, release note update is not required for this fix.

I'll remove it, thank you.

Quoted 6 lines
Quoted 6 lines

Allow testing only compress and only decompress.

#19 lore raw

PATCH v3 3/4 app/test-compress-perf: add algo option

In reply to Michael Baum

Add a command line option algo to select the compress algorithm supported by the compress API: null (DMA), deflate and lzs.

Default for deflate.

---
 app/test-compress-perf/comp_perf_options.h    |  2 +
 .../comp_perf_options_parse.c                 | 40 ++++++++++++++++++-
 .../comp_perf_test_cyclecount.c               |  4 +-
 .../comp_perf_test_throughput.c               |  4 +-
 .../comp_perf_test_verify.c                   |  4 +-
 app/test-compress-perf/main.c                 | 38 +++++++++++-------
 doc/guides/rel_notes/release_23_03.rst        |  4 ++
 doc/guides/tools/comp_perf.rst                |  2 +
 8 files changed, 76 insertions(+), 22 deletions(-)
app/test-compress-perf/comp_perf_options.h +2 -0
index d00b299247..5e5227a700 100644
@@ -63,6 +63,8 @@ struct comp_test_data {
 
 	enum rte_comp_huffman huffman_enc;
 	enum comp_operation test_op;
+	enum rte_comp_algorithm test_algo;
+
 	int window_sz;
 	struct range_list level_lst;
 	uint8_t level;
app/test-compress-perf/comp_perf_options_parse.c +38 -2
index 7a992bf43e..97ddff87ef 100644
@@ -25,6 +25,7 @@
 #define CPERF_MAX_SGL_SEGS	("max-num-sgl-segs")
 #define CPERF_NUM_ITER		("num-iter")
 #define CPERF_OPTYPE		("operation")
+#define CPERF_ALGO		("algo")
 #define CPERF_HUFFMAN_ENC	("huffman-enc")
 #define CPERF_LEVEL		("compress-level")
 #define CPERF_WINDOW_SIZE	("window-sz")
@@ -56,6 +57,8 @@ usage(char *progname)
 		"		compressed/decompressed (default: 10000)\n"
 		" --operation [comp/decomp/comp_and_decomp]: perform test on\n"
 		"		compression, decompression or both operations\n"
+		" --algo [null/deflate/lzs]: perform test on algorithm\n"
+		"		null(DMA), deflate or lzs (default: deflate)\n"
 		" --huffman-enc [fixed/dynamic/default]: Huffman encoding\n"
 		"		(default: dynamic)\n"
 		" --compress-level N: compression level, which could be a single value, list or range\n"
@@ -470,6 +473,36 @@ parse_op_type(struct comp_test_data *test_data, const char *arg)
 	return 0;
 }
 
+static int
+parse_algo(struct comp_test_data *test_data, const char *arg)
+{
+	struct name_id_map algo_namemap[] = {
+		{
+			"null",
+			RTE_COMP_ALGO_NULL
+		},
+		{
+			"deflate",
+			RTE_COMP_ALGO_DEFLATE
+		},
+		{
+			"lzs",
+			RTE_COMP_ALGO_LZS
+		}
+	};
+
+	int id = get_str_key_id_mapping(algo_namemap,
+			RTE_DIM(algo_namemap), arg);
+	if (id < 0) {
+		RTE_LOG(ERR, USER1, "Invalid algorithm specified\n");
+		return -1;
+	}
+
+	test_data->test_algo = (enum rte_comp_algorithm)id;
+
+	return 0;
+}
+
 static int
 parse_huffman_enc(struct comp_test_data *test_data, const char *arg)
 {
@@ -491,7 +524,7 @@ parse_huffman_enc(struct comp_test_data *test_data, const char *arg)
 	int id = get_str_key_id_mapping(huffman_namemap,
 			RTE_DIM(huffman_namemap), arg);
 	if (id < 0) {
-		RTE_LOG(ERR, USER1, "Invalid Huffmane encoding specified\n");
+		RTE_LOG(ERR, USER1, "Invalid Huffman encoding specified\n");
 		return -1;
 	}
 
@@ -507,7 +540,7 @@ parse_level(struct comp_test_data *test_data, const char *arg)
 
 	/*
 	 * Try parsing the argument as a range, if it fails,
-	 * arse it as a list
+	 * parse it as a list
 	 */
 	if (parse_range(arg, &test_data->level_lst.min,
 			&test_data->level_lst.max,
@@ -572,6 +605,7 @@ static struct option lgopts[] = {
 	{ CPERF_MAX_SGL_SEGS, required_argument, 0, 0},
 	{ CPERF_NUM_ITER, required_argument, 0, 0 },
 	{ CPERF_OPTYPE,	required_argument, 0, 0 },
+	{ CPERF_ALGO, required_argument, 0, 0 },
 	{ CPERF_HUFFMAN_ENC, required_argument, 0, 0 },
 	{ CPERF_LEVEL, required_argument, 0, 0 },
 	{ CPERF_WINDOW_SIZE, required_argument, 0, 0 },
@@ -594,6 +628,7 @@ comp_perf_opts_parse_long(int opt_idx, struct comp_test_data *test_data)
 		{ CPERF_MAX_SGL_SEGS,	parse_max_num_sgl_segs },
 		{ CPERF_NUM_ITER,	parse_num_iter },
 		{ CPERF_OPTYPE,		parse_op_type },
+		{ CPERF_ALGO,		parse_algo },
 		{ CPERF_HUFFMAN_ENC,	parse_huffman_enc },
 		{ CPERF_LEVEL,		parse_level },
 		{ CPERF_WINDOW_SIZE,	parse_window_sz },
@@ -649,6 +684,7 @@ comp_perf_options_default(struct comp_test_data *test_data)
 	test_data->num_iter = 10000;
 	test_data->huffman_enc = RTE_COMP_HUFFMAN_DYNAMIC;
 	test_data->test_op = COMPRESS_DECOMPRESS;
+	test_data->test_algo = RTE_COMP_ALGO_DEFLATE;
 	test_data->window_sz = -1;
 	test_data->level_lst.min = RTE_COMP_LEVEL_MIN;
 	test_data->level_lst.max = RTE_COMP_LEVEL_MAX;
app/test-compress-perf/comp_perf_test_cyclecount.c +2 -2
index ce6c4d7605..81c3d30038 100644
@@ -193,7 +193,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
@@ -208,7 +208,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test-compress-perf/comp_perf_test_throughput.c +2 -2
index c9f8237626..2545ee9925 100644
@@ -84,7 +84,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
@@ -99,7 +99,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test-compress-perf/comp_perf_test_verify.c +2 -2
index 7d6b6abecd..88f4f41851 100644
@@ -87,7 +87,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
@@ -104,7 +104,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test-compress-perf/main.c +24 -14
index bbb4c7917b..d049527ba1 100644
@@ -57,29 +57,39 @@ comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id)
 {
 	const struct rte_compressdev_capabilities *cap;
 
-	cap = rte_compressdev_capability_get(cdev_id,
-					     RTE_COMP_ALGO_DEFLATE);
+	cap = rte_compressdev_capability_get(cdev_id, test_data->test_algo);
 
 	if (cap == NULL) {
 		RTE_LOG(ERR, USER1,
-			"Compress device does not support DEFLATE\n");
+			"Compress device does not support %u algorithm\n",
+			test_data->test_algo);
 		return -1;
 	}
 
 	uint64_t comp_flags = cap->comp_feature_flags;
 
-	/* Huffman encoding */
-	if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED &&
-			(comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0) {
-		RTE_LOG(ERR, USER1,
-			"Compress device does not supported Fixed Huffman\n");
-		return -1;
-	}
+	/* Algorithm type */
+	switch (test_data->test_algo) {
+	case RTE_COMP_ALGO_DEFLATE:
+		/* Huffman encoding */
+		if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED &&
+		    (comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not supported Fixed Huffman\n");
+			return -1;
+		}
 
-	if (test_data->huffman_enc == RTE_COMP_HUFFMAN_DYNAMIC &&
-			(comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) {
-		RTE_LOG(ERR, USER1,
-			"Compress device does not supported Dynamic Huffman\n");
+		if (test_data->huffman_enc == RTE_COMP_HUFFMAN_DYNAMIC &&
+		    (comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not supported Dynamic Huffman\n");
+			return -1;
+		}
+		break;
+	case RTE_COMP_ALGO_LZS:
+	case RTE_COMP_ALGO_NULL:
+		break;
+	default:
 		return -1;
 	}
 
doc/guides/rel_notes/release_23_03.rst +4 -0
index b2a8c921b9..6aa3f31246 100644
@@ -116,6 +116,10 @@ New Features
 
   Enable the application options for testing only compress and only decompress.
 
+* **Added algo option in test-compress-perf.**
+
+  Added support for testing other algorithms except for DEFLAT.
+
 
 Removed Items
 -------------
doc/guides/tools/comp_perf.rst +2 -0
index 9d2f4dbe4a..cf9035cc23 100644
@@ -84,6 +84,8 @@ Application Options
 
  ``--operation [comp/decomp/comp_and_decomp]``: perform test on compression, decompression or both operations
 
+ ``--algo [null/deflate/lzs]`` : perform test on algorithm null(DMA), Deflate or lzs (default: Deflate)
+
  ``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default: dynamic)
 
  ``--compress-level N``: compression level, which could be a single value, list or range (default: range between 1 and 9)
Signature · 1 line
2.25.1
#20 lore raw

RE: [EXT] [PATCH v3 3/4] app/test-compress-perf: add algo option

Quoted 51 lines

Add a command line option algo to select the compress algorithm supported by the compress API: null (DMA), deflate and lzs.

Default for deflate.

---
 app/test-compress-perf/comp_perf_options.h    |  2 +
 .../comp_perf_options_parse.c                 | 40 ++++++++++++++++++-
 .../comp_perf_test_cyclecount.c               |  4 +-
 .../comp_perf_test_throughput.c               |  4 +-
 .../comp_perf_test_verify.c                   |  4 +-
 app/test-compress-perf/main.c                 | 38 +++++++++++-------
 doc/guides/rel_notes/release_23_03.rst        |  4 ++
 doc/guides/tools/comp_perf.rst                |  2 +
 8 files changed, 76 insertions(+), 22 deletions(-)
app/test-compress- renamed from app/test-compress-perf/comp_perf_options.h +0 -0

perf/comp_perf_options.h
index d00b299247..5e5227a700 100644

app/test-compress-perf/comp_perf_options.h +2 -0
@@ -63,6 +63,8 @@ struct comp_test_data {
 
 	enum rte_comp_huffman huffman_enc;
 	enum comp_operation test_op;
+	enum rte_comp_algorithm test_algo;
+
 	int window_sz;
 	struct range_list level_lst;
 	uint8_t level;
app/test- renamed from app/test-compress-perf/comp_perf_options_parse.c +0 -0

compress-perf/comp_perf_options_parse.c
index 7a992bf43e..97ddff87ef 100644

app/test-compress-perf/comp_perf_options_parse.c +1 -0
@@ -25,6 +25,7 @@
 #define CPERF_MAX_SGL_SEGS	("max-num-sgl-segs")
 #define CPERF_NUM_ITER		("num-iter")
 #define CPERF_OPTYPE		("operation")
+#define CPERF_ALGO		("algo")
 #define CPERF_HUFFMAN_ENC	("huffman-enc")
 #define CPERF_LEVEL		("compress-level")
 #define CPERF_WINDOW_SIZE	("window-sz")
@@ -56,6 +57,8 @@ usage(char *progname)
 		"		compressed/decompressed (default: 10000)\n"
 		" --operation [comp/decomp/comp_and_decomp]: perform

test on\n"
" compression, decompression or both
operations\n"

app/test-compress-perf/comp_perf_options_parse.c usage(char *progname) · line 61

Quoted 2 lines
app/test-compress-perf/comp_perf_options_parse.c +2 -0
+		" --algo [null/deflate/lzs]: perform test on algorithm\n"
+		"		null(DMA), deflate or lzs (default: deflate)\n"

Since LZ4 is also supported now, is it not good to add that also in this patch?

Quoted 43 lines
app/test-compress-perf/comp_perf_options_parse.c +0 -0
 		" --huffman-enc [fixed/dynamic/default]: Huffman encoding\n"
 		"		(default: dynamic)\n"
 		" --compress-level N: compression level, which could be a single

value, list or range\n"

app/test-compress-perf/comp_perf_options_parse.c +30 -0
@@ -470,6 +473,36 @@ parse_op_type(struct comp_test_data *test_data,
 const char *arg)
 	return 0;
 }
 
+static int
+parse_algo(struct comp_test_data *test_data, const char *arg)
+{
+	struct name_id_map algo_namemap[] = {
+		{
+			"null",
+			RTE_COMP_ALGO_NULL
+		},
+		{
+			"deflate",
+			RTE_COMP_ALGO_DEFLATE
+		},
+		{
+			"lzs",
+			RTE_COMP_ALGO_LZS
+		}
+	};
+
+	int id = get_str_key_id_mapping(algo_namemap,
+			RTE_DIM(algo_namemap), arg);
+	if (id < 0) {
+		RTE_LOG(ERR, USER1, "Invalid algorithm specified\n");
+		return -1;
+	}
+
+	test_data->test_algo = (enum rte_comp_algorithm)id;
+
+	return 0;
+}
+
 static int
 parse_huffman_enc(struct comp_test_data *test_data, const char *arg)

{

app/test-compress-perf/comp_perf_options_parse.c parse_huffman_enc(struct comp_test_data *test_data, · line 529

Quoted 7 lines
app/test-compress-perf/comp_perf_options_parse.c +1 -1
@@ -491,7 +524,7 @@ parse_huffman_enc(struct comp_test_data *test_data,
 const char *arg)
 	int id = get_str_key_id_mapping(huffman_namemap,
 			RTE_DIM(huffman_namemap), arg);
 	if (id < 0) {
-		RTE_LOG(ERR, USER1, "Invalid Huffmane encoding
 specified\n");
+		RTE_LOG(ERR, USER1, "Invalid Huffman encoding specified\n");

This change is not related to this patch. Should be a separate patch and should be backported.

app/test-compress-perf/comp_perf_options_parse.c parse_level(struct comp_test_data *test_data, const · line 544

Quoted 10 lines
app/test-compress-perf/comp_perf_options_parse.c +0 -0
 		return -1;
app/test-compress-perf/comp_perf_options_parse.c +1 -1
 	}
@@ -507,7 +540,7 @@ parse_level(struct comp_test_data *test_data, const
 char *arg)
 
 	/*
 	 * Try parsing the argument as a range, if it fails,
-	 * arse it as a list
+	 * parse it as a list

Same here.

Quoted 177 lines
app/test-compress-perf/comp_perf_options_parse.c +0 -0
 	 */
 	if (parse_range(arg, &test_data->level_lst.min,
app/test-compress-perf/comp_perf_options_parse.c +2 -0
 			&test_data->level_lst.max,
@@ -572,6 +605,7 @@ static struct option lgopts[] = {
 	{ CPERF_MAX_SGL_SEGS, required_argument, 0, 0},
 	{ CPERF_NUM_ITER, required_argument, 0, 0 },
 	{ CPERF_OPTYPE,	required_argument, 0, 0 },
+	{ CPERF_ALGO, required_argument, 0, 0 },
 	{ CPERF_HUFFMAN_ENC, required_argument, 0, 0 },
 	{ CPERF_LEVEL, required_argument, 0, 0 },
 	{ CPERF_WINDOW_SIZE, required_argument, 0, 0 },
@@ -594,6 +628,7 @@ comp_perf_opts_parse_long(int opt_idx, struct
 comp_test_data *test_data)
 		{ CPERF_MAX_SGL_SEGS,	parse_max_num_sgl_segs },
 		{ CPERF_NUM_ITER,	parse_num_iter },
 		{ CPERF_OPTYPE,		parse_op_type },
+		{ CPERF_ALGO,		parse_algo },
 		{ CPERF_HUFFMAN_ENC,	parse_huffman_enc },
 		{ CPERF_LEVEL,		parse_level },
app/test-compress-perf/comp_perf_options_parse.c +1 -0
 		{ CPERF_WINDOW_SIZE,	parse_window_sz },
@@ -649,6 +684,7 @@ comp_perf_options_default(struct comp_test_data
 *test_data)
 	test_data->num_iter = 10000;
 	test_data->huffman_enc = RTE_COMP_HUFFMAN_DYNAMIC;
 	test_data->test_op = COMPRESS_DECOMPRESS;
+	test_data->test_algo = RTE_COMP_ALGO_DEFLATE;
 	test_data->window_sz = -1;
 	test_data->level_lst.min = RTE_COMP_LEVEL_MIN;
app/test-compress-perf/comp_perf_options_parse.c +0 -0
 	test_data->level_lst.max = RTE_COMP_LEVEL_MAX;
app/test- renamed from app/test-compress-perf/comp_perf_test_cyclecount.c +0 -0

compress-perf/comp_perf_test_cyclecount.c
index ce6c4d7605..81c3d30038 100644

app/test-compress-perf/comp_perf_test_cyclecount.c +1 -1
@@ -193,7 +193,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum
 rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
app/test-compress-perf/comp_perf_test_cyclecount.c +1 -1
 				.window_size = test_data->window_sz,
@@ -208,7 +208,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum
 rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
app/test-compress-perf/comp_perf_test_cyclecount.c +0 -0
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test- renamed from app/test-compress-perf/comp_perf_test_throughput.c +0 -0

compress-perf/comp_perf_test_throughput.c
index c9f8237626..2545ee9925 100644

app/test-compress-perf/comp_perf_test_throughput.c +1 -1
@@ -84,7 +84,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum
 rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
app/test-compress-perf/comp_perf_test_throughput.c +1 -1
 				.window_size = test_data->window_sz,
@@ -99,7 +99,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum
 rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
app/test-compress-perf/comp_perf_test_throughput.c +0 -0
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test- renamed from app/test-compress-perf/comp_perf_test_verify.c +0 -0

compress-perf/comp_perf_test_verify.c
index 7d6b6abecd..88f4f41851 100644

app/test-compress-perf/comp_perf_test_verify.c +1 -1
@@ -87,7 +87,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum
 rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
app/test-compress-perf/comp_perf_test_verify.c +1 -1
 				.window_size = test_data->window_sz,
@@ -104,7 +104,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum
 rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
app/test-compress-perf/comp_perf_test_verify.c +0 -0
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
app/test-compress-perf/main.c +8 -10
index bbb4c7917b..d049527ba1 100644
@@ -57,29 +57,39 @@ comp_perf_check_capabilities(struct comp_test_data
 *test_data, uint8_t cdev_id)
 {
 	const struct rte_compressdev_capabilities *cap;
 
-	cap = rte_compressdev_capability_get(cdev_id,
-					     RTE_COMP_ALGO_DEFLATE);
+	cap = rte_compressdev_capability_get(cdev_id, test_data->test_algo);
 
 	if (cap == NULL) {
 		RTE_LOG(ERR, USER1,
-			"Compress device does not support DEFLATE\n");
+			"Compress device does not support %u algorithm\n",
+			test_data->test_algo);
 		return -1;
 	}
 
 	uint64_t comp_flags = cap->comp_feature_flags;
 
-	/* Huffman encoding */
-	if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED &&
-			(comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0)
 {
-		RTE_LOG(ERR, USER1,
-			"Compress device does not supported Fixed
 Huffman\n");
-		return -1;
-	}
+	/* Algorithm type */
+	switch (test_data->test_algo) {
+	case RTE_COMP_ALGO_DEFLATE:
+		/* Huffman encoding */
+		if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED

&&

app/test-compress-perf/main.c +3 -0
+		    (comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not supported Fixed

Huffman\n");

app/test-compress-perf/main.c +2 -2
+			return -1;
+		}
 
-	if (test_data->huffman_enc == RTE_COMP_HUFFMAN_DYNAMIC &&
-			(comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC)

== 0) {

app/test-compress-perf/main.c +0 -2
-		RTE_LOG(ERR, USER1,
-			"Compress device does not supported Dynamic

Huffman\n");

app/test-compress-perf/main.c +1 -0
+		if (test_data->huffman_enc ==

RTE_COMP_HUFFMAN_DYNAMIC &&

app/test-compress-perf/main.c +3 -0
+		    (comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not supported Dynamic

Huffman\n");

app/test-compress-perf/main.c +7 -0
+			return -1;
+		}
+		break;
+	case RTE_COMP_ALGO_LZS:
+	case RTE_COMP_ALGO_NULL:
+		break;
+	default:
 		return -1;
 	}
doc/guides/rel_notes/release_23_03.rst +0 -0
index b2a8c921b9..6aa3f31246 100644

doc/guides/rel_notes/release_23_03.rst New Features · line 122

Quoted 7 lines
doc/guides/rel_notes/release_23_03.rst +4 -0
@@ -116,6 +116,10 @@ New Features
 
   Enable the application options for testing only compress and only decompress.
 
+* **Added algo option in test-compress-perf.**
+
+  Added support for testing other algorithms except for DEFLAT.
+

This patch is only adding a new option to the test app. I believe release notes are not needed.

Quoted 22 lines
doc/guides/rel_notes/release_23_03.rst +0 -0
 Removed Items
 -------------
doc/guides/tools/comp_perf.rst +0 -0
index 9d2f4dbe4a..cf9035cc23 100644
@@ -84,6 +84,8 @@ Application Options
 
  ``--operation [comp/decomp/comp_and_decomp]``: perform test on

compression, decompression or both operations

+ ``--algo [null/deflate/lzs]`` : perform test on algorithm null(DMA), Deflate or lzs (default: Deflate)
+
``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default:
dynamic)

``--compress-level N``: compression level, which could be a single value, list or
range (default: range between 1 and 9)
--
2.25.1

#21 lore raw

RE: [EXT] [PATCH v3 3/4] app/test-compress-perf: add algo option

Quoted 48 lines · Akhil Goyal
Quoted 48 lines

Add a command line option algo to select the compress algorithm supported by the compress API: null (DMA), deflate and lzs.

Default for deflate.

---
 app/test-compress-perf/comp_perf_options.h    |  2 +
 .../comp_perf_options_parse.c                 | 40 ++++++++++++++++++-
 .../comp_perf_test_cyclecount.c               |  4 +-
 .../comp_perf_test_throughput.c               |  4 +-
 .../comp_perf_test_verify.c                   |  4 +-
 app/test-compress-perf/main.c                 | 38 +++++++++++-------
 doc/guides/rel_notes/release_23_03.rst        |  4 ++
 doc/guides/tools/comp_perf.rst                |  2 +
 8 files changed, 76 insertions(+), 22 deletions(-)
(quoted diff) +0 -0

b/app/test-compress- perf/comp_perf_options.h index d00b299247..5e5227a700 100644

app/test-compress-perf/comp_perf_options.h +2 -0
@@ -63,6 +63,8 @@ struct comp_test_data {
 
      enum rte_comp_huffman huffman_enc;
      enum comp_operation test_op;
+     enum rte_comp_algorithm test_algo;
+
      int window_sz;
      struct range_list level_lst;
      uint8_t level;
(quoted diff) +0 -0

b/app/test- compress-perf/comp_perf_options_parse.c index 7a992bf43e..97ddff87ef 100644

app/test-compress-perf/comp_perf_options_parse.c +1 -0
@@ -25,6 +25,7 @@
 #define CPERF_MAX_SGL_SEGS   ("max-num-sgl-segs")
 #define CPERF_NUM_ITER               ("num-iter")
 #define CPERF_OPTYPE         ("operation")
+#define CPERF_ALGO           ("algo")
 #define CPERF_HUFFMAN_ENC    ("huffman-enc")
 #define CPERF_LEVEL          ("compress-level")
 #define CPERF_WINDOW_SIZE    ("window-sz")
@@ -56,6 +57,8 @@ usage(char *progname)
              "               compressed/decompressed (default: 10000)\n"
Quoted 8 lines

app/test-compress-perf/comp_perf_options_parse.c usage(char *progname) · line 61

Quoted 6 lines
app/test-compress-perf/comp_perf_options_parse.c +0 -0
              " --operation [comp/decomp/comp_and_decomp]: perform

test on\n"
" compression, decompression or both
operations\n"

app/test-compress-perf/comp_perf_options_parse.c +2 -0
+             " --algo [null/deflate/lzs]: perform test on algorithm\n"
+             "               null(DMA), deflate or lzs (default: deflate)\n"

Since LZ4 is also supported now, is it not good to add that also in this patch?

I agree, I'll merge them in the next version.

Quoted 45 lines
Quoted 45 lines

" --huffman-enc [fixed/dynamic/default]: Huffman encoding\n"
" (default: dynamic)\n"
" --compress-level N: compression level, which could be
a single value, list or range\n"

app/test-compress-perf/comp_perf_options_parse.c +29 -0
@@ -470,6 +473,36 @@ parse_op_type(struct comp_test_data *test_data,
 const char *arg)
      return 0;
 }
 
+static int
+parse_algo(struct comp_test_data *test_data, const char *arg) {
+     struct name_id_map algo_namemap[] = {
+             {
+                     "null",
+                     RTE_COMP_ALGO_NULL
+             },
+             {
+                     "deflate",
+                     RTE_COMP_ALGO_DEFLATE
+             },
+             {
+                     "lzs",
+                     RTE_COMP_ALGO_LZS
+             }
+     };
+
+     int id = get_str_key_id_mapping(algo_namemap,
+                     RTE_DIM(algo_namemap), arg);
+     if (id < 0) {
+             RTE_LOG(ERR, USER1, "Invalid algorithm specified\n");
+             return -1;
+     }
+
+     test_data->test_algo = (enum rte_comp_algorithm)id;
+
+     return 0;
+}
+
 static int
 parse_huffman_enc(struct comp_test_data *test_data, const char *arg)

{ @@ -491,7 +524,7 @@ parse_huffman_enc(struct comp_test_data *test_data, const char *arg)
int id = get_str_key_id_mapping(huffman_namemap,
RTE_DIM(huffman_namemap), arg);

Quoted 8 lines
Quoted 5 lines
      if (id < 0) {

- RTE_LOG(ERR, USER1, "Invalid Huffmane encoding
specified\n");
+ RTE_LOG(ERR, USER1, "Invalid Huffman encoding
+ specified\n");

This change is not related to this patch. Should be a separate patch and should be backported.

I'll separate it and the next to different commit.

Quoted 197 lines

app/test-compress-perf/comp_perf_options_parse.c parse_level(struct comp_test_data *test_data, · line 544

Quoted 11 lines
              return -1;
      }
app/test-compress-perf/comp_perf_options_parse.c +1 -1
@@ -507,7 +540,7 @@ parse_level(struct comp_test_data *test_data,
 const char *arg)
 
      /*
       * Try parsing the argument as a range, if it fails,
-      * arse it as a list
+      * parse it as a list

Same here.

Quoted 179 lines
app/test-compress-perf/comp_perf_options_parse.c +0 -0
       */
      if (parse_range(arg, &test_data->level_lst.min,
                      &test_data->level_lst.max, @@ -572,6 +605,7 @@

static struct option lgopts[] = {
{ CPERF_MAX_SGL_SEGS, required_argument, 0, 0},
{ CPERF_NUM_ITER, required_argument, 0, 0 },
{ CPERF_OPTYPE, required_argument, 0, 0 },
+ { CPERF_ALGO, required_argument, 0, 0 },
{ CPERF_HUFFMAN_ENC, required_argument, 0, 0 },
{ CPERF_LEVEL, required_argument, 0, 0 },
{ CPERF_WINDOW_SIZE, required_argument, 0, 0 }, @@ -594,6 +628,7
@@ comp_perf_opts_parse_long(int opt_idx, struct comp_test_data *test_data)

              { CPERF_MAX_SGL_SEGS,   parse_max_num_sgl_segs },
              { CPERF_NUM_ITER,       parse_num_iter },
              { CPERF_OPTYPE,         parse_op_type },
app/test-compress-perf/comp_perf_options_parse.c +2 -0
+             { CPERF_ALGO,           parse_algo },
              { CPERF_HUFFMAN_ENC,    parse_huffman_enc },
              { CPERF_LEVEL,          parse_level },
              { CPERF_WINDOW_SIZE,    parse_window_sz },
@@ -649,6 +684,7 @@ comp_perf_options_default(struct comp_test_data
 *test_data)
      test_data->num_iter = 10000;
      test_data->huffman_enc = RTE_COMP_HUFFMAN_DYNAMIC;
      test_data->test_op = COMPRESS_DECOMPRESS;
+     test_data->test_algo = RTE_COMP_ALGO_DEFLATE;
      test_data->window_sz = -1;
      test_data->level_lst.min = RTE_COMP_LEVEL_MIN;
      test_data->level_lst.max = RTE_COMP_LEVEL_MAX; diff --git

a/app/test-compress-perf/comp_perf_test_cyclecount.c b/app/test- compress-perf/comp_perf_test_cyclecount.c
index ce6c4d7605..81c3d30038 100644

app/test-compress-perf/comp_perf_test_cyclecount.c +1 -1
@@ -193,7 +193,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum
 rte_comp_xform_type type)
              xform = (struct rte_comp_xform) {
                      .type = RTE_COMP_COMPRESS,
                      .compress = {
-                             .algo = RTE_COMP_ALGO_DEFLATE,
+                             .algo = test_data->test_algo,
                              .deflate.huffman = test_data->huffman_enc,
                              .level = test_data->level,
                              .window_size = test_data->window_sz, @@
-208,7 +208,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum
rte_comp_xform_type type)
              xform = (struct rte_comp_xform) {
                      .type = RTE_COMP_DECOMPRESS,
                      .decompress = {
app/test-compress-perf/comp_perf_test_cyclecount.c +1 -1
-                             .algo = RTE_COMP_ALGO_DEFLATE,
+                             .algo = test_data->test_algo,
                              .chksum = RTE_COMP_CHECKSUM_NONE,
                              .window_size = test_data->window_sz,
                              .hash_algo = RTE_COMP_HASH_ALGO_NONE
(quoted diff) +0 -0

b/app/test- compress-perf/comp_perf_test_throughput.c index c9f8237626..2545ee9925 100644

app/test-compress-perf/comp_perf_test_throughput.c +1 -1
@@ -84,7 +84,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum
 rte_comp_xform_type type)
              xform = (struct rte_comp_xform) {
                      .type = RTE_COMP_COMPRESS,
                      .compress = {
-                             .algo = RTE_COMP_ALGO_DEFLATE,
+                             .algo = test_data->test_algo,
                              .deflate.huffman = test_data->huffman_enc,
                              .level = test_data->level,
                              .window_size = test_data->window_sz, @@
-99,7 +99,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum
rte_comp_xform_type type)
              xform = (struct rte_comp_xform) {
                      .type = RTE_COMP_DECOMPRESS,
                      .decompress = {
app/test-compress-perf/comp_perf_test_throughput.c +1 -1
-                             .algo = RTE_COMP_ALGO_DEFLATE,
+                             .algo = test_data->test_algo,
                              .chksum = RTE_COMP_CHECKSUM_NONE,
                              .window_size = test_data->window_sz,
                              .hash_algo = RTE_COMP_HASH_ALGO_NONE
(quoted diff) +0 -0

b/app/test- compress-perf/comp_perf_test_verify.c
index 7d6b6abecd..88f4f41851 100644

app/test-compress-perf/comp_perf_test_verify.c +1 -1
@@ -87,7 +87,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum
 rte_comp_xform_type type)
              xform = (struct rte_comp_xform) {
                      .type = RTE_COMP_COMPRESS,
                      .compress = {
-                             .algo = RTE_COMP_ALGO_DEFLATE,
+                             .algo = test_data->test_algo,
                              .deflate.huffman = test_data->huffman_enc,
                              .level = test_data->level,
                              .window_size = test_data->window_sz, @@
-104,7 +104,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum
rte_comp_xform_type type)
              xform = (struct rte_comp_xform) {
                      .type = RTE_COMP_DECOMPRESS,
                      .decompress = {
app/test-compress-perf/comp_perf_test_verify.c +1 -1
-                             .algo = RTE_COMP_ALGO_DEFLATE,
+                             .algo = test_data->test_algo,
                              .chksum = RTE_COMP_CHECKSUM_NONE,
                              .window_size = test_data->window_sz,
                              .hash_algo = RTE_COMP_HASH_ALGO_NONE
(quoted diff) +0 -0

b/app/test-compress-perf/main.c index bbb4c7917b..d049527ba1 100644

app/test-compress-perf/main.c +9 -10
@@ -57,29 +57,39 @@ comp_perf_check_capabilities(struct comp_test_data
 *test_data, uint8_t cdev_id)  {
      const struct rte_compressdev_capabilities *cap;
 
-     cap = rte_compressdev_capability_get(cdev_id,
-                                          RTE_COMP_ALGO_DEFLATE);
+     cap = rte_compressdev_capability_get(cdev_id,
+ test_data->test_algo);
 
      if (cap == NULL) {
              RTE_LOG(ERR, USER1,
-                     "Compress device does not support DEFLATE\n");
+                     "Compress device does not support %u algorithm\n",
+                     test_data->test_algo);
              return -1;
      }
 
      uint64_t comp_flags = cap->comp_feature_flags;
 
-     /* Huffman encoding */
-     if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED &&
-                     (comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0)
 {
-             RTE_LOG(ERR, USER1,
-                     "Compress device does not supported Fixed
 Huffman\n");
-             return -1;
-     }
+     /* Algorithm type */
+     switch (test_data->test_algo) {
+     case RTE_COMP_ALGO_DEFLATE:
+             /* Huffman encoding */
+             if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED

&&

app/test-compress-perf/main.c +4 -0
+                 (comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0) {
+                     RTE_LOG(ERR, USER1,
+                             "Compress device does not supported
+ Fixed

Huffman\n");

app/test-compress-perf/main.c +2 -2
+                     return -1;
+             }
 
-     if (test_data->huffman_enc == RTE_COMP_HUFFMAN_DYNAMIC &&
-                     (comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC)

== 0) {

app/test-compress-perf/main.c +0 -2
-             RTE_LOG(ERR, USER1,
-                     "Compress device does not supported Dynamic

Huffman\n");
+ if (test_data->huffman_enc ==
RTE_COMP_HUFFMAN_DYNAMIC &&

app/test-compress-perf/main.c +4 -0
+                 (comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) {
+                     RTE_LOG(ERR, USER1,
+                             "Compress device does not supported
+ Dynamic

Huffman\n");

app/test-compress-perf/main.c +7 -0
+                     return -1;
+             }
+             break;
+     case RTE_COMP_ALGO_LZS:
+     case RTE_COMP_ALGO_NULL:
+             break;
+     default:
              return -1;
      }
doc/guides/rel_notes/release_23_03.rst +0 -0
index b2a8c921b9..6aa3f31246 100644

doc/guides/rel_notes/release_23_03.rst New Features · line 117

Quoted 2 lines
doc/guides/rel_notes/release_23_03.rst +0 -0
@@ -116,6 +116,10 @@ New Features
 
   Enable the application options for testing only compress and only

decompress.

Quoted 1 line
Quoted 7 lines

doc/guides/rel_notes/release_23_03.rst New Features · line 122

Quoted 4 lines
doc/guides/rel_notes/release_23_03.rst +4 -0
+* **Added algo option in test-compress-perf.**
+
+  Added support for testing other algorithms except for DEFLAT.
+

This patch is only adding a new option to the test app. I believe release notes are not needed.

Thank you, I'm removing it.

Quoted 25 lines
Quoted 25 lines

Removed Items
-------------

(quoted diff) +0 -0

b/doc/guides/tools/comp_perf.rst index 9d2f4dbe4a..cf9035cc23 100644

doc/guides/tools/comp_perf.rst +0 -0
@@ -84,6 +84,8 @@ Application Options
 
  ``--operation [comp/decomp/comp_and_decomp]``: perform test on

compression, decompression or both operations

+ ``--algo [null/deflate/lzs]`` : perform test on algorithm null(DMA),
+ Deflate or lzs
(default: Deflate)
+
``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default:
dynamic)

``--compress-level N``: compression level, which could be a single
value, list or range (default: range between 1 and 9)
--
2.25.1

#22 lore raw

PATCH v3 4/4 app/test-compress-perf: add LZ4 support

In reply to Michael Baum

Add support for LZ4 algorithm and add a new parameter for it.

---
 app/test-compress-perf/comp_perf_options.h    |  1 +
 .../comp_perf_options_parse.c                 | 43 ++++++++++++++++++-
 .../comp_perf_test_cyclecount.c               |  7 ++-
 .../comp_perf_test_throughput.c               |  7 ++-
 .../comp_perf_test_verify.c                   |  7 ++-
 app/test-compress-perf/main.c                 | 17 ++++++++
 doc/guides/rel_notes/release_23_03.rst        |  1 +
 doc/guides/tools/comp_perf.rst                |  5 ++-
 8 files changed, 82 insertions(+), 6 deletions(-)
app/test-compress-perf/comp_perf_options.h +1 -0
index 5e5227a700..828a7309d8 100644
@@ -61,6 +61,7 @@ struct comp_test_data {
 	uint16_t max_sgl_segs;
 	uint32_t total_segs;
 
+	uint8_t lz4_flags;
 	enum rte_comp_huffman huffman_enc;
 	enum comp_operation test_op;
 	enum rte_comp_algorithm test_algo;
app/test-compress-perf/comp_perf_options_parse.c +41 -2

93 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_cyclecount.c +6 -1
index 81c3d30038..4d336ec8d6 100644
@@ -194,13 +194,16 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
 				.algo = test_data->test_algo,
-				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->decomp_bufs;
 		output_bufs = mem->comp_bufs;
 		out_seg_sz = test_data->out_seg_sz;
@@ -214,6 +217,8 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->comp_bufs;
 		output_bufs = mem->decomp_bufs;
 		out_seg_sz = test_data->seg_sz;
app/test-compress-perf/comp_perf_test_throughput.c +6 -1
index 2545ee9925..1f7072d223 100644
@@ -85,13 +85,16 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
 				.algo = test_data->test_algo,
-				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->decomp_bufs;
 		output_bufs = mem->comp_bufs;
 		out_seg_sz = test_data->out_seg_sz;
@@ -105,6 +108,8 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->comp_bufs;
 		output_bufs = mem->decomp_bufs;
 		out_seg_sz = test_data->seg_sz;
app/test-compress-perf/comp_perf_test_verify.c +6 -1
index 88f4f41851..7bd18073cf 100644
@@ -88,13 +88,16 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
 				.algo = test_data->test_algo,
-				.deflate.huffman = test_data->huffman_enc,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		output_data_ptr = ctx->mem.compressed_data;
 		output_data_sz = &ctx->comp_data_sz;
 		input_bufs = mem->decomp_bufs;
@@ -110,6 +113,8 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		output_data_ptr = ctx->mem.decompressed_data;
 		output_data_sz = &ctx->decomp_data_sz;
 		input_bufs = mem->comp_bufs;
app/test-compress-perf/main.c +17 -0
index d049527ba1..fa366123ed 100644
@@ -86,6 +86,23 @@ comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id)
 			return -1;
 		}
 		break;
+	case RTE_COMP_ALGO_LZ4:
+		/* LZ4 flags */
+		if ((test_data->lz4_flags & RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM) &&
+		    (comp_flags & RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not support LZ4 block with checksum\n");
+			return -1;
+		}
+
+		if ((test_data->lz4_flags &
+		     RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE) &&
+		    (comp_flags & RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE) == 0) {
+			RTE_LOG(ERR, USER1,
+				"Compress device does not support LZ4 independent blocks\n");
+			return -1;
+		}
+		break;
 	case RTE_COMP_ALGO_LZS:
 	case RTE_COMP_ALGO_NULL:
 		break;
doc/guides/rel_notes/release_23_03.rst +1 -0
index 6aa3f31246..c05fa2930b 100644
@@ -111,6 +111,7 @@ New Features
 
   * Added support for ``RTE_COMP_ALGO_LZ4``.
   * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
+  * Added support for ``lz4`` in test-compress-perf algo options.
 
 * **Allowed test single compress operation in test-compress-perf.**
 
doc/guides/tools/comp_perf.rst +4 -1
index cf9035cc23..5eb05a095c 100644
@@ -84,10 +84,13 @@ Application Options
 
  ``--operation [comp/decomp/comp_and_decomp]``: perform test on compression, decompression or both operations
 
- ``--algo [null/deflate/lzs]`` : perform test on algorithm null(DMA), Deflate or lzs (default: Deflate)
+ ``--algo [null/deflate/lzs/lz4]`` : perform test on algorithm null(DMA), Deflate, lzs or lz4 (default: Deflate)
 
  ``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default: dynamic)
 
+ ``--lz4-flags N``: flags to for LZ4 parameters,
+ see `LZ4 Frame Descriptor <https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#frame-descriptor>`_ (default: no flags)
+
  ``--compress-level N``: compression level, which could be a single value, list or range (default: range between 1 and 9)
 
  ``--window-sz N``: base two log value of compression window size (default: max supported by PMD)
Signature · 1 line
2.25.1
#23 lore raw
In reply to Michael Baum

Hi,

The patchset only have library change and added performance tests.

We shall have at least one PMD to support the added algorithms and unit test updated for the PMD/algorithms.

Regards,

Fan

Quoted 37 lines · Michael Baum

Add support for LZ4 compression algorithm for both API and app.

In addition, enable app to test de/comp only. This option already exists in both app doc and arguments but isn't implemented inside the application.

LZ4: https://github.com/lz4/lz4
RFC:
https://patchwork.dpdk.org/project/dpdk/patch/[email protected]/

v2:
- Add macros for all LZ4 flags in the standard.
- Rebase.

v3:
- Rebase.

Michael Baum (4):
compressdev: add LZ4 algorithm support app/test-compress-perf: allow test single compress operation app/test-compress-perf: add algo option app/test-compress-perf: add LZ4 support

  app/test-compress-perf/comp_perf_options.h    |   9 +-
  .../comp_perf_options_parse.c                 |  83 +++++++++++-
  .../comp_perf_test_common.c                   | 126 +++++++++++++-----
  .../comp_perf_test_cyclecount.c               |  82 +++++++-----
  .../comp_perf_test_throughput.c               |  75 +++++++----
  .../comp_perf_test_verify.c                   |  75 +++++++----
  app/test-compress-perf/main.c                 |  63 +++++++--
  doc/guides/compressdevs/features/default.ini  |   7 +
  doc/guides/rel_notes/release_23_03.rst        |  16 +++
  doc/guides/tools/comp_perf.rst                |   5 +
  lib/compressdev/rte_comp.c                    |  12 ++
  lib/compressdev/rte_comp.h                    |  79 ++++++++++-
  12 files changed, 488 insertions(+), 144 deletions(-)
#24 lore raw
Quoted 6 lines · Fan

Hi,

The patchset only have library change and added performance tests.

We shall have at least one PMD to support the added algorithms and unit test updated for the PMD/algorithms.

There is another patchset which adds it in MLX5 compress PMD:
series-26766 ("compress/mlx5: add LZ4 support") https://patchwork.dpdk.org/project/dpdk/cover/[email protected]/

Quoted 44 lines

Regards,

Fan

Quoted 38 lines · Michael Baum

Add support for LZ4 compression algorithm for both API and app.

In addition, enable app to test de/comp only. This option already exists in both app doc and arguments but isn't implemented inside the application.

LZ4: https://github.com/lz4/lz4
RFC:
https://patchwork.dpdk.org/project/dpdk/patch/20220410182622.8828-1-rz
[email protected]/

v2:
- Add macros for all LZ4 flags in the standard.
- Rebase.

v3:
- Rebase.

Michael Baum (4):
compressdev: add LZ4 algorithm support app/test-compress-perf: allow test single compress operation app/test-compress-perf: add algo option app/test-compress-perf: add LZ4 support

  app/test-compress-perf/comp_perf_options.h    |   9 +-
  .../comp_perf_options_parse.c                 |  83 +++++++++++-
  .../comp_perf_test_common.c                   | 126 +++++++++++++-----
  .../comp_perf_test_cyclecount.c               |  82 +++++++-----
  .../comp_perf_test_throughput.c               |  75 +++++++----
  .../comp_perf_test_verify.c                   |  75 +++++++----
  app/test-compress-perf/main.c                 |  63 +++++++--
  doc/guides/compressdevs/features/default.ini  |   7 +
  doc/guides/rel_notes/release_23_03.rst        |  16 +++
  doc/guides/tools/comp_perf.rst                |   5 +
  lib/compressdev/rte_comp.c                    |  12 ++
  lib/compressdev/rte_comp.h                    |  79 ++++++++++-
  12 files changed, 488 insertions(+), 144 deletions(-)
#25 lore raw

v4 cover compressdev: add LZ4 support

In reply to Michael Baum

Add support for LZ4 compression algorithm for both API and app.

In addition, enable app to test de/comp only. This option already exists in both app doc and arguments but isn't implemented inside the application.

LZ4: https://github.com/lz4/lz4
RFC:
https://patchwork.dpdk.org/project/dpdk/patch/[email protected]/

v2:
- Add macros for all LZ4 flags in the standard.
- Rebase.

v3:
- Rebase.

v4:
- Change "allow test single" commit to fix and add "Cc: [email protected]".
- Take all typo fixes into separate commit.
- Squash "add LZ4 algo" commit into "add algo option" commit.
- Remove unnecessary release notes updates.

Michael Baum (4):
compressdev: add LZ4 algorithm support app/test-compress-perf: fix some typos app/test-compress-perf: fix missing test single compress op app/test-compress-perf: add algo option

 app/test-compress-perf/comp_perf_options.h    |   9 +-
 .../comp_perf_options_parse.c                 |  83 +++++++++++-
 .../comp_perf_test_common.c                   | 126 +++++++++++++-----
 .../comp_perf_test_cyclecount.c               |  82 +++++++-----
 .../comp_perf_test_throughput.c               |  75 +++++++----
 .../comp_perf_test_verify.c                   |  75 +++++++----
 app/test-compress-perf/main.c                 |  63 +++++++--
 doc/guides/compressdevs/features/default.ini  |   7 +
 doc/guides/rel_notes/release_23_03.rst        |   7 +
 doc/guides/tools/comp_perf.rst                |   5 +
 lib/compressdev/rte_comp.c                    |  12 ++
 lib/compressdev/rte_comp.h                    |  79 ++++++++++-
 12 files changed, 479 insertions(+), 144 deletions(-)
Signature · 1 line
2.25.1
#26 lore raw

PATCH v4 1/4 compressdev: add LZ4 algorithm support

Add support for LZ4 algorithm:
- Add Lz4 param structure to XFORM structures.
- Add capabilities flags for LZ4 params.
- Add xxHash-32 checksum and capabilities flag.

---
 doc/guides/compressdevs/features/default.ini |  7 ++
 doc/guides/rel_notes/release_23_03.rst       |  7 ++
 lib/compressdev/rte_comp.c                   | 12 +++
 lib/compressdev/rte_comp.h                   | 79 +++++++++++++++++++-
 4 files changed, 103 insertions(+), 2 deletions(-)
doc/guides/compressdevs/features/default.ini +7 -0
index e1419ee8db..2f178c5efd 100644
@@ -20,8 +20,15 @@ OOP SGL In LB  Out     =
 OOP LB  In SGL Out     =
 Deflate                =
 LZS                    =
+LZ4                    =
 Adler32                =
 Crc32                  =
 Adler32&Crc32          =
+xxHash32               =
 Fixed                  =
 Dynamic                =
+LZ4 Dictionary ID      =
+LZ4 Content Checksum   =
+LZ4 Content Size       =
+LZ4 Block Checksum     =
+LZ4 Block Independence =
doc/guides/rel_notes/release_23_03.rst +7 -0
index 7527c6d57f..790c2a5eef 100644
@@ -105,6 +105,13 @@ New Features
   * Added support to capture packets at each graph node with packet metadata and
     node name.
 
+* **Added LZ4 algorithm in Compressdev Library.**
+
+  Added new compression algorithm, including:
+
+  * Added support for ``RTE_COMP_ALGO_LZ4``.
+  * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
+
 
 Removed Items
 -------------
lib/compressdev/rte_comp.c +12 -0
index 320c6dab92..f060c68557 100644
@@ -39,6 +39,18 @@ rte_comp_get_feature_name(uint64_t flag)
 		return "HUFFMAN_FIXED";
 	case RTE_COMP_FF_HUFFMAN_DYNAMIC:
 		return "HUFFMAN_DYNAMIC";
+	case RTE_COMP_FF_XXHASH32_CHECKSUM:
+		return "XXHASH32_CHECKSUM";
+	case RTE_COMP_FF_LZ4_DICT_ID:
+		return "LZ4_DICT_ID";
+	case RTE_COMP_FF_LZ4_CONTENT_WITH_CHECKSUM:
+		return "LZ4_CONTENT_WITH_CHECKSUM";
+	case RTE_COMP_FF_LZ4_CONTENT_SIZE:
+		return "LZ4_CONTENT_SIZE";
+	case RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE:
+		return "LZ4_BLOCK_INDEPENDENCE";
+	case RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM:
+		return "LZ4_BLOCK_WITH_CHECKSUM";
 	default:
 		return NULL;
 	}
lib/compressdev/rte_comp.h +77 -2

123 lines not on this page · raw message

Signature · 1 line
2.25.1
#27 lore raw

PATCH v4 2/4 app/test-compress-perf: fix some typos

In reply to Michael Baum

Fix some typos in comments and prints in this app.

Fixes e0b6287c035d ("app/compress-perf: add parser")
Fixes 2695db95a147 ("test/compress: add cycle-count mode to perf tool")
Fixes 971d89f5116b ("app/compress-perf: prevent output buffer overflow")
---
 app/test-compress-perf/comp_perf_options_parse.c   | 4 ++--
 app/test-compress-perf/comp_perf_test_cyclecount.c | 2 +-
 app/test-compress-perf/comp_perf_test_verify.c     | 4 ++--
 3 files changed, 5 insertions(+), 5 deletions(-)
app/test-compress-perf/comp_perf_options_parse.c +2 -2
index 019eddb7bd..9b9d4e6554 100644
@@ -491,7 +491,7 @@ parse_huffman_enc(struct comp_test_data *test_data, const char *arg)
 	int id = get_str_key_id_mapping(huffman_namemap,
 			RTE_DIM(huffman_namemap), arg);
 	if (id < 0) {
-		RTE_LOG(ERR, USER1, "Invalid Huffmane encoding specified\n");
+		RTE_LOG(ERR, USER1, "Invalid Huffman encoding specified\n");
 		return -1;
 	}
 
@@ -507,7 +507,7 @@ parse_level(struct comp_test_data *test_data, const char *arg)
 
 	/*
 	 * Try parsing the argument as a range, if it fails,
-	 * arse it as a list
+	 * parse it as a list
 	 */
 	if (parse_range(arg, &test_data->level_lst.min,
 			&test_data->level_lst.max,
app/test-compress-perf/comp_perf_test_cyclecount.c +1 -1
index 7473cb6277..ce457fefbb 100644
@@ -563,7 +563,7 @@ cperf_cyclecount_test_runner(void *test_ctx)
 		"    [D-e] - decompression enqueue\n"
 		"    [D-d] - decompression dequeue\n"
 		"  - Cycles section: number of cycles per 'op' for the following operations:\n"
-		"    setup/op - memory allocation, op configuration and memory dealocation\n"
+		"    setup/op - memory allocation, op configuration and memory deallocation\n"
 		"    [C-e] - compression enqueue\n"
 		"    [C-d] - compression dequeue\n"
 		"    [D-e] - decompression enqueue\n"
app/test-compress-perf/comp_perf_test_verify.c +2 -2
index 8964442891..6b61a9194f 100644
@@ -226,7 +226,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 				  op->status ==
 				  RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE) {
 					RTE_LOG(ERR, USER1,
-"Out of space error occurred due to uncompressible input data expanding to larger than destination buffer. Increase the EXPANSE_RATIO constant to use this data.\n");
+"Out of space error occurred due to incompressible input data expanding to larger than destination buffer. Increase the EXPANSE_RATIO constant to use this data.\n");
 					res = -1;
 					goto end;
 				} else if (op->status !=
@@ -311,7 +311,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 				  op->status ==
 				  RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE) {
 					RTE_LOG(ERR, USER1,
-"Out of space error occurred due to uncompressible input data expanding to larger than destination buffer. Increase the EXPANSE_RATIO constant to use this data.\n");
+"Out of space error occurred due to incompressible input data expanding to larger than destination buffer. Increase the EXPANSE_RATIO constant to use this data.\n");
 					res = -1;
 					goto end;
 				} else if (op->status !=
Signature · 1 line
2.25.1
#28 lore raw

PATCH v4 3/4 app/test-compress-perf: fix missing test single compress op

In reply to Michael Baum

Part of the application options is to test only compress and only decompress but actually the application ignores this user option and tries to test always both compress and decompress.

Allow testing only compress and only decompress.

Fixes e0b6287c035d ("app/compress-perf: add parser")
---
 app/test-compress-perf/comp_perf_options.h    |   6 +-
 .../comp_perf_options_parse.c                 |   4 +-
 .../comp_perf_test_common.c                   | 126 +++++++++++++-----
 .../comp_perf_test_cyclecount.c               |  69 ++++++----
 .../comp_perf_test_throughput.c               |  64 +++++----
 .../comp_perf_test_verify.c                   |  60 ++++++---
 app/test-compress-perf/main.c                 |   8 ++
 7 files changed, 223 insertions(+), 114 deletions(-)
app/test-compress-perf/comp_perf_options.h +3 -3
index 57dd146330..d00b299247 100644
@@ -32,9 +32,9 @@ enum cperf_test_type {
 };
 
 enum comp_operation {
-	COMPRESS_ONLY,
-	DECOMPRESS_ONLY,
-	COMPRESS_DECOMPRESS
+	COMPRESS = (1 << 0),
+	DECOMPRESS = (1 << 1),
+	COMPRESS_DECOMPRESS = (COMPRESS | DECOMPRESS),
 };
 
 struct range_list {
app/test-compress-perf/comp_perf_options_parse.c +2 -2
index 9b9d4e6554..303e714cda 100644
@@ -446,11 +446,11 @@ parse_op_type(struct comp_test_data *test_data, const char *arg)
 	struct name_id_map optype_namemap[] = {
 		{
 			"comp",
-			COMPRESS_ONLY
+			COMPRESS
 		},
 		{
 			"decomp",
-			DECOMPRESS_ONLY
+			DECOMPRESS
 		},
 		{
 			"comp_and_decomp",
app/test-compress-perf/comp_perf_test_common.c +91 -35

222 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_cyclecount.c +43 -26

81 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_throughput.c +38 -26

79 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_verify.c +38 -22

89 lines not on this page · raw message

app/test-compress-perf/main.c +8 -0
index 41b8edc2bd..bbb4c7917b 100644
@@ -254,6 +254,14 @@ comp_perf_dump_input_data(struct comp_test_data *test_data)
 		goto end;
 	}
 
+	if (!(test_data->test_op & COMPRESS) &&
+	    test_data->input_data_sz >
+	    (size_t) test_data->seg_sz * (size_t) test_data->max_sgl_segs) {
+		RTE_LOG(ERR, USER1,
+			"Size of input must be less than total segments\n");
+		goto end;
+	}
+
 	test_data->input_data = rte_zmalloc_socket(NULL,
 				test_data->input_data_sz, 0, rte_socket_id());
 
Signature · 1 line
2.25.1
#29 lore raw

PATCH v4 4/4 app/test-compress-perf: add algo option

In reply to Michael Baum

Add a command line option algo to select the compress algorithm supported by the compress API: null (DMA), deflate, lz4 and lzs.

Default for deflate.

---
 app/test-compress-perf/comp_perf_options.h    |  3 +
 .../comp_perf_options_parse.c                 | 75 +++++++++++++++++++
 .../comp_perf_test_cyclecount.c               | 11 ++-
 .../comp_perf_test_throughput.c               | 11 ++-
 .../comp_perf_test_verify.c                   | 11 ++-
 app/test-compress-perf/main.c                 | 55 ++++++++++----
 doc/guides/tools/comp_perf.rst                |  5 ++
 7 files changed, 148 insertions(+), 23 deletions(-)
app/test-compress-perf/comp_perf_options.h +3 -0
index d00b299247..828a7309d8 100644
@@ -61,8 +61,11 @@ struct comp_test_data {
 	uint16_t max_sgl_segs;
 	uint32_t total_segs;
 
+	uint8_t lz4_flags;
 	enum rte_comp_huffman huffman_enc;
 	enum comp_operation test_op;
+	enum rte_comp_algorithm test_algo;
+
 	int window_sz;
 	struct range_list level_lst;
 	uint8_t level;
app/test-compress-perf/comp_perf_options_parse.c +75 -0

130 lines not on this page · raw message

app/test-compress-perf/comp_perf_test_cyclecount.c +8 -3
index ce6c4d7605..4d336ec8d6 100644
@@ -193,14 +193,17 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
-				.deflate.huffman = test_data->huffman_enc,
+				.algo = test_data->test_algo,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->decomp_bufs;
 		output_bufs = mem->comp_bufs;
 		out_seg_sz = test_data->out_seg_sz;
@@ -208,12 +211,14 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->comp_bufs;
 		output_bufs = mem->decomp_bufs;
 		out_seg_sz = test_data->seg_sz;
app/test-compress-perf/comp_perf_test_throughput.c +8 -3
index c9f8237626..1f7072d223 100644
@@ -84,14 +84,17 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
-				.deflate.huffman = test_data->huffman_enc,
+				.algo = test_data->test_algo,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->decomp_bufs;
 		output_bufs = mem->comp_bufs;
 		out_seg_sz = test_data->out_seg_sz;
@@ -99,12 +102,14 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		input_bufs = mem->comp_bufs;
 		output_bufs = mem->decomp_bufs;
 		out_seg_sz = test_data->seg_sz;
app/test-compress-perf/comp_perf_test_verify.c +8 -3
index 7d6b6abecd..7bd18073cf 100644
@@ -87,14 +87,17 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_COMPRESS,
 			.compress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
-				.deflate.huffman = test_data->huffman_enc,
+				.algo = test_data->test_algo,
 				.level = test_data->level,
 				.window_size = test_data->window_sz,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE)
+			xform.compress.deflate.huffman = test_data->huffman_enc;
+		else if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.compress.lz4.flags = test_data->lz4_flags;
 		output_data_ptr = ctx->mem.compressed_data;
 		output_data_sz = &ctx->comp_data_sz;
 		input_bufs = mem->decomp_bufs;
@@ -104,12 +107,14 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type)
 		xform = (struct rte_comp_xform) {
 			.type = RTE_COMP_DECOMPRESS,
 			.decompress = {
-				.algo = RTE_COMP_ALGO_DEFLATE,
+				.algo = test_data->test_algo,
 				.chksum = RTE_COMP_CHECKSUM_NONE,
 				.window_size = test_data->window_sz,
 				.hash_algo = RTE_COMP_HASH_ALGO_NONE
 			}
 		};
+		if (test_data->test_algo == RTE_COMP_ALGO_LZ4)
+			xform.decompress.lz4.flags = test_data->lz4_flags;
 		output_data_ptr = ctx->mem.decompressed_data;
 		output_data_sz = &ctx->decomp_data_sz;
 		input_bufs = mem->comp_bufs;
app/test-compress-perf/main.c +41 -14

70 lines not on this page · raw message

doc/guides/tools/comp_perf.rst +5 -0
index 9d2f4dbe4a..5eb05a095c 100644
@@ -84,8 +84,13 @@ Application Options
 
  ``--operation [comp/decomp/comp_and_decomp]``: perform test on compression, decompression or both operations
 
+ ``--algo [null/deflate/lzs/lz4]`` : perform test on algorithm null(DMA), Deflate, lzs or lz4 (default: Deflate)
+
  ``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default: dynamic)
 
+ ``--lz4-flags N``: flags to for LZ4 parameters,
+ see `LZ4 Frame Descriptor <https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#frame-descriptor>`_ (default: no flags)
+
  ``--compress-level N``: compression level, which could be a single value, list or range (default: range between 1 and 9)
 
  ``--window-sz N``: base two log value of compression window size (default: max supported by PMD)
Signature · 1 line
2.25.1
Applied
#30 lore raw
In reply to Michael Baum
Quoted 26 lines

Add support for LZ4 compression algorithm for both API and app.

In addition, enable app to test de/comp only. This option already exists in both app doc and arguments but isn't implemented inside the application.

LZ4: https://urldefense.proofpoint.com/v2/url?u=https-
3A__github.com_lz4_lz4&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=DnL7Si2 wl_PRwpZ9TWey3eu68gBzn7DkPwuqhd6WNyo&m=zuvFwq- wCfo4E5qcsXuVW4Tg-xq4cnKWlMdx8BrlcwM9- mjeKpQgAejfzaUzSwyP&s=89JL5ZLAeh2gshZ3Gqohsr3JZomgK9pjaCnLRHpnG0w &e=
RFC:
https://urldefense.proofpoint.com/v2/url?u=https-
3A__patchwork.dpdk.org_project_dpdk_patch_20220410182622.8828-2D1- 2Drzidane- 40nvidia.com_&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=DnL7Si2wl_PRwpZ 9TWey3eu68gBzn7DkPwuqhd6WNyo&m=zuvFwq-wCfo4E5qcsXuVW4Tg- xq4cnKWlMdx8BrlcwM9- mjeKpQgAejfzaUzSwyP&s=HhD8pa6ivCYKy89RMuLN2OhehLXxbluIyVWxF0QdjXE &e=

v2:
- Add macros for all LZ4 flags in the standard.
- Rebase.

Quoted 8 lines

v3:
- Rebase.

v4:
- Change "allow test single" commit to fix and add "Cc: [email protected]".
- Take all typo fixes into separate commit.
- Squash "add LZ4 algo" commit into "add algo option" commit.
- Remove unnecessary release notes updates.

Applied to dpdk-next-crypto

Thanks