Skip to contentSkip to conversation

New crypto acceleration benchmark mode

PATCH Superseded Anatoly Burakov 2 participants 16 messages Aug 24, 2017 – Sep 20, 2017

lore
Share & tools
lore
#0 lore raw

cover New crypto acceleration benchmark mode

From: "Burakov, Anatoly" <[email protected]>

This patchset adds a new "PMD cyclecount" test mode for test-crypto-perf application. This mode is intended to measure hardware acceleration cost (in terms of cycle count) more accurately than throughput test.

The general idea is the following:
- Measure build-alloc-free cycle separately
- Alloc and build ops
- Measure completely filling up the TX ring
- Wait until ops are processed
- Measure completely draining the RX ring
- Free all allocated ops

In order to make measurements more accurate, the enqueue/dequeue is still done in bursts of specified size, but all of the bursts are now part of a "superburst" of size equal to number of descriptors configured for the device. So, if the number of descriptors configured was 2048 (the default), then 2048 ops will be enqueued and dequeued, in bursts of size specified by test command line.

The following command-line switch will run the test:
--ptest=pmd-cyclecount

In addition to a new mode, two mode command line switches are added:
- --desc-nb - configure number of cryptodev descriptors. This value was previously hardcoded to 2048, but is now configurable and set to 2048 by default (so existing behavior is unchanged).
- --pmd-cyclecount-delay-ms - pmd-cyclecount-specific parameter that configures the delay (in milliseconds) between TX and RX superbursts, to allow hardware to process ops. Set to 0 by default, and it is expected that each user will tune it for every device. This has no effect on other benchmark modes.

PMD cyclecount mode can be used to benchmark software cryptodev drivers as well, but the results will be far less accurate for smaller burst sizes.

Anatoly Burakov (4):
test-crypto-perf: add nb-desc parameter
doc: document new nb-desc parameter for test-crypto-perf app
test-crypto-perf: add new PMD benchmarking mode
doc: document new pmd-cyclecount benchmarking mode in test-crypto-perf

 app/test-crypto-perf/Makefile                    |   1 +
 app/test-crypto-perf/cperf_options.h             |  11 +-
 app/test-crypto-perf/cperf_options_parsing.c     |  54 ++
 app/test-crypto-perf/cperf_test_pmd_cyclecount.c | 707 +++++++++++++++++++++++
 app/test-crypto-perf/cperf_test_pmd_cyclecount.h |  61 ++
 app/test-crypto-perf/main.c                      |  11 +-
 doc/guides/rel_notes/release_17_11.rst           |   6 +
 doc/guides/tools/cryptoperf.rst                  |  14 +-
 8 files changed, 861 insertions(+), 4 deletions(-)
 create mode 100644 app/test-crypto-perf/cperf_test_pmd_cyclecount.c
 create mode 100644 app/test-crypto-perf/cperf_test_pmd_cyclecount.h
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/4 test-crypto-perf: add nb-desc parameter

This parameter makes number of cryptodev descriptors adjustable and defaults to earlier hardcoded default of 2048.

---
 app/test-crypto-perf/cperf_options.h         |  2 ++
 app/test-crypto-perf/cperf_options_parsing.c | 21 +++++++++++++++++++++
 app/test-crypto-perf/main.c                  |  2 +-
 3 files changed, 24 insertions(+), 1 deletion(-)
app/test-crypto-perf/cperf_options.h +2 -0
index 10cd2d8..edd6b79 100644
@@ -12,6 +12,7 @@
 #define CPERF_BURST_SIZE	("burst-sz")
 #define CPERF_BUFFER_SIZE	("buffer-sz")
 #define CPERF_SEGMENTS_NB	("segments-nb")
+#define CPERF_DESC_NB		("desc-nb")
 
 #define CPERF_DEVTYPE		("devtype")
 #define CPERF_OPTYPE		("optype")
@@ -68,6 +69,7 @@ struct cperf_options {
 	uint32_t total_ops;
 	uint32_t segments_nb;
 	uint32_t test_buffer_size;
+	uint32_t nb_descriptors;
 
 	uint32_t sessionless:1;
 	uint32_t out_of_place:1;
app/test-crypto-perf/cperf_options_parsing.c +21 -0
index 085aa8f..f4097d9 100644
@@ -340,6 +340,24 @@ parse_segments_nb(struct cperf_options *opts, const char *arg)
 }
 
 static int
+parse_desc_nb(struct cperf_options *opts, const char *arg)
+{
+	int ret = parse_uint32_t(&opts->nb_descriptors, arg);
+
+	if (ret) {
+		RTE_LOG(ERR, USER1, "failed to parse descriptors number\n");
+		return -1;
+	}
+
+	if (opts->nb_descriptors == 0) {
+		RTE_LOG(ERR, USER1, "invalid descriptors number specified\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+static int
 parse_device_type(struct cperf_options *opts, const char *arg)
 {
 	if (strlen(arg) > (sizeof(opts->device_type) - 1))
@@ -641,6 +659,7 @@ static struct option lgopts[] = {
 	{ CPERF_BURST_SIZE, required_argument, 0, 0 },
 	{ CPERF_BUFFER_SIZE, required_argument, 0, 0 },
 	{ CPERF_SEGMENTS_NB, required_argument, 0, 0 },
+	{ CPERF_DESC_NB, required_argument, 0, 0 },
 
 	{ CPERF_DEVTYPE, required_argument, 0, 0 },
 	{ CPERF_OPTYPE, required_argument, 0, 0 },
@@ -684,6 +703,7 @@ cperf_options_default(struct cperf_options *opts)
 
 	opts->pool_sz = 8192;
 	opts->total_ops = 10000000;
+	opts->nb_descriptors = 2048;
 
 	opts->buffer_size_list[0] = 64;
 	opts->buffer_size_count = 1;
@@ -740,6 +760,7 @@ cperf_opts_parse_long(int opt_idx, struct cperf_options *opts)
 		{ CPERF_BURST_SIZE,	parse_burst_sz },
 		{ CPERF_BUFFER_SIZE,	parse_buffer_sz },
 		{ CPERF_SEGMENTS_NB,	parse_segments_nb },
+		{ CPERF_DESC_NB,	parse_desc_nb },
 		{ CPERF_DEVTYPE,	parse_device_type },
 		{ CPERF_OPTYPE,		parse_op_type },
 		{ CPERF_SESSIONLESS,	parse_sessionless },
app/test-crypto-perf/main.c +1 -1
index 99f5d3e..7e6ca8e 100644
@@ -123,7 +123,7 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs,
 		};
 
 		struct rte_cryptodev_qp_conf qp_conf = {
-				.nb_descriptors = 2048
+			    .nb_descriptors = opts->nb_descriptors
 		};
 
 
Signature · 1 line
2.7.4
#3 lore raw

PATCH 2/4 doc: document new nb-desc parameter for test-crypto-perf app

In reply to Anatoly Burakov
---
 doc/guides/tools/cryptoperf.rst | 4 ++++
 1 file changed, 4 insertions(+)
doc/guides/tools/cryptoperf.rst +4 -0
index 457f817..985848b 100644
@@ -325,6 +325,10 @@ The following are the appication command-line options:
 
         Set the size of digest.
 
+* ``--desc-nb <n>``
+
+        Set default number of descriptors in cryptodev.
+
 * ``--csv-friendly``
 
         Enable test result output CSV friendly rather than human friendly.
Signature · 1 line
2.7.4
#5 lore raw

PATCH 3/4 test-crypto-perf: add new PMD benchmarking mode

In reply to Anatoly Burakov

This patch adds a new benchmarking mode, which is intended for microbenchmarking individual parts of the cryptodev framework, specifically crypto ops alloc-build-free, cryptodev PMD enqueue and cryptodev PMD dequeue.

It works by first benchmarking crypto operation alloc-build-free loop (no enqueues/dequeues happening), and then benchmarking enqueue and dequeue separately, by first completely filling up the TX queue, and then completely draining the RX queue.

Results are shown as cycle counts per alloc/build/free, PMD enqueue and PMD dequeue.

One new test mode is added: "pmd-cyclecount"
(called with --ptest=pmd-cyclecount)

New command-line argument is also added:
--pmd-cyclecount-delay-ms: this is a pmd-cyclecount-specific parameter
that controls the delay between enqueue and dequeue. This is useful for benchmarking hardware acceleration, as hardware may not be able to keep up with enqueued packets. This parameter can be increased if there are large amounts of dequeue retries.

---
 app/test-crypto-perf/Makefile                    |   1 +
 app/test-crypto-perf/cperf_options.h             |   9 +-
 app/test-crypto-perf/cperf_options_parsing.c     |  33 ++
 app/test-crypto-perf/cperf_test_pmd_cyclecount.c | 707 +++++++++++++++++++++++
 app/test-crypto-perf/cperf_test_pmd_cyclecount.h |  61 ++
 app/test-crypto-perf/main.c                      |   9 +-
 6 files changed, 818 insertions(+), 2 deletions(-)
 create mode 100644 app/test-crypto-perf/cperf_test_pmd_cyclecount.c
 create mode 100644 app/test-crypto-perf/cperf_test_pmd_cyclecount.h
app/test-crypto-perf/Makefile +1 -0
index e4a989f..821e8e5 100644
@@ -42,6 +42,7 @@ SRCS-y += cperf_options_parsing.c
 SRCS-y += cperf_test_vectors.c
 SRCS-y += cperf_test_throughput.c
 SRCS-y += cperf_test_latency.c
+SRCS-y += cperf_test_pmd_cyclecount.c
 SRCS-y += cperf_test_verify.c
 SRCS-y += cperf_test_vector_parsing.c
 
app/test-crypto-perf/cperf_options.h +8 -1
index edd6b79..b82d78b 100644
@@ -41,12 +41,17 @@
 
 #define CPERF_CSV		("csv-friendly")
 
+/* benchmark-specific options */
+#define CPERF_PMD_CYCLECOUNT_DELAY_MS \
+	("pmd-cyclecount-delay-ms")
+
 #define MAX_LIST 32
 
 enum cperf_perf_test_type {
 	CPERF_TEST_TYPE_THROUGHPUT,
 	CPERF_TEST_TYPE_LATENCY,
-	CPERF_TEST_TYPE_VERIFY
+	CPERF_TEST_TYPE_VERIFY,
+	CPERF_TEST_TYPE_PMD_CYCLECOUNT
 };
 
 
@@ -115,6 +120,8 @@ struct cperf_options {
 	uint32_t min_burst_size;
 	uint32_t inc_burst_size;
 
+	/* pmd-cyclecount specific options */
+	uint32_t pmdcc_delay;
 };
 
 void
app/test-crypto-perf/cperf_options_parsing.c +33 -0
index f4097d9..42a920f 100644
@@ -76,6 +76,10 @@ parse_cperf_test_type(struct cperf_options *opts, const char *arg)
 		{
 			cperf_test_type_strs[CPERF_TEST_TYPE_LATENCY],
 			CPERF_TEST_TYPE_LATENCY
+		},
+		{
+			cperf_test_type_strs[CPERF_TEST_TYPE_PMD_CYCLECOUNT],
+			CPERF_TEST_TYPE_PMD_CYCLECOUNT
 		}
 	};
 
@@ -641,6 +645,20 @@ parse_csv_friendly(struct cperf_options *opts, const char *arg __rte_unused)
 	return 0;
 }
 
+static int
+parse_pmd_cyclecount_delay_ms(struct cperf_options *opts,
+			const char *arg)
+{
+	int ret = parse_uint32_t(&opts->pmdcc_delay, arg);
+
+	if (ret) {
+		RTE_LOG(ERR, USER1, "failed to parse pmd-cyclecount delay\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 typedef int (*option_parser_t)(struct cperf_options *opts,
 		const char *arg);
 
@@ -693,6 +711,8 @@ static struct option lgopts[] = {
 
 	{ CPERF_CSV, no_argument, 0, 0},
 
+	{ CPERF_PMD_CYCLECOUNT_DELAY_MS, required_argument, 0, 0 },
+
 	{ NULL, 0, 0, 0 }
 };
 
@@ -747,6 +767,8 @@ cperf_options_default(struct cperf_options *opts)
 	opts->aead_aad_sz = 0;
 
 	opts->digest_sz = 12;
+
+	opts->pmdcc_delay = 0;
 }
 
 static int
@@ -782,6 +804,7 @@ cperf_opts_parse_long(int opt_idx, struct cperf_options *opts)
 		{ CPERF_AEAD_AAD_SZ,	parse_aead_aad_sz },
 		{ CPERF_DIGEST_SZ,	parse_digest_sz },
 		{ CPERF_CSV,		parse_csv_friendly},
+		{ CPERF_PMD_CYCLECOUNT_DELAY_MS, parse_pmd_cyclecount_delay_ms},
 	};
 	unsigned int i;
 
@@ -925,6 +948,14 @@ cperf_options_check(struct cperf_options *options)
 		return -EINVAL;
 	}
 
+	if (options->test == CPERF_TEST_TYPE_PMD_CYCLECOUNT &&
+			options->pool_sz < options->nb_descriptors) {
+		RTE_LOG(ERR, USER1, "For pmd cyclecount benchmarks, pool size "
+				"must be equal or greater than the number of "
+				"cryptodev descriptors.\n");
+		return -EINVAL;
+	}
+
 	if (options->op_type == CPERF_CIPHER_THEN_AUTH) {
 		if (options->cipher_op != RTE_CRYPTO_CIPHER_OP_ENCRYPT &&
 				options->auth_op !=
@@ -993,6 +1024,8 @@ cperf_options_dump(struct cperf_options *opts)
 	printf("# crypto operation: %s\n", cperf_op_type_strs[opts->op_type]);
 	printf("# sessionless: %s\n", opts->sessionless ? "yes" : "no");
 	printf("# out of place: %s\n", opts->out_of_place ? "yes" : "no");
+	if (opts->test == CPERF_TEST_TYPE_PMD_CYCLECOUNT)
+		printf("# inter-burst delay: %u ms\n", opts->pmdcc_delay);
 
 	printf("#\n");
 
app/test-crypto-perf/cperf_test_pmd_cyclecount.c new file +707 -0

707 lines not on this page · raw message

app/test-crypto-perf/cperf_test_pmd_cyclecount.h new file +61 -0

61 lines not on this page · raw message

app/test-crypto-perf/main.c +8 -1
index 7e6ca8e..ec11678 100644
@@ -42,6 +42,7 @@
 #include "cperf_test_throughput.h"
 #include "cperf_test_latency.h"
 #include "cperf_test_verify.h"
+#include "cperf_test_pmd_cyclecount.h"
 
 #define NUM_SESSIONS 2048
 #define SESS_MEMPOOL_CACHE_SIZE 64
@@ -49,7 +50,8 @@
 const char *cperf_test_type_strs[] = {
 	[CPERF_TEST_TYPE_THROUGHPUT] = "throughput",
 	[CPERF_TEST_TYPE_LATENCY] = "latency",
-	[CPERF_TEST_TYPE_VERIFY] = "verify"
+	[CPERF_TEST_TYPE_VERIFY] = "verify",
+	[CPERF_TEST_TYPE_PMD_CYCLECOUNT] = "pmd-cyclecount"
 };
 
 const char *cperf_op_type_strs[] = {
@@ -75,6 +77,11 @@ const struct cperf_test cperf_testmap[] = {
 				cperf_verify_test_constructor,
 				cperf_verify_test_runner,
 				cperf_verify_test_destructor
+		},
+		[CPERF_TEST_TYPE_PMD_CYCLECOUNT] = {
+				cperf_pmd_cyclecount_test_constructor,
+				cperf_pmd_cyclecount_test_runner,
+				cperf_pmd_cyclecount_test_destructor
 		}
 };
 
Signature · 1 line
2.7.4
#8 lore raw

PATCH 4/4 doc: document new pmd-cyclecount benchmarking mode in test-crypto-perf

In reply to Anatoly Burakov

Also, document the new pmd-cyclecount-specific flag.

---
 doc/guides/rel_notes/release_17_11.rst |  6 ++++++
 doc/guides/tools/cryptoperf.rst        | 10 +++++++++-
 2 files changed, 15 insertions(+), 1 deletion(-)
doc/guides/rel_notes/release_17_11.rst +6 -0
index 170f4f9..860858c 100644
@@ -41,6 +41,12 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+* **Add new benchmarking mode to dpdk-crypto-perf-test application.**
+
+  Added new "PMD cyclecount" benchmark mode to dpdk-crypto-perf-test application
+  that displays more detailed breakdown of CPU cycles used by hardware
+  acceleration.
+
 
 Resolved Issues
 ---------------
doc/guides/tools/cryptoperf.rst +9 -1
index 985848b..482e1cf 100644
@@ -50,7 +50,8 @@ offload are still consumed by the test tool and included in the cycle-count.
 These cycles are consumed by retries and inefficient API calls enqueuing and
 dequeuing smaller bursts than specified by the cmdline parameter. This results
 in a larger cycle-count measurement and should not be interpreted as an offload
-cost measurement.
+cost measurement. Using "pmd-cyclecount" mode will give a better idea of
+actual costs of hardware acceleration.
 
 On hardware devices the throughput measurement is not necessarily the maximum
 possible for the device, e.g. it may be necessary to use multiple cores to keep
@@ -134,6 +135,7 @@ The following are the appication command-line options:
            throughput
            latency
            verify
+           pmd-cyclecount
 
 * ``--silent``
 
@@ -329,6 +331,12 @@ The following are the appication command-line options:
 
         Set default number of descriptors in cryptodev.
 
+* ``--pmd-cyclecount-delay-ms <n>``
+
+        Add a delay (in milliseconds) between enqueue and dequeue in
+        pmd-cyclecount benchmarking mode (useful when benchmarking
+        hardware acceleration).
+
 * ``--csv-friendly``
 
         Enable test result output CSV friendly rather than human friendly.
Signature · 1 line
2.7.4
#2 lore raw

Re: [PATCH 1/4] test-crypto-perf: add nb-desc parameter

In reply to Anatoly Burakov
Quoted 10 lines

-----Original Message-----
From: Burakov, Anatoly
Sent: Thursday, August 24, 2017 11:48 AM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Doherty, Declan <[email protected]>; Burakov, Anatoly <[email protected]>
Subject: [PATCH 1/4] test-crypto-perf: add nb-desc parameter

Title should be "app/crypto-perf". Rest of the patch is OK.

Thanks,
Pablo

Quoted 11 lines

This parameter makes number of cryptodev descriptors adjustable and defaults to earlier hardcoded default of 2048.

---
 app/test-crypto-perf/cperf_options.h         |  2 ++
 app/test-crypto-perf/cperf_options_parsing.c | 21
+++++++++++++++++++++
 app/test-crypto-perf/main.c                  |  2 +-
 3 files changed, 24 insertions(+), 1 deletion(-)
#4 lore raw

Re: [PATCH 2/4] doc: document new nb-desc parameter for test-crypto-perf app

In reply to Anatoly Burakov
Quoted 11 lines

-----Original Message-----
From: Burakov, Anatoly
Sent: Thursday, August 24, 2017 11:48 AM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Doherty, Declan <[email protected]>; Burakov, Anatoly <[email protected]>
Subject: [PATCH 2/4] doc: document new nb-desc parameter for test- crypto-perf app

Since this is related to the previous patch and it is a small addition, merge it with the previous patch.

Quoted 15 lines
---
 doc/guides/tools/cryptoperf.rst | 4 ++++
 1 file changed, 4 insertions(+)
(quoted diff) +0 -0

b/doc/guides/tools/cryptoperf.rst index 457f817..985848b 100644

doc/guides/tools/cryptoperf.rst +0 -0
@@ -325,6 +325,10 @@ The following are the appication command-line

options:

Set the size of digest.

doc/guides/tools/cryptoperf.rst The following are the appication command-line options: · line 330

Quoted 3 lines
doc/guides/tools/cryptoperf.rst +3 -0
+* ``--desc-nb <n>``
+
+        Set default number of descriptors in cryptodev.

I think this should be reworded, as it is not setting the default number of descriptor (which is 2048), but it is setting the number of descriptors for each crypto device.

Quoted 6 lines

+
* ``--csv-friendly``

Enable test result output CSV friendly rather than human friendly.
--
2.7.4

#6 lore raw

Re: [PATCH 3/4] test-crypto-perf: add new PMD benchmarking mode

In reply to Anatoly Burakov
Quoted 35 lines

-----Original Message-----
From: Burakov, Anatoly
Sent: Thursday, August 24, 2017 11:48 AM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Doherty, Declan <[email protected]>; Burakov, Anatoly <[email protected]>
Subject: [PATCH 3/4] test-crypto-perf: add new PMD benchmarking mode

This patch adds a new benchmarking mode, which is intended for microbenchmarking individual parts of the cryptodev framework, specifically crypto ops alloc-build-free, cryptodev PMD enqueue and cryptodev PMD dequeue.

It works by first benchmarking crypto operation alloc-build-free loop (no enqueues/dequeues happening), and then benchmarking enqueue and dequeue separately, by first completely filling up the TX queue, and then completely draining the RX queue.

Results are shown as cycle counts per alloc/build/free, PMD enqueue and PMD dequeue.

One new test mode is added: "pmd-cyclecount"
(called with --ptest=pmd-cyclecount)

New command-line argument is also added:
--pmd-cyclecount-delay-ms: this is a pmd-cyclecount-specific parameter
that controls the delay between enqueue and dequeue. This is useful for benchmarking hardware acceleration, as hardware may not be able to keep up with enqueued packets. This parameter can be increased if there are large amounts of dequeue retries.

Quoted 1 line

Git-check-log is complaining about this tag. Actually, the author is "Anatoly Burakov", but here is "Burakov, Anatoly". Looks like this is something to fix.

Quoted 1 line

---

...

Quoted 5 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +0 -0

crypto-perf/cperf_test_pmd_cyclecount.c
new file mode 100644
index 0000000..ef7fa83

...

Quoted 17 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +17 -0
+void *
+cperf_pmd_cyclecount_test_constructor(struct rte_mempool *sess_mp,
+			uint8_t dev_id, uint16_t qp_id,
+			const struct cperf_options *options,
+			const struct cperf_test_vector *test_vector,
+			const struct cperf_op_fns *op_fns)
+{
+	struct cperf_pmd_cyclecount_ctx *ctx = NULL;
+	unsigned int mbuf_idx = 0;
+	char pool_name[32] = "";
+
+	/* preallocate buffers for crypto ops as they can get quite big */
+	size_t alloc_sz = sizeof(struct rte_crypto_op *) *
+			options->nb_descriptors;
+
+	ctx = rte_malloc(NULL, sizeof(struct cperf_pmd_cyclecount_ctx), 0);
+	if (ctx == NULL)

app/test-

Quoted 10 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +10 -0
+		goto err;
+
+	ctx->dev_id = dev_id;
+	ctx->qp_id = qp_id;
+
+	ctx->populate_ops = op_fns->populate_ops;
+	ctx->options = options;
+	ctx->test_vector = test_vector;
+
+	/* IV goes at the end of the cryptop operation */

Typo here: "crypto"

Quoted 3 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +3 -0
+	uint16_t iv_offset = sizeof(struct rte_crypto_op) +
+		sizeof(struct rte_crypto_sym_op);
+

...

app/test-

Quoted 6 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +6 -0
+
+	snprintf(pool_name, sizeof(pool_name), "cperf_op_pool_cdev_%d",
+			dev_id);
+
+	uint16_t priv_size = test_vector->cipher_iv.length +
+		test_vector->auth_iv.length;

Missing "+ test_vector->aead_iv.length", added in a patch at the end of the previous release.

Quoted 1 line

+

...

app/test-

Quoted 4 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +2 -0
+/* benchmark alloc-build-free of ops */ static inline int
+pmd_cyclecount_bench_ops(struct pmd_cyclecount_state *state,

uint32_t cur_op,

app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +1 -0
+			 uint16_t test_burst_size)

Use two tabs for the next line, instead of aligning with the parenthesis above. Look at other functions that have the same issue.

app/test-

Quoted 8 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +8 -0
+{
+	uint32_t iter_ops_left = state->opts->total_ops - cur_op;
+	uint32_t iter_ops_needed = RTE_MIN(state->opts->nb_descriptors,
+					   iter_ops_left);
+	uint32_t cur_iter_op;
+
+	for (cur_iter_op = 0; cur_iter_op < iter_ops_needed;
+	     cur_iter_op += test_burst_size) {

Same comment as above, about the alignment.

app/test-

Quoted 10 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +1 -0
+		uint32_t burst_size = RTE_MIN(state->opts->total_ops -

cur_op,

app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +2 -0
+					      test_burst_size);
+		struct rte_crypto_op **ops = &state->ctx-
Quoted 1 line

ops[cur_iter_op];

app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +5 -0
+
+		if (burst_size != rte_crypto_op_bulk_alloc(
+			state->ctx->crypto_op_pool,
+			RTE_CRYPTO_OP_TYPE_SYMMETRIC,
+			ops, burst_size))

Same as above, add an extra tab.

Quoted 28 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +28 -0
+			return -1;
+
+		/* Setup crypto op, attach mbuf etc */
+		(state->ctx->populate_ops)(ops,
+				&state->ctx->mbufs_in[cur_iter_op],
+				&state->ctx->mbufs_out[cur_iter_op],
+				burst_size, state->ctx->sess,
+				state->opts,
+				state->ctx->test_vector, iv_offset);
+
+#ifdef CPERF_LINEARIZATION_ENABLE
+		/* Check if source mbufs require coalescing */
+		if (state->linearize) {
+			uint8_t i;
+			for (i = 0; i < burst_size; i++) {
+				struct rte_mbuf *src =
+						ops[i]->sym->m_src;
+				rte_pktmbuf_linearize(src);
+			}
+		}
+#endif /* CPERF_LINEARIZATION_ENABLE */
+		rte_mempool_put_bulk(state->ctx->crypto_op_pool,
+				(void **)ops, burst_size);
+	}
+
+	return 0;
+}
+

...

Quoted 7 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +2 -0
+
+/* benchmark enqueue, returns number of ops enqueued */ static

uint32_t

app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +2 -0
+pmd_cyclecount_bench_enq(struct pmd_cyclecount_state *state,
+			 uint32_t iter_ops_needed, uint16_t

test_burst_size) {

app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +1 -0
+	/* Enqueue full descriptor ring of ops on crypto device */

app/test-

Quoted 10 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +10 -0
+	uint32_t cur_iter_op = 0;
+	while (cur_iter_op < iter_ops_needed) {
+		uint32_t burst_size =
+				RTE_MIN(iter_ops_needed - cur_iter_op,
+					test_burst_size);
+		struct rte_crypto_op **ops =
+				&state->ctx->ops[cur_iter_op];
+		uint32_t burst_enqd;
+
+		burst_enqd =

No need to place the function in a separate line, I think.

Quoted 20 lines
app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +8 -0
+				rte_cryptodev_enqueue_burst(
+					state->ctx->dev_id,
+					state->ctx->qp_id,
+					ops, burst_size);
+
+		/* if we couldn't enqueue anything, the queue is full */
+		if (!burst_enqd) {
+			/* don't try to dequeue anything we didn't enqueue

*/

app/test- renamed from app/test-crypto-perf/cperf_test_pmd_cyclecount.c +11 -0
+			return cur_iter_op;
+		}
+
+		if (burst_enqd < burst_size)
+			state->ops_enq_retries++;
+		state->ops_enqd += burst_enqd;
+		cur_iter_op += burst_enqd;
+	}
+	return iter_ops_needed;
+}
+
#7 lore raw

Re: [PATCH 3/4] test-crypto-perf: add new PMD benchmarking mode

In reply to Anatoly Burakov
Quoted 36 lines

-----Original Message-----
From: Burakov, Anatoly
Sent: Thursday, August 24, 2017 11:48 AM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Doherty, Declan <[email protected]>; Burakov, Anatoly <[email protected]>
Subject: [PATCH 3/4] test-crypto-perf: add new PMD benchmarking mode

This patch adds a new benchmarking mode, which is intended for microbenchmarking individual parts of the cryptodev framework, specifically crypto ops alloc-build-free, cryptodev PMD enqueue and cryptodev PMD dequeue.

It works by first benchmarking crypto operation alloc-build-free loop (no enqueues/dequeues happening), and then benchmarking enqueue and dequeue separately, by first completely filling up the TX queue, and then completely draining the RX queue.

Results are shown as cycle counts per alloc/build/free, PMD enqueue and PMD dequeue.

One new test mode is added: "pmd-cyclecount"
(called with --ptest=pmd-cyclecount)

New command-line argument is also added:
--pmd-cyclecount-delay-ms: this is a pmd-cyclecount-specific parameter
that controls the delay between enqueue and dequeue. This is useful for benchmarking hardware acceleration, as hardware may not be able to keep up with enqueued packets. This parameter can be increased if there are large amounts of dequeue retries.

...

Quoted 5 lines
app/test-crypto- renamed from app/test-crypto-perf/cperf_options_parsing.c +0 -0

perf/cperf_options_parsing.c
index f4097d9..42a920f 100644

app/test-crypto-perf/cperf_options_parsing.c +0 -0

...

app/test-crypto-perf/cperf_options_parsing.c cperf_opts_parse_long(int opt_idx, struct · line 809

Quoted 8 lines

static int

app/test-crypto-perf/cperf_options_parsing.c +1 -0
@@ -782,6 +804,7 @@ cperf_opts_parse_long(int opt_idx, struct
 cperf_options *opts)
 		{ CPERF_AEAD_AAD_SZ,	parse_aead_aad_sz },
 		{ CPERF_DIGEST_SZ,	parse_digest_sz },
 		{ CPERF_CSV,		parse_csv_friendly},
+		{ CPERF_PMD_CYCLECOUNT_DELAY_MS,
 parse_pmd_cyclecount_delay_ms},

Sorry, I missed this one in the previous email.
To have a consistent alignment, could you make the name of this macro shorter?

Thanks,
Pablo

#9 lore raw

Re: [PATCH 4/4] doc: document new pmd-cyclecount benchmarking mode in test-crypto-perf

In reply to Anatoly Burakov
Quoted 12 lines

-----Original Message-----
From: Burakov, Anatoly
Sent: Thursday, August 24, 2017 11:48 AM
To: [email protected]
Cc: De Lara Guarch, Pablo <[email protected]>; Doherty, Declan <[email protected]>; Burakov, Anatoly <[email protected]>
Subject: [PATCH 4/4] doc: document new pmd-cyclecount benchmarking mode in test-crypto-perf

Quoted 1 line

Also, document the new pmd-cyclecount-specific flag.

As said with patch 2, I would merge this patch with patch number of 3.

Quoted 17 lines
---
 doc/guides/rel_notes/release_17_11.rst |  6 ++++++
 doc/guides/tools/cryptoperf.rst        | 10 +++++++++-
 2 files changed, 15 insertions(+), 1 deletion(-)
doc/guides/rel_notes/release_17_11.rst +0 -0
index 170f4f9..860858c 100644
@@ -41,6 +41,12 @@ New Features
      Also, make sure to start the actual text at the margin.
 

=========================================================

Quoted 1 line

+* **Add new benchmarking mode to dpdk-crypto-perf-test application.**

Name of the app is "dpdk-test-crypto-perf (above and below).

Quoted 6 lines
doc/guides/rel_notes/release_17_11.rst +5 -0
+
+  Added new "PMD cyclecount" benchmark mode to dpdk-crypto-perf-test
+ application  that displays more detailed breakdown of CPU cycles used
+ by hardware  acceleration.
+
#12 lore raw

PATCH v2 2/2 app/crypto-perf: add new PMD benchmarking mode

In reply to Anatoly Burakov

This patch adds a new benchmarking mode, which is intended for microbenchmarking individual parts of the cryptodev framework, specifically crypto ops alloc-build-free, cryptodev PMD enqueue and cryptodev PMD dequeue.

It works by first benchmarking crypto operation alloc-build-free loop (no enqueues/dequeues happening), and then benchmarking enqueue and dequeue separately, by first completely filling up the TX queue, and then completely draining the RX queue.

Results are shown as cycle counts per alloc/build/free, PMD enqueue and PMD dequeue.

One new test mode is added: "pmd-cyclecount"
(called with --ptest=pmd-cyclecount)

New command-line argument is also added:
--pmd-cyclecount-delay-ms: this is a pmd-cyclecount-specific parameter
that controls the delay between enqueue and dequeue. This is useful for benchmarking hardware acceleration, as hardware may not be able to keep up with enqueued packets. This parameter can be increased if there are large amounts of dequeue retries.

---
v2: fixed commit message
fixed code style and naming

 app/test-crypto-perf/Makefile                    |   1 +
 app/test-crypto-perf/cperf_options.h             |   8 +-
 app/test-crypto-perf/cperf_options_parsing.c     |  33 ++
 app/test-crypto-perf/cperf_test_pmd_cyclecount.c | 675 +++++++++++++++++++++++
 app/test-crypto-perf/cperf_test_pmd_cyclecount.h |  61 ++
 app/test-crypto-perf/main.c                      |   9 +-
 doc/guides/rel_notes/release_17_11.rst           |   6 +
 doc/guides/tools/cryptoperf.rst                  |  10 +-
 8 files changed, 800 insertions(+), 3 deletions(-)
 create mode 100644 app/test-crypto-perf/cperf_test_pmd_cyclecount.c
 create mode 100644 app/test-crypto-perf/cperf_test_pmd_cyclecount.h
app/test-crypto-perf/Makefile +1 -0
index e4a989f..821e8e5 100644
@@ -42,6 +42,7 @@ SRCS-y += cperf_options_parsing.c
 SRCS-y += cperf_test_vectors.c
 SRCS-y += cperf_test_throughput.c
 SRCS-y += cperf_test_latency.c
+SRCS-y += cperf_test_pmd_cyclecount.c
 SRCS-y += cperf_test_verify.c
 SRCS-y += cperf_test_vector_parsing.c
 
app/test-crypto-perf/cperf_options.h +7 -1
index edd6b79..2f42cb6 100644
@@ -41,12 +41,16 @@
 
 #define CPERF_CSV		("csv-friendly")
 
+/* benchmark-specific options */
+#define CPERF_PMDCC_DELAY_MS	("pmd-cyclecount-delay-ms")
+
 #define MAX_LIST 32
 
 enum cperf_perf_test_type {
 	CPERF_TEST_TYPE_THROUGHPUT,
 	CPERF_TEST_TYPE_LATENCY,
-	CPERF_TEST_TYPE_VERIFY
+	CPERF_TEST_TYPE_VERIFY,
+	CPERF_TEST_TYPE_PMDCC
 };
 
 
@@ -115,6 +119,8 @@ struct cperf_options {
 	uint32_t min_burst_size;
 	uint32_t inc_burst_size;
 
+	/* pmd-cyclecount specific options */
+	uint32_t pmdcc_delay;
 };
 
 void
app/test-crypto-perf/cperf_options_parsing.c +33 -0
index f4097d9..bc7d121 100644
@@ -76,6 +76,10 @@ parse_cperf_test_type(struct cperf_options *opts, const char *arg)
 		{
 			cperf_test_type_strs[CPERF_TEST_TYPE_LATENCY],
 			CPERF_TEST_TYPE_LATENCY
+		},
+		{
+			cperf_test_type_strs[CPERF_TEST_TYPE_PMDCC],
+			CPERF_TEST_TYPE_PMDCC
 		}
 	};
 
@@ -641,6 +645,20 @@ parse_csv_friendly(struct cperf_options *opts, const char *arg __rte_unused)
 	return 0;
 }
 
+static int
+parse_pmd_cyclecount_delay_ms(struct cperf_options *opts,
+			const char *arg)
+{
+	int ret = parse_uint32_t(&opts->pmdcc_delay, arg);
+
+	if (ret) {
+		RTE_LOG(ERR, USER1, "failed to parse pmd-cyclecount delay\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 typedef int (*option_parser_t)(struct cperf_options *opts,
 		const char *arg);
 
@@ -693,6 +711,8 @@ static struct option lgopts[] = {
 
 	{ CPERF_CSV, no_argument, 0, 0},
 
+	{ CPERF_PMDCC_DELAY_MS, required_argument, 0, 0 },
+
 	{ NULL, 0, 0, 0 }
 };
 
@@ -747,6 +767,8 @@ cperf_options_default(struct cperf_options *opts)
 	opts->aead_aad_sz = 0;
 
 	opts->digest_sz = 12;
+
+	opts->pmdcc_delay = 0;
 }
 
 static int
@@ -782,6 +804,7 @@ cperf_opts_parse_long(int opt_idx, struct cperf_options *opts)
 		{ CPERF_AEAD_AAD_SZ,	parse_aead_aad_sz },
 		{ CPERF_DIGEST_SZ,	parse_digest_sz },
 		{ CPERF_CSV,		parse_csv_friendly},
+		{ CPERF_PMDCC_DELAY_MS,	parse_pmd_cyclecount_delay_ms},
 	};
 	unsigned int i;
 
@@ -925,6 +948,14 @@ cperf_options_check(struct cperf_options *options)
 		return -EINVAL;
 	}
 
+	if (options->test == CPERF_TEST_TYPE_PMDCC &&
+			options->pool_sz < options->nb_descriptors) {
+		RTE_LOG(ERR, USER1, "For pmd cyclecount benchmarks, pool size "
+				"must be equal or greater than the number of "
+				"cryptodev descriptors.\n");
+		return -EINVAL;
+	}
+
 	if (options->op_type == CPERF_CIPHER_THEN_AUTH) {
 		if (options->cipher_op != RTE_CRYPTO_CIPHER_OP_ENCRYPT &&
 				options->auth_op !=
@@ -993,6 +1024,8 @@ cperf_options_dump(struct cperf_options *opts)
 	printf("# crypto operation: %s\n", cperf_op_type_strs[opts->op_type]);
 	printf("# sessionless: %s\n", opts->sessionless ? "yes" : "no");
 	printf("# out of place: %s\n", opts->out_of_place ? "yes" : "no");
+	if (opts->test == CPERF_TEST_TYPE_PMDCC)
+		printf("# inter-burst delay: %u ms\n", opts->pmdcc_delay);
 
 	printf("#\n");
 
app/test-crypto-perf/cperf_test_pmd_cyclecount.c new file +675 -0

675 lines not on this page · raw message

app/test-crypto-perf/cperf_test_pmd_cyclecount.h new file +61 -0

61 lines not on this page · raw message

app/test-crypto-perf/main.c +8 -1
index 7e6ca8e..ffa7180 100644
@@ -42,6 +42,7 @@
 #include "cperf_test_throughput.h"
 #include "cperf_test_latency.h"
 #include "cperf_test_verify.h"
+#include "cperf_test_pmd_cyclecount.h"
 
 #define NUM_SESSIONS 2048
 #define SESS_MEMPOOL_CACHE_SIZE 64
@@ -49,7 +50,8 @@
 const char *cperf_test_type_strs[] = {
 	[CPERF_TEST_TYPE_THROUGHPUT] = "throughput",
 	[CPERF_TEST_TYPE_LATENCY] = "latency",
-	[CPERF_TEST_TYPE_VERIFY] = "verify"
+	[CPERF_TEST_TYPE_VERIFY] = "verify",
+	[CPERF_TEST_TYPE_PMDCC] = "pmd-cyclecount"
 };
 
 const char *cperf_op_type_strs[] = {
@@ -75,6 +77,11 @@ const struct cperf_test cperf_testmap[] = {
 				cperf_verify_test_constructor,
 				cperf_verify_test_runner,
 				cperf_verify_test_destructor
+		},
+		[CPERF_TEST_TYPE_PMDCC] = {
+				cperf_pmd_cyclecount_test_constructor,
+				cperf_pmd_cyclecount_test_runner,
+				cperf_pmd_cyclecount_test_destructor
 		}
 };
 
doc/guides/rel_notes/release_17_11.rst +6 -0
index 170f4f9..cb1e0d4 100644
@@ -41,6 +41,12 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+* **Add new benchmarking mode to dpdk-test-crypto-perf application.**
+
+  Added new "PMD cyclecount" benchmark mode to dpdk-test-crypto-perf application
+  that displays more detailed breakdown of CPU cycles used by hardware
+  acceleration.
+
 
 Resolved Issues
 ---------------
doc/guides/tools/cryptoperf.rst +9 -1
index d2a6f82c..2f526c6 100644
@@ -50,7 +50,8 @@ offload are still consumed by the test tool and included in the cycle-count.
 These cycles are consumed by retries and inefficient API calls enqueuing and
 dequeuing smaller bursts than specified by the cmdline parameter. This results
 in a larger cycle-count measurement and should not be interpreted as an offload
-cost measurement.
+cost measurement. Using "pmd-cyclecount" mode will give a better idea of
+actual costs of hardware acceleration.
 
 On hardware devices the throughput measurement is not necessarily the maximum
 possible for the device, e.g. it may be necessary to use multiple cores to keep
@@ -134,6 +135,7 @@ The following are the appication command-line options:
            throughput
            latency
            verify
+           pmd-cyclecount
 
 * ``--silent``
 
@@ -329,6 +331,12 @@ The following are the appication command-line options:
 
         Set number of descriptors for each crypto device.
 
+* ``--pmd-cyclecount-delay-ms <n>``
+
+        Add a delay (in milliseconds) between enqueue and dequeue in
+        pmd-cyclecount benchmarking mode (useful when benchmarking
+        hardware acceleration).
+
 * ``--csv-friendly``
 
         Enable test result output CSV friendly rather than human friendly.
Signature · 1 line
2.7.4
#10 lore raw

PATCH v2 1/2 app/crypto-perf: add nb-desc parameter

In reply to Anatoly Burakov

This parameter makes number of cryptodev descriptors adjustable and defaults to earlier hardcoded default of 2048.

---
v2: squashed documentation into code patch
fixed commit message
fixed documentation

 app/test-crypto-perf/cperf_options.h         |  2 ++
 app/test-crypto-perf/cperf_options_parsing.c | 21 +++++++++++++++++++++
 app/test-crypto-perf/main.c                  |  2 +-
 doc/guides/tools/cryptoperf.rst              |  4 ++++
 4 files changed, 28 insertions(+), 1 deletion(-)
app/test-crypto-perf/cperf_options.h +2 -0
index 10cd2d8..edd6b79 100644
@@ -12,6 +12,7 @@
 #define CPERF_BURST_SIZE	("burst-sz")
 #define CPERF_BUFFER_SIZE	("buffer-sz")
 #define CPERF_SEGMENTS_NB	("segments-nb")
+#define CPERF_DESC_NB		("desc-nb")
 
 #define CPERF_DEVTYPE		("devtype")
 #define CPERF_OPTYPE		("optype")
@@ -68,6 +69,7 @@ struct cperf_options {
 	uint32_t total_ops;
 	uint32_t segments_nb;
 	uint32_t test_buffer_size;
+	uint32_t nb_descriptors;
 
 	uint32_t sessionless:1;
 	uint32_t out_of_place:1;
app/test-crypto-perf/cperf_options_parsing.c +21 -0
index 085aa8f..f4097d9 100644
@@ -340,6 +340,24 @@ parse_segments_nb(struct cperf_options *opts, const char *arg)
 }
 
 static int
+parse_desc_nb(struct cperf_options *opts, const char *arg)
+{
+	int ret = parse_uint32_t(&opts->nb_descriptors, arg);
+
+	if (ret) {
+		RTE_LOG(ERR, USER1, "failed to parse descriptors number\n");
+		return -1;
+	}
+
+	if (opts->nb_descriptors == 0) {
+		RTE_LOG(ERR, USER1, "invalid descriptors number specified\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+static int
 parse_device_type(struct cperf_options *opts, const char *arg)
 {
 	if (strlen(arg) > (sizeof(opts->device_type) - 1))
@@ -641,6 +659,7 @@ static struct option lgopts[] = {
 	{ CPERF_BURST_SIZE, required_argument, 0, 0 },
 	{ CPERF_BUFFER_SIZE, required_argument, 0, 0 },
 	{ CPERF_SEGMENTS_NB, required_argument, 0, 0 },
+	{ CPERF_DESC_NB, required_argument, 0, 0 },
 
 	{ CPERF_DEVTYPE, required_argument, 0, 0 },
 	{ CPERF_OPTYPE, required_argument, 0, 0 },
@@ -684,6 +703,7 @@ cperf_options_default(struct cperf_options *opts)
 
 	opts->pool_sz = 8192;
 	opts->total_ops = 10000000;
+	opts->nb_descriptors = 2048;
 
 	opts->buffer_size_list[0] = 64;
 	opts->buffer_size_count = 1;
@@ -740,6 +760,7 @@ cperf_opts_parse_long(int opt_idx, struct cperf_options *opts)
 		{ CPERF_BURST_SIZE,	parse_burst_sz },
 		{ CPERF_BUFFER_SIZE,	parse_buffer_sz },
 		{ CPERF_SEGMENTS_NB,	parse_segments_nb },
+		{ CPERF_DESC_NB,	parse_desc_nb },
 		{ CPERF_DEVTYPE,	parse_device_type },
 		{ CPERF_OPTYPE,		parse_op_type },
 		{ CPERF_SESSIONLESS,	parse_sessionless },
app/test-crypto-perf/main.c +1 -1
index 99f5d3e..7e6ca8e 100644
@@ -123,7 +123,7 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs,
 		};
 
 		struct rte_cryptodev_qp_conf qp_conf = {
-				.nb_descriptors = 2048
+			    .nb_descriptors = opts->nb_descriptors
 		};
 
 
doc/guides/tools/cryptoperf.rst +4 -0
index 457f817..d2a6f82c 100644
@@ -325,6 +325,10 @@ The following are the appication command-line options:
 
         Set the size of digest.
 
+* ``--desc-nb <n>``
+
+        Set number of descriptors for each crypto device.
+
 * ``--csv-friendly``
 
         Enable test result output CSV friendly rather than human friendly.
Signature · 1 line
2.7.4
#14 lore raw

v2 cover New crypto acceleration benchmark mode

In reply to Anatoly Burakov

This patchset adds a new "PMD cyclecount" test mode for test-crypto-perf application. This mode is intended to measure hardware acceleration cost (in terms of cycle count) more accurately than throughput test.

The general idea is the following:
- Measure build-alloc-free cycle separately
- Alloc and build ops
- Measure completely filling up the TX ring
- Wait until ops are processed
- Measure completely draining the RX ring
- Free all allocated ops

In order to make measurements more accurate, the enqueue/dequeue is still done in bursts of specified size, but all of the bursts are now part of a "superburst" of size equal to number of descriptors configured for the device. So, if the number of descriptors configured was 2048 (the default), then 2048 ops will be enqueued and dequeued, in bursts of size specified by test command line.

The following command-line switch will run the test:
--ptest=pmd-cyclecount

In addition to a new mode, two mode command line switches are added:
- --desc-nb - configure number of cryptodev descriptors. This value was previously hardcoded to 2048, but is now configurable and set to 2048 by default (so existing behavior is unchanged).
- --pmd-cyclecount-delay-ms - pmd-cyclecount-specific parameter that configures the delay (in milliseconds) between TX and RX superbursts, to allow hardware to process ops. Set to 0 by default, and it is expected that each user will tune it for every device. This has no effect on other benchmark modes.

PMD cyclecount mode can be used to benchmark software cryptodev drivers as well, but the results will be far less accurate for smaller burst sizes.

v2: squashed documentation patches into relevant commits
fixed documentation
fixed commit messages
formatting and naming fixes

Anatoly Burakov (2):
app/crypto-perf: add nb-desc parameter
app/crypto-perf: add new PMD benchmarking mode

 app/test-crypto-perf/Makefile                    |   1 +
 app/test-crypto-perf/cperf_options.h             |  10 +-
 app/test-crypto-perf/cperf_options_parsing.c     |  54 ++
 app/test-crypto-perf/cperf_test_pmd_cyclecount.c | 675 +++++++++++++++++++++++
 app/test-crypto-perf/cperf_test_pmd_cyclecount.h |  61 ++
 app/test-crypto-perf/main.c                      |  11 +-
 doc/guides/rel_notes/release_17_11.rst           |   6 +
 doc/guides/tools/cryptoperf.rst                  |  14 +-
 8 files changed, 828 insertions(+), 4 deletions(-)
 create mode 100644 app/test-crypto-perf/cperf_test_pmd_cyclecount.c
 create mode 100644 app/test-crypto-perf/cperf_test_pmd_cyclecount.h
Signature · 1 line
2.7.4
Reviewed-by on v2 1/2
#11 lore raw

Re: [PATCH v2 1/2] app/crypto-perf: add nb-desc parameter

In reply to Anatoly Burakov
Quoted 10 lines

-----Original Message-----
From: Burakov, Anatoly
Sent: Tuesday, September 12, 2017 10:36 AM
To: [email protected]
Cc: Doherty, Declan <[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: [PATCH v2 1/2] app/crypto-perf: add nb-desc parameter

Quoted 4 lines

This parameter makes number of cryptodev descriptors adjustable and defaults to earlier hardcoded default of 2048.

Reviewed-by on v2 2/2
#13 lore raw

Re: [PATCH v2 2/2] app/crypto-perf: add new PMD benchmarking mode

In reply to Anatoly Burakov
Quoted 35 lines

-----Original Message-----
From: Burakov, Anatoly
Sent: Tuesday, September 12, 2017 10:36 AM
To: [email protected]
Cc: Doherty, Declan <[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: [PATCH v2 2/2] app/crypto-perf: add new PMD benchmarking mode

This patch adds a new benchmarking mode, which is intended for microbenchmarking individual parts of the cryptodev framework, specifically crypto ops alloc-build-free, cryptodev PMD enqueue and cryptodev PMD dequeue.

It works by first benchmarking crypto operation alloc-build-free loop (no enqueues/dequeues happening), and then benchmarking enqueue and dequeue separately, by first completely filling up the TX queue, and then completely draining the RX queue.

Results are shown as cycle counts per alloc/build/free, PMD enqueue and PMD dequeue.

One new test mode is added: "pmd-cyclecount"
(called with --ptest=pmd-cyclecount)

New command-line argument is also added:
--pmd-cyclecount-delay-ms: this is a pmd-cyclecount-specific parameter
that controls the delay between enqueue and dequeue. This is useful for benchmarking hardware acceleration, as hardware may not be able to keep up with enqueued packets. This parameter can be increased if there are large amounts of dequeue retries.

Quoted 1 line
#15 lore raw
In reply to Anatoly Burakov
Quoted 68 lines

-----Original Message-----
From: Burakov, Anatoly
Sent: Tuesday, September 12, 2017 10:36 AM
To: [email protected]
Cc: Doherty, Declan <[email protected]>; De Lara Guarch, Pablo <[email protected]>
Subject: [PATCH v2 0/2] New crypto acceleration benchmark mode

This patchset adds a new "PMD cyclecount" test mode for test-crypto-perf application. This mode is intended to measure hardware acceleration cost (in terms of cycle count) more accurately than throughput test.

The general idea is the following:
- Measure build-alloc-free cycle separately
- Alloc and build ops
- Measure completely filling up the TX ring
- Wait until ops are processed
- Measure completely draining the RX ring
- Free all allocated ops

In order to make measurements more accurate, the enqueue/dequeue is still done in bursts of specified size, but all of the bursts are now part of a "superburst" of size equal to number of descriptors configured for the device. So, if the number of descriptors configured was 2048 (the default), then 2048 ops will be enqueued and dequeued, in bursts of size specified by test command line.

The following command-line switch will run the test:
--ptest=pmd-cyclecount

In addition to a new mode, two mode command line switches are added:
- --desc-nb - configure number of cryptodev descriptors. This value was previously hardcoded to 2048, but is now configurable and set to 2048 by default (so existing behavior is unchanged).
- --pmd-cyclecount-delay-ms - pmd-cyclecount-specific parameter that configures the delay (in milliseconds) between TX and RX superbursts, to allow hardware to process ops. Set to 0 by default, and it is expected that each user will tune it for every device. This has no effect on other benchmark modes.

PMD cyclecount mode can be used to benchmark software cryptodev drivers as well, but the results will be far less accurate for smaller burst sizes.

v2: squashed documentation patches into relevant commits
fixed documentation
fixed commit messages
formatting and naming fixes

Anatoly Burakov (2):
app/crypto-perf: add nb-desc parameter
app/crypto-perf: add new PMD benchmarking mode

 app/test-crypto-perf/Makefile                    |   1 +
 app/test-crypto-perf/cperf_options.h             |  10 +-
 app/test-crypto-perf/cperf_options_parsing.c     |  54 ++
 app/test-crypto-perf/cperf_test_pmd_cyclecount.c | 675
+++++++++++++++++++++++  app/test-crypto-
perf/cperf_test_pmd_cyclecount.h |  61 ++
 app/test-crypto-perf/main.c                      |  11 +-
 doc/guides/rel_notes/release_17_11.rst           |   6 +
 doc/guides/tools/cryptoperf.rst                  |  14 +-
 8 files changed, 828 insertions(+), 4 deletions(-)  create mode 100644
app/test-crypto-perf/cperf_test_pmd_cyclecount.c
 create mode 100644 app/test-crypto-perf/cperf_test_pmd_cyclecount.h
Quoted 2 lines

--
2.7.4

Applied to dpdk-next-crypto.

I think it will be a good idea to extend the documentation and explain more in detail the different modes that the application offers, before the release is out.

Thanks,

Pablo