app/dma-perf: introduce dma-perf application
Hi Anoob,
Replies are inline.
Thanks,
Cheng
Quoted 379 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 2:32 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationHi Cheng,
Please see inline.
Thanks,
AnoobQuoted 4 lines
-----Original Message-----
From: Jiang, Cheng1 <[email protected]>
Sent: Friday, June 16, 2023 8:26 AM
To: Anoob Joseph <[email protected]>Quoted 8 lines
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perfapplication
Quoted 15 lines
Hi Anoob,
Replies are inline.
Thanks,
ChengQuoted 7 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Thursday, June 15, 2023 11:48 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia,Chenbo
Quoted 14 lines
Quoted 5 lines
<[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perfapplication
Quoted 8 lines
Hi Cheng,
Please see inline.
Thanks,
AnoobQuoted 10 lines
Quoted 7 lines
Quoted 7 lines
-----Original Message-----
From: Jiang, Cheng1 <[email protected]>
Sent: Thursday, June 15, 2023 7:36 PM
To: Anoob Joseph <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia,Chenbo
Quoted 2 lines
Quoted 2 lines
<[email protected]>; Amit Prakash Shukla
<[email protected]>; Ma, WenwuXQuoted 19 lines
Quoted 19 lines
Quoted 3 lines
Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perfapplication
Quoted 15 lines
Hi Anoob,
Replies are inline.
Thanks,
ChengQuoted 7 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Thursday, June 15, 2023 4:45 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia,Quoted 22 lines
Quoted 3 lines
Chenbo
Quoted 2 lines
Quoted 2 lines
<[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX
Quoted 18 lines
Quoted 18 lines
Quoted 3 lines
Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perfapplication
Quoted 14 lines
Hi Cheng,
Please see inline.
Thanks,
AnoobQuoted 6 lines
-----Original Message-----
From: Jiang, Cheng1 <[email protected]>
Sent: Thursday, June 15, 2023 1:31 PM
To: Anoob Joseph <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, ChenboQuoted 7 lines
Quoted 7 lines
Quoted 4 lines
Amit
Quoted 3 lines
Quoted 3 lines
Prakash Shukla <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>;Wang,
Quoted 2 lines
Quoted 2 lines
YuanX
Quoted 1 line
<[email protected]>; He, Xingguang
Quoted 18 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Hi,
Thanks for your comments, the replies are inline.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Thanks,
ChengQuoted 4 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Thursday, June 15, 2023 1:22 PM
To: Jiang, Cheng1 <[email protected]>;Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Richardson, Bruce <[email protected]>;
[email protected]; Xia, ChenboQuoted 14 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Amit
Quoted 3 lines
Quoted 3 lines
Prakash Shukla <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>;Wang,
Quoted 2 lines
Quoted 2 lines
YuanX
Quoted 1 line
<[email protected]>; He, Xingguang
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Hi,
Thanks for working on the comments. Few more top level
comment
Quoted 23 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
inline.
Quoted 9 lines
Quoted 9 lines
Thanks,
AnoobQuoted 5 lines
-----Original Message-----
From: Cheng Jiang <[email protected]>
Sent: Tuesday, June 13, 2023 10:02 AM
To: [email protected]; [email protected]; [email protected]; [email protected]; AmitPrakash
Quoted 12 lines
Quoted 12 lines
Quoted 2 lines
Quoted 2 lines
Shukla
Quoted 1 line
Quoted 1 line
<[email protected]>; Anoob Joseph
Quoted 9 lines
Quoted 9 lines
Quoted 2 lines
Quoted 2 lines
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected];
Quoted 6 lines
Quoted 6 lines
Cheng Jiang <[email protected]>
Subject: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationExternal Email
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
------------------------------------------------------------------
--
-- There are many high-performance DMA devices supported inDPDK
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before
integrating
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
DMA into applications, developers need to know the
performance
Quoted 8 lines
Quoted 8 lines
of
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
these DMA devices in various scenarios and the performance of
CPUs
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios
automatically
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
Quoted 1 line
with a pre- set config file. Memory Copy performance test are
supported for now.
Quoted 25 lines
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>---
v6:
improved code based on Anoob's comments; fixed some code structure issues;
v5:
fixed someLONG_LINEwarnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removedworker_threadsparameter from config.ini; improved the logs;
improved config file;app/meson.build | 1 + app/test-dma-perf/benchmark.c | 477++++++++++++++++++++++++++++
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/config.ini | 59 ++++
app/test-dma-perf/main.c | 569 ++++++++++++++++++++++++++++++++++ app/test-dma-perf/main.h | 69 +++++ app/test-dma-perf/meson.build | 17 + 6 files changed, 1192 insertions(+) create mode 100644
app/test-dma-perf/benchmark.ccreate mode 100644 app/test-dma-perf/config.ini create mode 100644 app/test-dma-
Quoted 68 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
perf/main.ccreate mode 100644app/test-dma-perf/main.hcreate
Quoted 34 lines
Quoted 34 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
mode
100644 app/test-dma-perf/meson.buildapp/meson.build index renamed from app/meson.build +0
-0
74d2420f67..4fc1a83eba100644app/meson.build +1
-0@@ -19,6 +19,7 @@ apps = [ 'test-cmdline', 'test-compress-perf', 'test-crypto-perf', + 'test-dma-perf', 'test-eventdev', 'test-fib', 'test-flow-perf',app/test-dma- renamed from app/test-dma-perf/benchmark.c +0
-0Quoted 1 line
perf/benchmark.cnew file mode 100644 index0000000000..bc1ca82297
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
app/test-dma-perf/benchmark.c new file +13
-0@@ -0,0 +1,477 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Intel Corporation */ + +#include <inttypes.h> +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> + +#include <rte_time.h> +#include <rte_mbuf.h> +#include <rte_dmadev.h> +#include <rte_malloc.h> +#include <rte_lcore.h> Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c line 21
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+ +#include "main.h" + +#define MAX_DMA_CPL_NB 255 + +#define TEST_WAIT_U_SECOND 10000 + +#define CSV_LINE_DMA_FMT "Scenario%u,%u,%s,%u,%u,%.2lf,%"
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
PRIu64
Quoted 2 lines
Quoted 2 lines
",%.3lf,%.3lf\n"
+#defineCSV_LINE_CPU_FMT"Scenario%u,%u,NA,%u,%u,%.2lf,%"
Quoted 19 lines
Quoted 19 lines
Quoted 19 lines
Quoted 19 lines
PRIu64
Quoted 18 lines
Quoted 18 lines
",%.3lf,%.3lf\n"
app/test-dma-perf/benchmark.c new file +17
-0+ +struct worker_info { +bool ready_flag; +bool start_flag; +bool stop_flag; +uint32_t total_cpl; +uint32_t test_cpl; +}; + +struct lcore_params { +uint8_t scenario_id; +unsigned int lcore_id; +char *dma_name; +uint16_t worker_id; +uint16_t dev_id; +uint32_t nr_buf; +uint16_t kick_batch;Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+uint32_t buf_size; +uint16_t test_secs; +struct rte_mbuf **srcs; +struct rte_mbuf **dsts; +struct worker_info worker_info; +}; + +static struct rte_mempool *src_pool; static struct
rte_mempoolQuoted 43 lines
Quoted 32 lines
Quoted 32 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
app/test-dma-perf/benchmark.c
Quoted 3 lines
app/test-dma-perf/benchmark.c new file +3
-0+*dst_pool; + +static volatile struct lcore_params*worker_params[MAX_WORKER_NB];
Quoted 2 lines
app/test-dma-perf/benchmark.c
Quoted 2 lines
app/test-dma-perf/benchmark.c new file +2
-0+ +#define PRINT_ERR (...) print_err(__func__, __LINE__,
__VA_ARGS__)Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
app/test-dma-perf/benchmark.c new file +12
-0+ +static inline int +__rte_format_printf(3, 4) +print_err(const char *func, int lineno, const char *format, ...) { + va_list ap; + int ret; + + ret = fprintf(stderr, "In %s:%d - ", func, lineno); + va_start(ap, format); + ret += vfprintf(stderr, format, ap); + va_end(ap); +Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +6
-0+return ret; +} + +static inline void +calc_result(uint32_t buf_size, uint32_t nr_buf, uint16_t +nb_workers,
uint16_ttest_secs,app/test-dma-perf/benchmark.c new file +1
-0+ uint32_t total_cnt, float *memory,
uint32_tQuoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/benchmark.c
Quoted 3 lines
*ave_cycle,app/test-dma-perf/benchmark.c new file +2
-0+ float *bandwidth, float *mops) { + *memory = (float)(buf_size * (nr_buf / nb_workers) * 2) /(1024
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
+*
1024);app/test-dma-perf/benchmark.c new file +2
-0+ *ave_cycle = test_secs *rte_get_timer_hz () / total_cnt; + *bandwidth = (buf_size * 8 * (rte_get_timer_hz() /(float)*ave_cycle)) / 1000000000;
[Anoob] The above calculation may not yield actual results. '
ave_cycle'would
Quoted 3 lines
get converted to integer and then bandwidth would be allowed to report only very few values. Instead, we can do the calculation directly like,
Quoted 12 lines
Quoted 5 lines
*bandwidth = ((float)buf_size * 8 *
total_cnt/test_secs) /1000000000; *mops = (float)total_cnt / test_secs / 1000000;Same issue is there with below calculation as well. Please check.
[Cheng] Yes, I've noticed as well. Dengdui also mentioned this in his comments. I will address this issue in v7. Thank you very much.
Quoted 3 lines
Side note: in bandwidth calculation, shouldn't we be dividing by 1024*1024*1024? I've just carried the calculation that you used. Feel free
to
Quoted 10 lines
Quoted 1 line
correct as required.
[Cheng] The unit I'm using in my calculations is Gb/s (Gigabits per second), which is based on the decimal system. Therefore, I use the factor of 1000^3 (or 1,000,000,000).
The method you mentioned, dividing by 1024^3, is typically used when calculating GiB/s (Gibibits per second), a binary-based unit.
I think both methods are acceptable as long as the units and calculation methods correspond.
What do you think?
Quoted 1 line
[Anoob] You are right. Existing logic is correct. Thanks for the explanation.
[Cheng] sure, no problem. Feel free to contact me if you have any more questions.
Quoted 1,495 lines
Quoted 47 lines
Quoted 47 lines
Quoted 47 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
app/test-dma-perf/benchmark.c
Quoted 10 lines
app/test-dma-perf/benchmark.c new file +5
-0+ *mops = (float )rte_get_timer_hz () / *ave_cycle /1000000 ; } + +static void +output_result(uint8_t scenario_id, uint32_t lcore_id, char +*dma_name,
uint64_tave_cycle,app/test-dma-perf/benchmark.c new file +1
-0+ uint32_t buf_size, uint32_t nr_buf, floatmemory,
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
app/test-dma-perf/benchmark.c
Quoted 7 lines
app/test-dma-perf/benchmark.c new file +7
-0+ float bandwidth, float mops, bool is_dma) { + if (is_dma) + printf("lcore %u, DMA %s:\n", lcore_id, dma_name); + else + printf("lcore %u\n", lcore_id); + + printf("average cycles/op: %" PRIu64 ", buffer size: %u,
nr_buf:Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
app/test-dma-perf/benchmark.c
Quoted 5 lines
+%u,
memory: %.2lfMB, frequency: %" PRIu64 ".\n",app/test-dma-perf/benchmark.c new file +1
-0+ ave_cycle, buf_size, nr_buf, memory,
rte_get_timer_hz());app/test-dma-perf/benchmark.c new file +1
-0+ printf("Average bandwidth: %.3lfGbps, MOps: %.3lf\n",bandwidth,
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/benchmark.c
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+mops); + +if (is_dma) + snprintf(output_str[lcore_id],
MAX_OUTPUT_STR_LEN,Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/benchmark.c
Quoted 3 lines
CSV_LINE_DMA_FMT,app/test-dma-perf/benchmark.c new file +2
-0+ scenario_id, lcore_id, dma_name, buf_size, + nr_buf, memory, ave_cycle, bandwidth,mops);
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
app/test-dma-perf/benchmark.c
Quoted 2 lines
app/test-dma-perf/benchmark.c new file +2
-0+else + snprintf(output_str[lcore_id],
MAX_OUTPUT_STR_LEN,Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/benchmark.c
Quoted 3 lines
CSV_LINE_CPU_FMT,app/test-dma-perf/benchmark.c new file +2
-0+ scenario_id, lcore_id, buf_size, + nr_buf, memory, ave_cycle, bandwidth,mops); }
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
app/test-dma-perf/benchmark.c
Quoted 5 lines
app/test-dma-perf/benchmark.c new file +5
-0+ +static inline void +cache_flush_buf (__maybe_unused struct rte_mbuf **array, +__maybe_unused uint32_t buf_size, + __maybe_unused uint32_t nr_buf) { #ifdef
RTE_ARCH_X86_64Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
app/test-dma-perf/benchmark.c new file +7
-0+char *data; +struct rte_mbuf **srcs = array; +uint32_t i, offset; + +for (i =0 ; i < nr_buf; i++) { + data = rte_pktmbuf_mtod(srcs[i], char *); +for (offset =0 ; offset < buf_size; offset +=64 )Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+ __builtin_ia32_clflush(data + offset); + } +#endif +} + +/* Configuration of device. */ +static void +configure_dmadev_queue (uint32_t dev_id,uint32_t ring_size){
Quoted 191 lines
Quoted 69 lines
Quoted 69 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
app/test-dma-perf/benchmark.c
Quoted 10 lines
app/test-dma-perf/benchmark.c new file +10
-0+uint16_t vchan =0 ; +struct rte_dma_info info; +struct rte_dma_conf dev_config = { .nb_vchans =1 }; +struct rte_dma_vchan_conf qconf = { + .direction =RTE_DMA_DIR_MEM_TO_MEM , + .nb_desc = ring_size + }; + +if (rte_dma_configure (dev_id, &dev_config) !=0 ) + rte_exit(EXIT_FAILURE, "Error with dmaconfigure.\n");
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/benchmark.c
Quoted 3 lines
app/test-dma-perf/benchmark.c new file +3
-0+ +if (rte_dma_vchan_setup (dev_id, vchan, &qconf) !=0 ) + rte_exit(EXIT_FAILURE, "Error with queueconfiguration.\n");
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
app/test-dma-perf/benchmark.c new file +4
-0+ +rte_dma_info_get (dev_id, &info); +if (info.nb_vchans !=1 ) + rte_exit(EXIT_FAILURE, "Error, no configured queuesreported on device id. %u\n",
app/test-dma-perf/benchmark.c new file +10
-0+ dev_id); + +if (rte_dma_start (dev_id) !=0 ) +rte_exit (EXIT_FAILURE ,"Error with dma start.\n" ); } + +static int +config_dmadevs (struct test_configure *cfg) { +uint32_t ring_size = cfg->ring_size.cur; +struct lcore_dma_map_t *ldm = &cfg->lcore_dma_map; +uint32_t nb_workers = ldm->cnt;Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+uint32_t i; +int dev_id; +uint16_t nb_dmadevs =0 ; +char *dma_name; + +for (i =0 ; i < ldm->cnt; i++) { + dma_name = ldm->dma_names[i]; + dev_id =
rte_dma_get_dev_id_by_name(dma_name);Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
app/test-dma-perf/benchmark.c new file +2
-0+if (dev_id ==-1 ) { + fprintf(stderr, "Error: Fail to find DMA %s.\n",
dma_name);app/test-dma-perf/benchmark.c new file +3
-0+ goto end; + } +Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+ ldm->dma_ids[i] = dev_id; +configure_dmadev_queue (dev_id, ring_size); + ++nb_dmadevs; + } + +end: +if (nb_dmadevs < nb_workers) { + printf("Not enough dmadevs (%u) for all workers(%u).\n",
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
app/test-dma-perf/benchmark.c
Quoted 12 lines
nb_dmadevs,nb_workers);app/test-dma-perf/benchmark.c new file +11
-0+return -1 ; + } + +printf ("Number of used dmadevs: %u.\n" , nb_dmadevs); + +return 0 ; +} + +#define POLL_MAX 1000 + +[Anoob] Extra blank line. You can consider removing.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
[Cheng] sure, sorry for the miss.
Quoted 3 lines
app/test-dma-perf/benchmark.c
Quoted 3 lines
app/test-dma-perf/benchmark.c new file +2
-0+static inline void +do_dma_submit_and_poll(uint16_t dev_id, uint64_t
*async_cnt,Quoted 116 lines
Quoted 116 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+ volatile struct worker_info *worker_info) { +int ret; +uint16_t nr_cpl; +Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+ ret =rte_dma_submit (dev_id,0 ); +if (ret <0 ) { +rte_dma_stop (dev_id); +rte_dma_close (dev_id); +rte_exit (EXIT_FAILURE ,"Error with dma submit.\n" ); + } + + nr_cpl = rte_dma_completed(dev_id, 0,
MAX_DMA_CPL_NB, NULL,Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
app/test-dma-perf/benchmark.c
Quoted 9 lines
NULL);
app/test-dma-perf/benchmark.c new file +8
-0+ *async_cnt -= nr_cpl; + worker_info->total_cpl += nr_cpl; } + +static inline int +do_dma_mem_copy (void *p) +{ +const uint16_t *para_idx = (uint16_t *)p; + volatile struct lcore_params *para =worker_params[*para_idx];
Quoted 18 lines
Quoted 1 line
Quoted 1 line
app/test-dma-perf/benchmark.c
Quoted 1 line
app/test-dma-perf/benchmark.c new file +1
-0+ volatile struct worker_info *worker_info = &(para-
worker_info);Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
app/test-dma-perf/benchmark.c new file +16
-0+const uint16_t dev_id = para->dev_id; +const uint32_t nr_buf = para->nr_buf; +const uint16_t kick_batch = para->kick_batch; +const uint32_t buf_size = para->buf_size; +struct rte_mbuf **srcs = para->srcs; +struct rte_mbuf **dsts = para->dsts; +uint16_t nr_cpl; +uint64_t async_cnt =0 ; +uint32_t i; +uint32_t poll_cnt =0 ; +int ret; + + worker_info->stop_flag = false; + worker_info->ready_flag = true; + +while (!worker_info->start_flag)Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +6
-0+ ; + +while (1 ) { +for (i =0 ; i < nr_buf; i++) { +dma_copy: + ret = rte_dma_copy(dev_id, 0,
rte_pktmbuf_iova(srcs[i]),app/test-dma-perf/benchmark.c new file +1
-0+ rte_pktmbuf_iova(dsts[i]), buf_size,0);
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/benchmark.c
Quoted 3 lines
app/test-dma-perf/benchmark.c new file +3
-0+if (unlikely (ret <0 )) { +if (ret == -ENOSPC ) { +do_dma_submit_and_poll(dev_id,
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
app/test-dma-perf/benchmark.c
Quoted 6 lines
&async_cnt,worker_info);app/test-dma-perf/benchmark.c new file +5
-0+ goto dma_copy; + }else { +/* Error exit */ +rte_dma_stop (dev_id); + rte_exit(EXIT_FAILURE,"DMA
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
enqueue failed\n");
app/test-dma-perf/benchmark.c new file +6
-0+ } + } + async_cnt++; + +if ((async_cnt % kick_batch) ==0 ) + do_dma_submit_and_poll(dev_id,
&async_cnt,worker_info);app/test-dma-perf/benchmark.c new file +9
-0+ } + +if (worker_info->stop_flag) +break ; + } + +rte_dma_submit (dev_id,0 ); +while ((async_cnt >0 ) && (poll_cnt++ <POLL_MAX )) { + nr_cpl = rte_dma_completed(dev_id, 0,
MAX_DMA_CPL_NB, NULL, NULL);app/test-dma-perf/benchmark.c new file +3
-0+ async_cnt -= nr_cpl; + } +Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+return 0 ; +} + +static inline int +do_cpu_mem_copy (void *p) +{ +const uint16_t *para_idx = (uint16_t *)p; + volatile struct lcore_params *para =worker_params[*para_idx];
Quoted 25 lines
Quoted 1 line
Quoted 1 line
app/test-dma-perf/benchmark.c
Quoted 1 line
app/test-dma-perf/benchmark.c new file +1
-0+ volatile struct worker_info *worker_info = &(para-
worker_info);Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
app/test-dma-perf/benchmark.c new file +16
-0+const uint32_t nr_buf = para->nr_buf; +const uint32_t buf_size = para->buf_size; +struct rte_mbuf **srcs = para->srcs; +struct rte_mbuf **dsts = para->dsts; +uint32_t i; + + worker_info->stop_flag = false; + worker_info->ready_flag = true; + +while (!worker_info->start_flag) + ; + +while (1 ) { +for (i =0 ; i < nr_buf; i++) { +/* copy buffer form src to dst */ + rte_memcpy((void*)(uintptr_t)rte_mbuf_data_iova(dsts[i]),
app/test-dma-perf/benchmark.c new file +1
-0+ (void*)(uintptr_t)rte_mbuf_data_iova(srcs[i]),
app/test-dma-perf/benchmark.c new file +4
-0+ (size_t)buf_size); + worker_info->total_cpl++; + } +if (worker_info->stop_flag)Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+break ; + } + +return 0 ; +} + +static int +setup_memory_env(struct test_configure *cfg, struct
rte_mbufQuoted 58 lines
Quoted 58 lines
Quoted 58 lines
Quoted 11 lines
Quoted 11 lines
***srcs,
Quoted 10 lines
app/test-dma-perf/benchmark.c
Quoted 10 lines
app/test-dma-perf/benchmark.c new file +10
-0+ struct rte_mbuf ***dsts) +{ +unsigned int buf_size = cfg->buf_size.cur; +unsigned int nr_sockets; +uint32_t nr_buf = cfg->nr_buf; + + nr_sockets =rte_socket_count (); +if (cfg->src_numa_node >= nr_sockets || + cfg->dst_numa_node >= nr_sockets) { + printf("Error: Source or destination numa exceedsthe acture
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
app/test-dma-perf/benchmark.c
Quoted 5 lines
numa nodes.\n");
app/test-dma-perf/benchmark.c new file +4
-0+return -1 ; + } + + src_pool =rte_pktmbuf_pool_create("Benchmark_DMA_SRC",
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
app/test-dma-perf/benchmark.c
Quoted 7 lines
app/test-dma-perf/benchmark.c new file +7
-0+ nr_buf,/* n == num elements */ +64 ,/* cache size */ +0 ,/* priv size */ + buf_size +RTE_PKTMBUF_HEADROOM , + cfg->src_numa_node); +if (src_pool ==NULL ) { + PRINT_ERR("Error with source mempoolcreation.\n");
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/benchmark.c
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+return -1 ; + } + + dst_pool =rte_pktmbuf_pool_create("Benchmark_DMA_DST",
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
app/test-dma-perf/benchmark.c
Quoted 2 lines
app/test-dma-perf/benchmark.c new file +2
-0+ nr_buf,/* n == num elements */ + 64, /* cache size */[Anoob] We do not alloc or free pointers in the datapath, right? So why bother with cache?
[Cheng] Yes, you are right, the cache size is not necessary here, I'll fix it in the next version.
Quoted 1 line
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
app/test-dma-perf/benchmark.c
Quoted 5 lines
app/test-dma-perf/benchmark.c new file +5
-0+0 ,/* priv size */ + buf_size +RTE_PKTMBUF_HEADROOM , + cfg->dst_numa_node); +if (dst_pool ==NULL ) { + PRINT_ERR("Error with destination mempoolcreation.\n");
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/benchmark.c
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+return -1 ; + } + + *srcs = rte_malloc(NULL, nr_buf * sizeof(struct rte_mbuf *),0);
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
app/test-dma-perf/benchmark.c
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+if (*srcs ==NULL ) { +printf ("Error: srcs malloc failed.\n" ); +return -1 ; + }[Anoob] Are we freeing these memory? The ones allocated with
Quoted 10 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
rte_malloc.[Cheng] yes, we freed the memory in the end of
mem_copy_benchmark()Quoted 3 lines
Quoted 1 line
when we finished the test.
[Anoob] I think we are not freeing this mem. In the place where we
free
Quoted 2 lines
all
Quoted 1 line
Quoted 1 line
mem, we do free all objects to mempool as well as the mempools.
But
Quoted 4 lines
Quoted 4 lines
this
Quoted 3 lines
Quoted 3 lines
memory is to hold the pointers, right? Is that getting freed anywhere?
Also, in the mem clearing paths, do we need to clear the static
variables
Quoted 7 lines
Quoted 7 lines
(ie,
Quoted 1 line
Quoted 1 line
set srcs,
src_pool, dsts,dst_poolto NULL) so that there won't be anyscope
Quoted 4 lines
Quoted 2 lines
for any double free.
[Cheng] My apologies for the misunderstanding earlier. I now
understand
Quoted 10 lines
Quoted 1 line
Quoted 1 line
your point that you are right, the memory used to store the pointers is
not
Quoted 4 lines
Quoted 2 lines
being freed. I will fix this issue in the next version. Regarding the static variables you mentioned, I agree with your view that they should be
cleared.
Quoted 1 line
I will address this in the upcoming version as well. Thank you very much
for
Quoted 3 lines
Quoted 3 lines
the feedback. It is greatly appreciated.
In addition, I think we also need to nullify these variables when
initializing
Quoted 31 lines
Quoted 7 lines
Quoted 1 line
them to ensure safety and standardization of use. What do you think?
[Anoob] Since these are static variables, it is probably okay to skip the init part. But when we use it, we should clear it after use.
Please check above. I've posted one more comment. In case you missed.
[Cheng] sure, thanks for your advice, I'll clear it after use in the next version, thanks.
Quoted 20 lines
Quoted 20 lines
Thanks!
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/benchmark.c
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +2
-0+ + *dsts = rte_malloc(NULL, nr_buf * sizeof(struct rte_mbuf *),0);
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
app/test-dma-perf/benchmark.c
Quoted 6 lines
app/test-dma-perf/benchmark.c new file +6
-0+if (*dsts ==NULL ) { +printf ("Error: dsts malloc failed.\n" ); +return -1 ; + } + + if (rte_mempool_get_bulk(src_pool, (void **)*srcs, nr_buf)!= 0) {
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/benchmark.c
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+printf ("get src mbufs failed.\n" ); +return -1 ; + } + if (rte_mempool_get_bulk(dst_pool, (void **)*dsts, nr_buf)!= 0) {
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
app/test-dma-perf/benchmark.c
Quoted 9 lines
app/test-dma-perf/benchmark.c new file +9
-0+printf ("get dst mbufs failed.\n" ); +return -1 ; + } + +return 0 ; +} + +void +mem_copy_benchmark(struct test_configure *cfg, bool
is_dma)Quoted 158 lines
{
Quoted 143 lines
Quoted 143 lines
Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
app/test-dma-perf/benchmark.c new file +9
-0+uint16_t i; +uint32_t offset; +unsigned int lcore_id =0 ; +struct rte_mbuf **srcs =NULL , **dsts =NULL ; +struct lcore_dma_map_t *ldm = &cfg->lcore_dma_map; +unsigned int buf_size = cfg->buf_size.cur; +uint16_t kick_batch = cfg->kick_batch.cur; +uint32_t nr_buf = cfg->nr_buf = (cfg->mem_size.cur *1024 * +1024) /(
cfg->buf_size.cur* 2);app/test-dma-perf/benchmark.c new file +22
-0+uint16_t nb_workers = ldm->cnt; +uint16_t test_secs = cfg->test_secs; +float memory; +uint32_t avg_cycles =0 ; +float mops; +float bandwidth; + +if (setup_memory_env (cfg, &srcs, &dsts) <0 ) + goto out; + +if (is_dma) +if (config_dmadevs (cfg) <0 ) + goto out; + +if (cfg->cache_flush) { +cache_flush_buf (srcs, buf_size, nr_buf); +cache_flush_buf (dsts, buf_size, nr_buf); +rte_mb (); + } + +printf ("Start testing....\n" ); +Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +5
-0+for (i =0 ; i < nb_workers; i++) { + lcore_id = ldm->lcores[i]; + offset = nr_buf / nb_workers * i; + + worker_params[i] = rte_malloc(NULL, sizeof(struct
lcore_params), 0);app/test-dma-perf/benchmark.c new file +2
-0+if (!worker_params[i]) { + printf("lcore parameters malloc failure forlcore
Quoted 15 lines
Quoted 6 lines
Quoted 5 lines
app/test-dma-perf/benchmark.c
Quoted 3 lines
%d\n",
lcore_id);app/test-dma-perf/benchmark.c new file +2
-0+break ; + }[Anoob] Are we freeing the above memory?
Quoted 8 lines
[Cheng] sorry, I missed that, I'll add
worker_paramsmemory free in the next version, thanks.Quoted 5 lines
app/test-dma-perf/benchmark.c
Quoted 5 lines
app/test-dma-perf/benchmark.c new file +2
-0+if (is_dma) { + worker_params[i]->dma_name = ldm-Quoted 1 line
dma_names[i];
app/test-dma-perf/benchmark.c new file +1
-0+ worker_params[i]->dev_id = ldm-dma_ids[i];
Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
app/test-dma-perf/benchmark.c new file +4
-0+ worker_params[i]->kick_batch = kick_batch; + } + worker_params[i]->worker_id = i; + worker_params[i]->nr_buf = (uint32_t)(nr_buf /
nb_workers);app/test-dma-perf/benchmark.c new file +1
-0+ worker_params[i]->buf_size = buf_size;Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+ worker_params[i]->test_secs = test_secs; + worker_params[i]->srcs = srcs + offset; + worker_params[i]->dsts = dsts + offset; + worker_params[i]->scenario_id = cfg->scenario_id; + worker_params[i]->lcore_id = lcore_id; + +if (is_dma) +rte_eal_remote_launch(do_dma_mem_copy, (void
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/benchmark.c
Quoted 3 lines
*)(&i),
lcore_id);app/test-dma-perf/benchmark.c new file +2
-0+else + rte_eal_remote_launch(do_cpu_mem_copy,(void
Quoted 31 lines
Quoted 7 lines
Quoted 7 lines
app/test-dma-perf/benchmark.c
Quoted 7 lines
*)(&i),
lcore_id);app/test-dma-perf/benchmark.c new file +6
-0+ } + +while (1 ) { +bool ready = true; +for (i =0 ; i < nb_workers; i++) { + if (worker_params[i]-worker_info.ready_flag ==
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
false) {
app/test-dma-perf/benchmark.c new file +19
-0+ ready =0 ; +break ; + } + } +if (ready) +break ; + } + +for (i =0 ; i < nb_workers; i++) + worker_params[i]->worker_info.start_flag = true; + +usleep (TEST_WAIT_U_SECOND ); +for (i =0 ; i < nb_workers; i++) + worker_params[i]->worker_info.test_cpl = +worker_params[i]->worker_info.total_cpl; + +usleep (test_secs *1000 *1000 ); +for (i =0 ; i < nb_workers; i++) + worker_params[i]->worker_info.test_cpl =worker_params[i]->worker_info.total_cpl -
app/test-dma-perf/benchmark.c new file +1
-0+ worker_params[i]-Quoted 1 line
worker_info.test_cpl;
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
app/test-dma-perf/benchmark.c
Quoted 10 lines
app/test-dma-perf/benchmark.c new file +10
-0+ +for (i =0 ; i < nb_workers; i++) + worker_params[i]->worker_info.stop_flag = true; + +rte_eal_mp_wait_lcore (); + +for (i =0 ; i < nb_workers; i++) { + calc_result(buf_size, nr_buf, nb_workers, test_secs, + worker_params[i]->worker_info.test_cpl, + &memory, &avg_cycles, &bandwidth,&mops);
Quoted 6 lines
Quoted 1 line
Quoted 1 line
app/test-dma-perf/benchmark.c
Quoted 1 line
app/test-dma-perf/benchmark.c new file +1
-0+ output_result(cfg->scenario_id, worker_params[i]-
lcore_id,Quoted 1 line
Quoted 1 line
app/test-dma-perf/benchmark.c
Quoted 1 line
app/test-dma-perf/benchmark.c new file +1
-0+ worker_params[i]-
dma_name,Quoted 2 lines
Quoted 2 lines
app/test-dma-perf/benchmark.c
Quoted 2 lines
avg_cycles,buf_size,app/test-dma-perf/benchmark.c new file +1
-0+ nr_buf / nb_workers,memory,
Quoted 19 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
bandwidth, mops,
is_dma);app/test-dma-perf/benchmark.c new file +9
-0+ } + +out: +/* free env */ +if (srcs) +rte_pktmbuf_free_bulk (srcs, nr_buf); +if (dsts) +rte_pktmbuf_free_bulk (dsts, nr_buf); +Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/benchmark.c
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+if (src_pool) +rte_mempool_free (src_pool); +if (dst_pool) +rte_mempool_free (dst_pool); + +if (is_dma) { +for (i =0 ; i < nb_workers; i++) { + printf("Stopping dmadev %d\n", ldm-dma_ids[i]);
Quoted 13 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
app/test-dma-perf/benchmark.c new file +4
-0+rte_dma_stop (ldm->dma_ids[i]); + } + } +}(quoted diff) +0
-0Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/config.ini line 3
Quoted 8 lines
b/app/test-dma-perf/config.ini new file mode 100644 index 0000000000..2fd9c3c387
app/test-dma-perf/config.ini new file +3
-0@@ -0,0 +1,59 @@ + +; This is an example configuration file for dma-perf, which +details the meanings of each parameter ; and instructions onhow
Quoted 10 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
app/test-dma-perf/config.ini line 4
Quoted 1 line
app/test-dma-perf/config.ini new file +1
-0+to use dma-perf.
Quoted 2 lines
Quoted 2 lines
app/test-dma-perf/config.ini new file +2
-0+ +; Supported test types are DMA_MEM_COPY and
CPU_MEM_COPY.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
app/test-dma-perf/config.ini new file +5
-0+ +; Parameters: +; "mem_size" denotes the size of the memory footprint. +; "buf_size" denotes the memory size of a single operation. +; "dma_ring_size" denotes the dma ring buffer size. It shouldbe
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/config.ini new file +1
-0+greaterthan 64 normally.
+; "kick_batch" denotes the dma operation batch size, andshould
Quoted 69 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/config.ini
Quoted 4 lines
+be greater
than 1 normally.app/test-dma-perf/config.ini new file +2
-0+ +; The format for variables isvariable=first,last,increment,ADD|MUL.
Quoted 11 lines
Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
app/test-dma-perf/config.ini
Quoted 2 lines
app/test-dma-perf/config.ini new file +2
-0+ +; src_numa_node is used to control the numa node where thesource
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
app/test-dma-perf/config.ini
Quoted 7 lines
memory is allocated.
app/test-dma-perf/config.ini new file +2
-0+; dst_numa_node is used to control the numa node where the +destinationmemory is allocated.
app/test-dma-perf/config.ini new file +3
-0+ +; cache_flush is used to determine whether or not the cache +should be flushed, with 1 indicating to ; flush and 0 indicating tonot
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
flush.
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/config.ini
Quoted 4 lines
app/test-dma-perf/config.ini new file +4
-0+ +; test_seconds controls the test time of the whole case. + +; To use DMA for a test, please specify the "lcore_dma"parameter.
Quoted 37 lines
Quoted 37 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/config.ini
Quoted 8 lines
app/test-dma-perf/config.ini new file +3
-0+; If you have already set the "-l" and "-a" parameters using EAL, +; make sure that the value of "lcore_dma" falls within their +range of thevalues.
app/test-dma-perf/config.ini new file +4
-0+ +; To use CPU for a test, please specify the "lcore" parameter. +; If you have already set the "-l" and "-a" parameters using EAL, +; make sure that the value of "lcore" falls within their range ofvalues.
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 14 lines
app/test-dma-perf/config.ini new file +3
-0+ +; To specify a configuration file, use the "--config" flag +followed by the pathto the file.
app/test-dma-perf/config.ini new file +3
-0+ +; To specify a result file, use the "--result" flag followed by +the path to thefile.
app/test-dma-perf/config.ini new file +3
-0+; If you do not specify a result file, one will be generated with +the same name as the configuration ; file, with the addition of +"_result.csv" atthe end.
app/test-dma-perf/config.ini new file +2
-0+ +[case1]app/test-dma-perf/config.ini
Quoted 10 lines
app/test-dma-perf/config.ini new file +10
-0+type=DMA_MEM_COPY +mem_size=10 +buf_size=64,8192,2,MUL +dma_ring_size=1024 +kick_batch=32 +src_numa_node=0 +dst_numa_node=0 +cache_flush=0 +test_seconds=2 +lcore_dma=lcore10@0000:00:04.2, lcore11@0000:00:04.3[Anoob] Isn't it better if we allow user to specify DMA dev ID rather than the PCI DBDF?
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
In the long run, I would expect config file to provide {core,
dma_dev_id,queue_id}Another thought is why to expose this at all? If we can restrict this perf application to have one thread only use one vchan, then application can easily create this mapping in run time. Unless you want one thread to use 2 different vchans which may not be
desirable
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
since this is a standalone perf app.
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
[Cheng] Thank you for the feedback.
Here are my thoughts:
Firstly, the user may not know which device the DMA dev IDcorresponds
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
to, or which NUMA node it is on. In my example, I used the CBDMA environment, so I did not specify the work queue ID. When using
DSA,
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
the configuration would be something like lcore10@0000:00:04.2-q0 which contains core, dma and work queue id. The reason for
exposing
Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
these options is that we want the user to fully understand which
cores
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
and devices are being used so that they know exactly where the performance data is coming from. For example, performance when
cores
Quoted 1 line
Quoted 1 line
Quoted 1 line
and DMA devices are not on the same NUMA node, etc. This allows
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
testing scenario to be precise and flexible. If the application handles the mapping itself, the user loses control over the mapping and may not get the performance data they want. We believe
control
Quoted 181 lines
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines
Quoted 1 line
should be given to the user rather than the application.
[Anoob] I understand your view points. Thanks for the explanation.
[Cheng] sure, no problem.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 13 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
app/test-dma-perf/config.ini new file +5
-0+eal_args=--in-memory --file-prefix=test + +[case2] +type=CPU_MEM_COPY +mem_size=10Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-
Quoted 8 lines
app/test-dma-perf/config.ini new file +7
-0+buf_size=64,8192,2,MUL +src_numa_node=0 +dst_numa_node=1 +cache_flush=0 +test_seconds=2 +lcore = 3, 4 +eal_args=--in-memory --no-pciapp/test-dma- renamed from app/test-dma-perf/main.c +0
-0
perf/main.cQuoted 80 lines
Quoted 80 lines
Quoted 71 lines
Quoted 71 lines
Quoted 71 lines
Quoted 71 lines
new file mode 100644 index 0000000000..d65655b87b
app/test-dma-perf/main.c new file +21
-0@@ -0,0 +1,569 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Intel Corporation */ + +#include <stdio.h> +#include <stdlib.h> +#include <getopt.h> +#include <signal.h> +#include <stdbool.h> +#include <unistd.h> +#include <sys/wait.h> +#include <inttypes.h> +#include <libgen.h> + +#include <rte_eal.h> +#include <rte_cfgfile.h> +#include <rte_string_fns.h> +#include <rte_lcore.h> + +#include "main.h" + +#define CSV_HDR_FMT "Case %u : %s,lcore,DMA,buffersize,nr_buf,memory(MB),cycle,bandwidth(Gbps),MOps\n"
app/test-dma-perf/main.c new file +45
-0+ +#define MAX_EAL_PARAM_NB 100 +#define MAX_EAL_PARAM_LEN 1024 + +#define DMA_MEM_COPY "DMA_MEM_COPY" +#define CPU_MEM_COPY "CPU_MEM_COPY" + +#define CMDLINE_CONFIG_ARG "--config" +#define CMDLINE_RESULT_ARG "--result" + +#define MAX_PARAMS_PER_ENTRY 4 + +#define MAX_LONG_OPT_SZ 64 + +enum { +TEST_TYPE_NONE =0 , +TEST_TYPE_DMA_MEM_COPY , +TEST_TYPE_CPU_MEM_COPY +}; + +#define MAX_TEST_CASES 16 +static struct test_configure test_cases[MAX_TEST_CASES ]; + +char output_str[MAX_WORKER_NB ][MAX_OUTPUT_STR_LEN ]; + +static FILE *fd; + +static void +output_csv (bool need_blankline) +{ +uint32_t i; + +if (need_blankline) { +fprintf (fd,",,,,,,,,\n" ); +fprintf (fd,",,,,,,,,\n" ); + } + +for (i =0 ; i <RTE_DIM (output_str); i++) { +if (output_str[i][0 ]) { +fprintf (fd,"%s" , output_str[i]); + output_str[i][0 ] ='\0' ; + } + } + +fflush (fd);Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +6
-0+} + +static void +output_env_info (void ) +{ + snprintf(output_str[0], MAX_OUTPUT_STR_LEN, "testenvironment:\n");
app/test-dma-perf/main.c new file +1
-0+ snprintf(output_str[1], MAX_OUTPUT_STR_LEN, "CPUfrequency,%"
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
app/test-dma-perf/main.c
Quoted 7 lines
app/test-dma-perf/main.c new file +7
-0+ PRIu64 "\n", rte_get_timer_hz()); + +output_csv (true); +} + +static void +output_header(uint32_t case_id, struct test_configure
*case_cfg) {Quoted 70 lines
Quoted 70 lines
Quoted 70 lines
Quoted 2 lines
Quoted 2 lines
app/test-dma-perf/main.c
Quoted 2 lines
app/test-dma-perf/main.c new file +2
-0+ snprintf(output_str[0], MAX_OUTPUT_STR_LEN, + CSV_HDR_FMT, case_id, case_cfg-
test_type_str);Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
app/test-dma-perf/main.c new file +7
-0+ +output_csv (true); +} + +static void +run_test_case (struct test_configure *case_cfg) { +switch (case_cfg->test_type) {Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +8
-0+case TEST_TYPE_DMA_MEM_COPY : +mem_copy_benchmark (case_cfg, true); +break ; +case TEST_TYPE_CPU_MEM_COPY : +mem_copy_benchmark (case_cfg, false); +break ; +default : + printf("Unknown test type. %s\n", case_cfg-
test_type_str);Quoted 9 lines
Quoted 9 lines
app/test-dma-perf/main.c
Quoted 9 lines
app/test-dma-perf/main.c new file +9
-0+break ; + } +} + +static void +run_test (uint32_t case_id,struct test_configure *case_cfg) { +uint32_t i; +uint32_t nb_lcores =rte_lcore_count (); + struct test_configure_entry *mem_size = &case_cfg-
mem_size;Quoted 1 line
Quoted 1 line
app/test-dma-perf/main.c
Quoted 1 line
app/test-dma-perf/main.c new file +1
-0+ struct test_configure_entry *buf_size = &case_cfg-
buf_size;Quoted 1 line
Quoted 1 line
app/test-dma-perf/main.c
Quoted 1 line
app/test-dma-perf/main.c new file +1
-0+ struct test_configure_entry *ring_size = &case_cfg-
ring_size;Quoted 1 line
Quoted 1 line
app/test-dma-perf/main.c
Quoted 1 line
app/test-dma-perf/main.c new file +1
-0+ struct test_configure_entry *kick_batch = &case_cfg-
kick_batch;Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
app/test-dma-perf/main.c new file +31
-0+struct test_configure_entry dummy = {0 }; +struct test_configure_entry *var_entry = &dummy; + +for (i =0 ; i <RTE_DIM (output_str); i++) +memset (output_str[i],0 ,MAX_OUTPUT_STR_LEN ); + +if (nb_lcores <= case_cfg->lcore_dma_map.cnt) { +printf ("Case %u: Not enough lcores.\n" , case_id); +return ; + } + +printf ("Number of used lcores: %u.\n" , nb_lcores); + +if (mem_size->incr !=0 ) + var_entry = mem_size; + +if (buf_size->incr !=0 ) + var_entry = buf_size; + +if (ring_size->incr !=0 ) + var_entry = ring_size; + +if (kick_batch->incr !=0 ) + var_entry = kick_batch; + + case_cfg->scenario_id =0 ; + +output_header (case_id, case_cfg); + + for (var_entry->cur = var_entry->first; var_entry->cur <= +var_entry-Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/main.c
Quoted 3 lines
Quoted 1 line
last;) {
app/test-dma-perf/main.c new file +2
-0+ case_cfg->scenario_id++; + printf("\nRunning scenario %d\n", case_cfg-
scenario_id);Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
app/test-dma-perf/main.c new file +7
-0+ +run_test_case (case_cfg); +output_csv (false); + +if (var_entry->op ==OP_ADD ) + var_entry->cur += var_entry->incr; +else if (var_entry->op ==OP_MUL )Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +8
-0+ var_entry->cur *= var_entry->incr; + else + break; + } +} + +static int +parse_lcore(struct test_configure *test_case, const char*value)
Quoted 414 lines
{
Quoted 85 lines
Quoted 85 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
app/test-dma-perf/main.c
Quoted 10 lines
app/test-dma-perf/main.c new file +4
-0+size_t len =strlen (value); +char *input = (char *)malloc ((len +1 ) *sizeof (char )); +strcpy (input, value); + struct lcore_dma_map_t *lcore_dma_map = &(test_case-Quoted 1 line
lcore_dma_map);app/test-dma-perf/main.c new file +5
-0+ +if (test_case ==NULL || value ==NULL ) +return -1 ; + + memset(lcore_dma_map, 0, sizeof(struct
lcore_dma_map_t));Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
app/test-dma-perf/main.c
Quoted 10 lines
app/test-dma-perf/main.c new file +10
-0+ +char *token =strtok (input,", " ); +while (token !=NULL ) { +if (lcore_dma_map->cnt >=MAX_LCORE_NB ) { +free (input); +return -1 ; + } + +uint16_t lcore_id =atoi (token); + lcore_dma_map->lcores[lcore_dma_map->cnt++] =
lcore_id;Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
app/test-dma-perf/main.c
Quoted 10 lines
app/test-dma-perf/main.c new file +10
-0+ + token =strtok (NULL ,", " ); + } + +free (input); +return 0 ; +} + +static int +parse_lcore_dma(struct test_configure *test_case, const char*value)
Quoted 40 lines
{
Quoted 39 lines
Quoted 39 lines
Quoted 39 lines
app/test-dma-perf/main.c new file +24
-0+struct lcore_dma_map_t *lcore_dma_map; +char *input =strndup (value,strlen (value) +1 ); +char *addrs = input; +char *ptrs[2 ]; +char *start, *end, *substr; +uint16_t lcore_id; +int ret =0 ; + +while (*addrs =='\0' ) + addrs++; +if (*addrs =='\0' ) { +fprintf (stderr,"No input DMA addresses\n" ); + ret =-1 ; + goto out; + } + + substr =strtok (addrs,"," ); +if (substr ==NULL ) { +fprintf (stderr,"No input DMA address\n" ); + ret =-1 ; + goto out; + } + + memset(&test_case->lcore_dma_map, 0, sizeof(struct
lcore_dma_map_t));app/test-dma-perf/main.c new file +14
-0+ + do { + rte_strsplit(substr, strlen(substr), ptrs, 2, '@'); + + start = strstr(ptrs[0], "lcore"); + if (start == NULL) { + fprintf(stderr, "Illegal lcore\n"); + ret = -1; + break; + } + + start += 5; + lcore_id = strtol(start, &end, 0); + if (end == start) {Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +1
-0+ fprintf(stderr, "No input lcore ID or ID %d iswrong\n",
lcore_id);app/test-dma-perf/main.c new file +6
-0+ ret =-1 ; +break ; + } + + lcore_dma_map = &test_case->lcore_dma_map; + lcore_dma_map->lcores[lcore_dma_map->cnt] =
lcore_id;Quoted 3 lines
Quoted 1 line
Quoted 1 line
app/test-dma-perf/main.c
Quoted 1 line
app/test-dma-perf/main.c new file +1
-0+ strcpy(lcore_dma_map-dma_names[lcore_dma_map-
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
cnt], ptrs[1]);
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
app/test-dma-perf/main.c
Quoted 11 lines
app/test-dma-perf/main.c new file +11
-0+ lcore_dma_map->cnt++; + substr =strtok (NULL ,"," ); + }while (substr !=NULL ); + +out: +free (input); +return ret; +} + +static int +parse_entry(const char *value, struct test_configure_entry*entry)
Quoted 316 lines
Quoted 316 lines
{
Quoted 54 lines
Quoted 54 lines
Quoted 54 lines
Quoted 54 lines
app/test-dma-perf/main.c new file +12
-0+char input[255 ] = {0 }; +char *args[MAX_PARAMS_PER_ENTRY ]; +int args_nr =-1 ; + +if (value ==NULL || entry ==NULL ) + goto out; + +strncpy (input, value,254 ); +if (*input =='\0' ) + goto out; + + args_nr = rte_strsplit(input, strlen(input), args,
MAX_PARAMS_PER_ENTRY, ',');app/test-dma-perf/main.c new file +35
-0+if (args_nr !=1 && args_nr !=4 ) + goto out; + + entry->cur = entry->first = (uint32_t )atoi (args[0 ]); + +if (args_nr ==4 ) { + entry->last = (uint32_t )atoi (args[1 ]); + entry->incr = (uint32_t )atoi (args[2 ]); +if (!strcmp (args[3 ],"MUL" )) + entry->op =OP_MUL ; +else if (!strcmp (args[3 ],"ADD" )) + entry->op =OP_ADD ; +else { +printf ("Invalid op %s.\n" , args[3 ]); + args_nr =-1 ; + } + }else { + entry->op =OP_NONE ; + entry->last =0 ; + entry->incr =0 ; + } +out: +return args_nr; +} + +static uint16_t +load_configs (const char *path) +{ +struct rte_cfgfile *cfgfile; +int nb_sections, i; +struct test_configure *test_case; +char section_name[CFG_NAME_LEN ]; +const char *case_type; +const char *lcore_dma; + const char *mem_size_str, *buf_size_str, *ring_size_str,
*kick_batch_str;app/test-dma-perf/main.c new file +5
-0+int args_nr, nb_vp; +bool is_dma; + +printf ("config file parsing...\n" ); + cfgfile =rte_cfgfile_load (path,0 );Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +8
-0+if (!cfgfile) { +printf ("Open configure file error.\n" ); +exit (1 ); + } + + nb_sections =rte_cfgfile_num_sections (cfgfile,NULL ,0 ); +if (nb_sections >MAX_TEST_CASES ) { + printf("Error: The maximum number of cases is%d.\n",
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
app/test-dma-perf/main.c
Quoted 6 lines
MAX_TEST_CASES);app/test-dma-perf/main.c new file +5
-0+exit (1 ); + } + +for (i =0 ; i < nb_sections; i++) { + snprintf(section_name, CFG_NAME_LEN, "case%d", i+ 1);
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
app/test-dma-perf/main.c
Quoted 2 lines
app/test-dma-perf/main.c new file +2
-0+ test_case = &test_cases[i]; + case_type = rte_cfgfile_get_entry(cfgfile,
section_name,Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/main.c
Quoted 3 lines
"type");
app/test-dma-perf/main.c new file +2
-0+if (!case_type) { + printf("Error: No case type in case %d, thetest will be
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
app/test-dma-perf/main.c
Quoted 10 lines
finished here.\n",
app/test-dma-perf/main.c new file +7
-0+ i + 1); + test_case->is_valid = false; +continue ; + } + +if (strcmp (case_type,DMA_MEM_COPY ) ==0 ) { + test_case->test_type =
TEST_TYPE_DMA_MEM_COPY;app/test-dma-perf/main.c new file +1
-0+ test_case->test_type_str =
DMA_MEM_COPY;Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
app/test-dma-perf/main.c
Quoted 5 lines
app/test-dma-perf/main.c new file +3
-0+ is_dma = true; + }else if (strcmp (case_type,CPU_MEM_COPY ) ==0 ) { + test_case->test_type =
TEST_TYPE_CPU_MEM_COPY;app/test-dma-perf/main.c new file +1
-0+ test_case->test_type_str =
CPU_MEM_COPY;Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/main.c
Quoted 3 lines
app/test-dma-perf/main.c new file +3
-0+ is_dma = false; + }else { + printf("Error: Cannot find case type %s incase%d.\n",
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
case_type, i + 1);app/test-dma-perf/main.c new file +7
-0+ test_case->is_valid = false; +continue ; + } + + nb_vp =0 ; + + test_case->src_numa_node =(int)atoi(rte_cfgfile_get_entry(cfgfile,
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
+
section_name, "src_numa_node"));app/test-dma-perf/main.c new file +1
-0+ test_case->dst_numa_node =(int)atoi(rte_cfgfile_get_entry(cfgfile,
+section_name, "dst_numa_node"));app/test-dma-perf/main.c new file +2
-0+ + mem_size_str = rte_cfgfile_get_entry(cfgfile,
section_name,Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
app/test-dma-perf/main.c
Quoted 11 lines
"
mem_size");app/test-dma-perf/main.c new file +1
-0+ args_nr = parse_entry(mem_size_str, &test_case-Quoted 1 line
mem_size);app/test-dma-perf/main.c new file +8
-0+if (args_nr <0 ) { +printf ("parse error in case %d.\n" , i +1 ); + test_case->is_valid = false; +continue ; + }else if (args_nr >1 ) + nb_vp++; + + buf_size_str = rte_cfgfile_get_entry(cfgfile,
section_name,Quoted 16 lines
Quoted 2 lines
Quoted 2 lines
app/test-dma-perf/main.c
Quoted 2 lines
"
buf_size");app/test-dma-perf/main.c new file +1
-0+ args_nr = parse_entry(buf_size_str, &test_case-
buf_size);Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
app/test-dma-perf/main.c new file +5
-0+if (args_nr <0 ) { +printf ("parse error in case %d.\n" , i +1 ); + test_case->is_valid = false; +continue ; + }else if (args_nr >1 )Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +4
-0+ nb_vp++; + +if (is_dma) { + ring_size_str = rte_cfgfile_get_entry(cfgfile,
section_name,
+"dma_ring_size");app/test-dma-perf/main.c new file +1
-0+ args_nr = parse_entry(ring_size_str,&test_case-
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/main.c
Quoted 3 lines
Quoted 1 line
ring_size);app/test-dma-perf/main.c new file +2
-0+if (args_nr <0 ) { + printf("parse error in case %d.\n", i +1);
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
app/test-dma-perf/main.c
Quoted 6 lines
app/test-dma-perf/main.c new file +6
-0+ test_case->is_valid = false; +continue ; + }else if (args_nr >1 ) + nb_vp++; + + kick_batch_str =rte_cfgfile_get_entry(cfgfile,
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
app/test-dma-perf/main.c
Quoted 2 lines
section_name, "kick_batch");app/test-dma-perf/main.c new file +1
-0+ args_nr = parse_entry(kick_batch_str,&test_case-
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/main.c
Quoted 3 lines
Quoted 1 line
kick_batch);app/test-dma-perf/main.c new file +2
-0+if (args_nr <0 ) { + printf("parse error in case %d.\n", i +1);
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
app/test-dma-perf/main.c
Quoted 11 lines
app/test-dma-perf/main.c new file +6
-0+ test_case->is_valid = false; +continue ; + }else if (args_nr >1 ) + nb_vp++; + + lcore_dma = rte_cfgfile_get_entry(cfgfile,
section_name, "lcore_dma");app/test-dma-perf/main.c new file +1
-0+ int lcore_ret = parse_lcore_dma(test_case,
lcore_dma);app/test-dma-perf/main.c new file +2
-0+if (lcore_ret <0 ) { + printf("parse lcore dma error in case%d.\n", i
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
1);
app/test-dma-perf/main.c
Quoted 7 lines
app/test-dma-perf/main.c new file +5
-0+ test_case->is_valid = false; +continue ; + } + }else { + lcore_dma = rte_cfgfile_get_entry(cfgfile,
section_name, "lcore");app/test-dma-perf/main.c new file +1
-0+ int lcore_ret = parse_lcore(test_case,
lcore_dma);Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
app/test-dma-perf/main.c
Quoted 2 lines
app/test-dma-perf/main.c new file +2
-0+if (lcore_ret <0 ) { + printf("parse lcore error in case%d.\n", i + 1);
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
app/test-dma-perf/main.c
Quoted 7 lines
app/test-dma-perf/main.c new file +7
-0+ test_case->is_valid = false; +continue ; + } + } + +if (nb_vp >1 ) { + printf("Error, each section can only have asingle
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
app/test-dma-perf/main.c
Quoted 7 lines
variable parameter.\n");
app/test-dma-perf/main.c new file +6
-0+ test_case->is_valid = false; +continue ; + } + + test_case->cache_flush = + (int)atoi(rte_cfgfile_get_entry(cfgfile,
section_name,Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/main.c
Quoted 4 lines
"
cache_flush"));app/test-dma-perf/main.c new file +1
-0+ test_case->test_secs =(uint16_t)atoi(rte_cfgfile_get_entry(cfgfile,
app/test-dma-perf/main.c new file +1
-0+ section_name,"
test_seconds"));Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
app/test-dma-perf/main.c new file +2
-0+ + test_case->eal_args = rte_cfgfile_get_entry(cfgfile,
section_name, "eal_args");app/test-dma-perf/main.c new file +11
-0+ test_case->is_valid = true; + } + +rte_cfgfile_close (cfgfile); +printf ("config file parsing complete.\n\n" ); +return i; +} + +/* Parse the argument given in the command line of the +application */ static int append_eal_args (int argc,char **argv, +const char *eal_args,char **new_argv) {Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +8
-0+int i; +char *tokens[MAX_EAL_PARAM_NB ]; +char args[MAX_EAL_PARAM_LEN ] = {0 }; +int token_nb, new_argc =0 ; + +for (i =0 ; i < argc; i++) { + if ((strcmp(argv[i], CMDLINE_CONFIG_ARG) == 0) || + (strcmp(argv[i],
CMDLINE_RESULT_ARG) ==Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
0)) {
app/test-dma-perf/main.c new file +4
-0+ i++; +continue ; + } + strlcpy(new_argv[new_argc], argv[i],Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
sizeof(new_argv[new_argc]));app/test-dma-perf/main.c new file +7
-0+ new_argc++; + } + +if (eal_args) { +strlcpy (args, eal_args,sizeof (args)); + token_nb = rte_strsplit(args, strlen(args), + tokens,
MAX_EAL_PARAM_NB, ' ');Quoted 35 lines
Quoted 35 lines
Quoted 35 lines
Quoted 35 lines
app/test-dma-perf/main.c new file +30
-0+for (i =0 ; i < token_nb; i++) +strcpy (new_argv[new_argc++], tokens[i]); + } + +return new_argc; +} + +int +main (int argc,char *argv[]) +{ +int ret; +uint16_t case_nb; +uint32_t i, nb_lcores; +pid_t cpid, wpid; +int wstatus; +char args[MAX_EAL_PARAM_NB ][MAX_EAL_PARAM_LEN ]; +char *pargs[MAX_EAL_PARAM_NB ]; +char *cfg_path_ptr =NULL ; +char *rst_path_ptr =NULL ; +char rst_path[PATH_MAX ]; +int new_argc; +bool is_first_case = true; + +memset (args,0 ,sizeof (args)); + +for (i =0 ; i <RTE_DIM (pargs); i++) + pargs[i] = args[i]; + +for (i =0 ; i < (uint32_t )argc; i++) { + if (strncmp(argv[i], CMDLINE_CONFIG_ARG,
MAX_LONG_OPT_SZ) == 0)app/test-dma-perf/main.c new file +2
-0+ cfg_path_ptr = argv[i +1 ]; + if (strncmp(argv[i], CMDLINE_RESULT_ARG,
MAX_LONG_OPT_SZ) == 0)app/test-dma-perf/main.c new file +1
-0+ rst_path_ptr = argv[i +1 ];Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +8
-0+ } +if (cfg_path_ptr ==NULL ) { +printf ("Config file not assigned.\n" ); +return -1 ; + } +if (rst_path_ptr ==NULL ) { +strlcpy (rst_path, cfg_path_ptr,PATH_MAX ); + strcat(strtok(basename(rst_path), "."),"_result.csv");
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
app/test-dma-perf/main.c new file +6
-0+ rst_path_ptr = rst_path; + } + + case_nb =load_configs (cfg_path_ptr); + fd =fopen (rst_path_ptr,"w" ); +if (fd ==NULL ) {Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +8
-0+printf ("Open output CSV file error.\n" ); +return -1 ; + } +fclose (fd); + +for (i =0 ; i < case_nb; i++) { +if (test_cases[i].test_type ==TEST_TYPE_NONE ) { + printf("No test type in test case %d.\n\n", i +1);
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +8
-0+continue ; + } +if (!test_cases[i].is_valid) { +printf ("Invalid test case %d.\n\n" , i +1 ); +continue ; + } + + cpid =fork ();[Anoob] Do we really need
fork()? Can't we use code like,Quoted 10 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
RTE_LCORE_FOREACH_WORKER(lcore_id) { ret |= rte_eal_wait_lcore(lcore_id); }to wait for all threads to exit?
[Cheng] Good question.
Fork()is used here to establish a newprocess
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
for the new test case. In order for each test case to have a new EAL environment (for the flexibility), the EAL must be reinitialized for
each
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
case.
Quoted 1 line
Quoted 1 line
However, the EAL parameters can only be initialized once per
process.
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Therefore, we use a new process to run each new test case.
Moreover,
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
each test case runs sequentially and does not affect the others, ensuring the accuracy of the performance data. Your code would
wait
Quoted 12 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
for all threads to exit in the same process. However, it would not
provide
Quoted 5 lines
a
Quoted 4 lines
"clean"
Quoted 3 lines
environment for each test case like
fork()does.Fork()allows us to have a fully reinitialized environment, with no impact or side effects from previous test cases. This results in clean, precise performancedata
Quoted 4 lines
Quoted 4 lines
for
Quoted 3 lines
each case.
Quoted 2 lines
Please let me know your thoughts on this. And please let me know
if
Quoted 169 lines
Quoted 79 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
you have any other questions or require any clarification.
[Anoob] This was just a generic observation. I do not have a strong
opinion
Quoted 75 lines
Quoted 2 lines
either way.
[Cheng] sure, got it.
Quoted 15 lines
Quoted 15 lines
Thanks,
ChengQuoted 11 lines
Quoted 11 lines
app/test-dma-perf/main.c new file +7
-0+if (cpid <0 ) { +printf ("Fork case %d failed.\n" , i +1 ); +exit (EXIT_FAILURE ); + }else if (cpid ==0 ) { +printf ("\nRunning case %u\n\n" , i +1 ); + + new_argc = append_eal_args(argc, argv,test_cases[i].eal_args, pargs);
app/test-dma-perf/main.c new file +2
-0+ ret =rte_eal_init (new_argc, pargs); +if (ret <0 )Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +1
-0+ rte_exit(EXIT_FAILURE, "Invalid EALarguments\n");
app/test-dma-perf/main.c new file +6
-0+ +/* Check lcores. */ + nb_lcores =rte_lcore_count (); +if (nb_lcores <2 ) + rte_exit(EXIT_FAILURE, + "There should be at least 2worker
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
app/test-dma-perf/main.c
Quoted 5 lines
lcores.\n");
app/test-dma-perf/main.c new file +4
-0+ + fd =fopen (rst_path_ptr,"a" ); +if (!fd) { + printf("Open output CSV fileerror.\n");
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
app/test-dma-perf/main.c new file +18
-0+return 0 ; + } + +if (is_first_case) { +output_env_info (); + is_first_case = false; + } +run_test (i +1 , &test_cases[i]); + +/* clean up the EAL */ +rte_eal_cleanup (); + +fclose (fd); + +printf ("\nCase %u completed.\n\n" , i +1 ); + +exit (EXIT_SUCCESS ); + }else {Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.c
Quoted 8 lines
app/test-dma-perf/main.c new file +8
-0+ wpid =waitpid (cpid, &wstatus,0 ); +if (wpid ==-1 ) { +printf ("waitpid error.\n" ); +exit (EXIT_FAILURE ); + } + +if (WIFEXITED (wstatus)) + printf("Case process exited. status%d\n\n",
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/main.c
Quoted 3 lines
app/test-dma-perf/main.c new file +3
-0+ WEXITSTATUS(wstatus)); + else if (WIFSIGNALED(wstatus)) + printf("Case process killed by signal%d\n\n",
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
app/test-dma-perf/main.c
Quoted 3 lines
app/test-dma-perf/main.c new file +3
-0+ WTERMSIG(wstatus)); + else if (WIFSTOPPED(wstatus)) + printf("Case process stopped bysignal
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/main.c
Quoted 4 lines
%d\n\n",
app/test-dma-perf/main.c new file +3
-0+ WSTOPSIG(wstatus)); + else if (WIFCONTINUED(wstatus)) + printf("Case processcontinued.\n\n");
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
app/test-dma-
Quoted 11 lines
app/test-dma-perf/main.c new file +2
-0+else + printf("Case process unknownterminated.\n\n");
app/test-dma-perf/main.c new file +7
-0+ } + } + +printf ("Bye...\n" ); +return 0 ; +} +app/test-dma- renamed from app/test-dma-perf/main.h +0
-0
perf/main.hQuoted 78 lines
Quoted 78 lines
Quoted 19 lines
Quoted 19 lines
Quoted 19 lines
Quoted 19 lines
new file mode 100644 index 0000000000..215ac42673
app/test-dma-perf/main.h new file +15
-0@@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Intel Corporation */ + +#ifndef _MAIN_H_ +#define _MAIN_H_ + + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_dev.h> +#include <rte_dmadev.h> + +#ifndef __maybe_unused +#define __maybe_unused __rte_unused +#endif Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.h line 23
Quoted 8 lines
app/test-dma-perf/main.h new file +8
-0+ +#define MAX_WORKER_NB 128 +#define MAX_OUTPUT_STR_LEN 512 + +#define MAX_DMA_NB 128 +#define MAX_LCORE_NB 256 + +extern charoutput_str[MAX_WORKER_NB][MAX_OUTPUT_STR_LEN];
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
app/test-dma-perf/main.h new file +10
-0+ +typedef enum { +OP_NONE =0 , +OP_ADD , +OP_MUL +}alg_op_type ; + +struct test_configure_entry { +uint32_t first; +uint32_t last;Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.h line 41
Quoted 8 lines
app/test-dma-perf/main.h new file +8
-0+uint32_t incr; +alg_op_type op; +uint32_t cur; +}; + +struct lcore_dma_map_t { +uint32_t lcores[MAX_WORKER_NB ]; +char dma_names[MAX_WORKER_NB][RTE_DEV_NAME_MAX_LEN];
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
app/test-dma-perf/main.h new file +17
-0+int16_t dma_ids[MAX_WORKER_NB ]; +uint16_t cnt; +}; + +struct test_configure { +bool is_valid; +uint8_t test_type; +const char *test_type_str; +uint16_t src_numa_node; +uint16_t dst_numa_node; +uint16_t opcode; +bool is_dma; +struct lcore_dma_map_t lcore_dma_map; +struct test_configure_entry mem_size; +struct test_configure_entry buf_size; +struct test_configure_entry ring_size; +struct test_configure_entry kick_batch;Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
app/test-dma-perf/main.h line 66
Quoted 8 lines
app/test-dma-perf/main.h new file +8
-0+uint32_t cache_flush; +uint32_t nr_buf; +uint16_t test_secs; +const char *eal_args; +uint8_t scenario_id; +}; + +void mem_copy_benchmark(struct test_configure *cfg, bool
is_dma);Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/main.h new file +2
-0+ +#endif /* _MAIN_H_ */ app/test-dma- renamed from app/test-dma-perf/meson.build +0
-0perf/meson.build new file mode 100644 index
0000000000..bd6c264002
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
app/test-dma-perf/meson.build line 1
Quoted 4 lines
app/test-dma-perf/meson.build new file +1
-0@@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019-2023
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
app/test-dma-perf/meson.build new file +16
-1+Intel Corporation + +# meson file, for building this app as part of a main DPDK build. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +deps += ['dmadev', 'mbuf', 'cfgfile'] + +sources = files( + 'main.c', + 'benchmark.c', +) --2.40.1
Hi Cheng,
Quoted 2 lines
[Cheng] sure, no problem. Feel free to contact me if you have any more questions.
No further comments from my side. Can you submit next version?
Top level, I think you may need to add a doc file for the new app.
Few issues that we are also checking in our end,
1. The app is having significant memory footprint. Need to see where we can improve.
2. EAL args passing doesn't seem to be working. Are you able to enable specific devices by making using of '-a' arg?
Thanks,
Anoob
Hi Cheng,
I think there is a bug in EAL parsing. Please check the below diff and see if we should do something similar.
app/test-dma-perf/main.c +1 -1
@@ -432,7 +432,7 @@ append_eal_args(int argc, char **argv, const char *eal_args, char **new_argv) i++;continue ; } -strlcpy (new_argv[new_argc], argv[i],sizeof (new_argv[new_argc])); +strlcpy (new_argv[new_argc], argv[i],MAX_EAL_PARAM_LEN ); new_argc++; }
Thanks,
Anoob
Quoted 29 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 3:19 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationHi Cheng,
Quoted 2 lines
[Cheng] sure, no problem. Feel free to contact me if you have any more questions.
No further comments from my side. Can you submit next version?
Top level, I think you may need to add a doc file for the new app.
Few issues that we are also checking in our end, 1. The app is having significant memory footprint. Need to see where we can improve.
2. EAL args passing doesn't seem to be working. Are you able to enable specific devices by making using of '-a' arg?Thanks,
Anoob
Hi Anoob,
Replies are inline.
Thanks,
Cheng
Quoted 32 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 6:53 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationHi Cheng,
I think there is a bug in EAL parsing. Please check the below diff and see if we should do something similar.
app/test-dma-perf/main.c index renamed from app/test-dma-perf/main.c +0
-0
d65655b87b..7fcaa5acf6100644app/test-dma-perf/main.c +1
-1@@ -432,7 +432,7 @@ append_eal_args(int argc, char **argv, const char *eal_args, char **new_argv) i++;continue ; } -strlcpy (new_argv[new_argc], argv[i],sizeof (new_argv[new_argc])); +strlcpy (new_argv[new_argc], argv[i],MAX_EAL_PARAM_LEN ); new_argc++; }
Quoted 2 lines
Thanks,
Anoob
[Cheng] yes there is an issue in it. And I have the same fix. I'll submit it later, thanks.
Quoted 10 lines
Quoted 10 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 3:19 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>;
Quoted 11 lines
Wang,
Quoted 10 lines
YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationHi Cheng,
Quoted 2 lines
[Cheng] sure, no problem. Feel free to contact me if you have any more questions.
No further comments from my side. Can you submit next version?
[Cheng] Glad to know. Yes, the next version is almost ready, I'll submit it this weekend.
Quoted 2 lines
Quoted 2 lines
Top level, I think you may need to add a doc file for the new app.
[Cheng] sure, I was thinking about this, and I'll add a doc for this, thanks.
Quoted 5 lines
Quoted 5 lines
Few issues that we are also checking in our end, 1. The app is having significant memory footprint. Need to see where we can improve.
2. EAL args passing doesn't seem to be working. Are you able to enable specific devices by making using of '-a' arg?
[Cheng] I'll take a look at the significant memory footprint issue later, thanks. As for the eal args process, yes there is an issue, Dengdui's comments also mentioned it, and I have fixed it in the v7 patch.
Quoted 3 lines
Quoted 3 lines
Thanks,
Anoob
Hi Anoob,
I have a question about the doc.
Do you think I should add a folder in the doc/guides/ just like doc/guides/dma_perf?
Thanks,
Cheng
Quoted 106 lines
-----Original Message-----
From: Jiang, Cheng1
Sent: Friday, June 16, 2023 11:16 PM
To: Anoob Joseph <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationHi Anoob,
Replies are inline.
Thanks,
ChengQuoted 9 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 6:53 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>;Wang,
Quoted 14 lines
YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationHi Cheng,
I think there is a bug in EAL parsing. Please check the below diff and see if we should do something similar.
app/test-dma-perf/main.c index renamed from app/test-dma-perf/main.c +0
-0
d65655b87b..7fcaa5acf6100644app/test-dma-perf/main.c append_eal_args(int argc, char **argv, const char · line 436
Quoted 5 lines
app/test-dma-perf/main.c +0
-1@@ -432,7 +432,7 @@ append_eal_args(int argc, char **argv, const char *eal_args, char **new_argv) i++;continue ; } - strlcpy(new_argv[new_argc], argv[i],
sizeof(new_argv[new_argc]));Quoted 7 lines
app/test-dma-perf/main.c +2
-0+strlcpy (new_argv[new_argc], argv[i], +MAX_EAL_PARAM_LEN ); new_argc++; }Thanks,
Anoob[Cheng] yes there is an issue in it. And I have the same fix. I'll submit it later, thanks.
Quoted 8 lines
Quoted 8 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 3:19 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia,Chenbo
Quoted 4 lines
Quoted 2 lines
<[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>;
Wang,
Quoted 1 line
YuanX <[email protected]>; He, Xingguang
Quoted 9 lines
Quoted 9 lines
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Hi Cheng,
Quoted 2 lines
[Cheng] sure, no problem. Feel free to contact me if you have any more questions.
No further comments from my side. Can you submit next version?
[Cheng] Glad to know. Yes, the next version is almost ready, I'll submit it this weekend.
Quoted 2 lines
Quoted 2 lines
Top level, I think you may need to add a doc file for the new app.
[Cheng] sure, I was thinking about this, and I'll add a doc for this, thanks.
Quoted 5 lines
Quoted 5 lines
Few issues that we are also checking in our end, 1. The app is having significant memory footprint. Need to see where we can improve.
2. EAL args passing doesn't seem to be working. Are you able to enable specific devices by making using of '-a' arg?[Cheng] I'll take a look at the significant memory footprint issue later, thanks. As for the eal args process, yes there is an issue, Dengdui's comments also mentioned it, and I have fixed it in the v7 patch.
Quoted 3 lines
Quoted 3 lines
Thanks,
Anoob
Hi Cheng,
Quoted 2 lines
Do you think I should add a folder in the doc/guides/ just like doc/guides/dma_perf?
I think it should be under doc/guides/tools. I'll let Thomas & DMA maintainers comment further.
You can use below patch as a reference,
commit 1f5cfe964eefe96b5f8de1fadf9cc8fd1e214240
Author: Anoob Joseph <[email protected]> Date: Thu Nov 3 18:16:11 2022 +0530
app/security-perf: add session performance test
Thanks,
Anoob
Hi Anoob,
Quoted 14 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Monday, June 19, 2023 1:49 PM
To: Jiang, Cheng1 <[email protected]>; Richardson, Bruce <[email protected]>; [email protected]; Laatz, Kevin <[email protected]>; Chengwen Feng <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Quoted 7 lines
Hi Cheng,
Quoted 2 lines
Do you think I should add a folder in the doc/guides/ just like doc/guides/dma_perf?
I think it should be under doc/guides/tools. I'll let Thomas & DMA maintainers comment further.
OK, I'll try to add the doc in the tools folder first.
Quoted 11 lines
You can use below patch as a reference,
commit
1f5cfe964eefe96b5f8de1fadf9cc8fd1e214240Author: Anoob Joseph <[email protected]> Date: Thu Nov 3 18:16:11 2022 +0530app/security-perf: add session performance test
Thanks,
Anoob
Sure, thanks a lot!
Cheng
Hi Anoob,
I've looked into the memory footprint issue, and I didn't find any.
So could you please help to share the config file you are using? Maybe it can help me to accelerate the debug process.
Thanks a lot,
Cheng
Quoted 105 lines
-----Original Message-----
From: Jiang, Cheng1
Sent: Friday, June 16, 2023 11:16 PM
To: Anoob Joseph <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationHi Anoob,
Replies are inline.
Thanks,
ChengQuoted 9 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 6:53 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>;Wang,
Quoted 24 lines
YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationHi Cheng,
I think there is a bug in EAL parsing. Please check the below diff and see if we should do something similar.
app/test-dma-perf/main.c index renamed from app/test-dma-perf/main.c +0
-0
d65655b87b..7fcaa5acf6100644app/test-dma-perf/main.c +2
-1@@ -432,7 +432,7 @@ append_eal_args(int argc, char **argv, const char *eal_args, char **new_argv) i++;continue ; } -strlcpy (new_argv[new_argc], argv[i],sizeof (new_argv[new_argc])); +strlcpy (new_argv[new_argc], argv[i], +MAX_EAL_PARAM_LEN ); new_argc++; }Quoted 2 lines
Thanks,
Anoob[Cheng] yes there is an issue in it. And I have the same fix. I'll submit it later, thanks.
Quoted 8 lines
Quoted 8 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 3:19 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia,Chenbo
Quoted 4 lines
Quoted 2 lines
<[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>;
Wang,
Quoted 1 line
YuanX <[email protected]>; He, Xingguang
Quoted 9 lines
Quoted 9 lines
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Hi Cheng,
Quoted 2 lines
[Cheng] sure, no problem. Feel free to contact me if you have any more questions.
No further comments from my side. Can you submit next version?
[Cheng] Glad to know. Yes, the next version is almost ready, I'll submit it this weekend.
Quoted 2 lines
Quoted 2 lines
Top level, I think you may need to add a doc file for the new app.
[Cheng] sure, I was thinking about this, and I'll add a doc for this, thanks.
Quoted 5 lines
Quoted 5 lines
Few issues that we are also checking in our end, 1. The app is having significant memory footprint. Need to see where we can improve.
2. EAL args passing doesn't seem to be working. Are you able to enable specific devices by making using of '-a' arg?[Cheng] I'll take a look at the significant memory footprint issue later, thanks. As for the eal args process, yes there is an issue, Dengdui's comments also mentioned it, and I have fixed it in the v7 patch.
Quoted 3 lines
Quoted 3 lines
Thanks,
Anoob
Hi Cheng,
Please see inline.
Thanks,
Anoob
Quoted 13 lines
-----Original Message-----
From: Jiang, Cheng1 <[email protected]>
Sent: Sunday, June 18, 2023 11:05 AM
To: Anoob Joseph <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Quoted 5 lines
Hi Anoob,
I've looked into the memory footprint issue, and I didn't find any. So could you please help to share the config file you are using? Maybe it can help me to accelerate the debug process.
[Anoob] We had to increase the hugepages to get the test running. Otherwise simple memory allocations were failing. Some of structs have redundant fields which can be easily addressed.
For example please check below members in struct lcore_params,
1. scenario_id
2. lcore_id
3. dma_name
4. worker_id
5. test_secs
Also, some of the parameters in the above struct is const for a thread (like buf_size, kick_batch etc). So I was thinking may be it is better to split lcore_params into two portions and have only the dynamic part as volatile. But that is something we can take up later. I leave that to your judgement.
Quoted 118 lines
Thanks a lot,
ChengQuoted 9 lines
-----Original Message-----
From: Jiang, Cheng1
Sent: Friday, June 16, 2023 11:16 PM
To: Anoob Joseph <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>;Wang,
Quoted 1 line
YuanX <[email protected]>; He, Xingguang
Quoted 17 lines
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Hi Anoob,
Replies are inline.
Thanks,
ChengQuoted 7 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 6:53 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia,Chenbo
Quoted 4 lines
Quoted 2 lines
<[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>;
Wang,
Quoted 1 line
YuanX <[email protected]>; He, Xingguang
Quoted 8 lines
Quoted 8 lines
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Hi Cheng,
I think there is a bug in EAL parsing. Please check the below diff and see if we should do something similar.
Quoted 11 lines
app/test-dma-perf/main.c append_eal_args(int argc, char **argv, const · line 436
Quoted 11 lines
app/test-dma-perf/main.c +0
-0index
d65655b87b..7fcaa5acf6100644app/test-dma-perf/main.c +0
-1@@ -432,7 +432,7 @@ append_eal_args(int argc, char **argv, const char *eal_args, char **new_argv) i++;continue ; } - strlcpy(new_argv[new_argc], argv[i],
sizeof(new_argv[new_argc]));Quoted 12 lines
Quoted 7 lines
app/test-dma-perf/main.c +2
-0+strlcpy (new_argv[new_argc], argv[i], +MAX_EAL_PARAM_LEN ); new_argc++; }Thanks,
Anoob[Cheng] yes there is an issue in it. And I have the same fix. I'll submit it later, thanks.
Quoted 1 line
Quoted 10 lines
Quoted 7 lines
Quoted 7 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 3:19 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia,Chenbo
Quoted 2 lines
Quoted 2 lines
<[email protected]>; Amit Prakash Shukla
<[email protected]>; Ma, WenwuXQuoted 13 lines
Quoted 2 lines
Wang,
Quoted 1 line
YuanX <[email protected]>; He, Xingguang
Quoted 9 lines
Quoted 9 lines
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Hi Cheng,
Quoted 2 lines
[Cheng] sure, no problem. Feel free to contact me if you have any more questions.
No further comments from my side. Can you submit next version?
Quoted 11 lines
[Cheng] Glad to know. Yes, the next version is almost ready, I'll submit it this weekend.
Quoted 2 lines
Quoted 2 lines
Top level, I think you may need to add a doc file for the new app.
[Cheng] sure, I was thinking about this, and I'll add a doc for this, thanks.
Quoted 3 lines
Quoted 3 lines
Few issues that we are also checking in our end, 1. The app is having significant memory footprint. Need to see where we can
improve.
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
2. EAL args passing doesn't seem to be working. Are you able to enable specific devices by making using of '-a' arg?
[Cheng] I'll take a look at the significant memory footprint issue later,
thanks.
Quoted 6 lines
As for the eal args process, yes there is an issue, Dengdui's comments also mentioned it, and I have fixed it in the v7 patch.
Quoted 3 lines
Quoted 3 lines
Thanks,
Anoob
Hi Anoob,
Replies are inline.
Thanks,
Cheng
Quoted 44 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Monday, June 19, 2023 1:26 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationHi Cheng,
Please see inline.
Thanks,
AnoobQuoted 9 lines
-----Original Message-----
From: Jiang, Cheng1 <[email protected]>
Sent: Sunday, June 18, 2023 11:05 AM
To: Anoob Joseph <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>;Wang,
Quoted 9 lines
YuanX <[email protected]>; He, Xingguang <[email protected]>
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf applicationHi Anoob,
I've looked into the memory footprint issue, and I didn't find any. So could you please help to share the config file you are using? Maybe it can help me to accelerate the debug process.
[Anoob] We had to increase the hugepages to get the test running. Otherwise simple memory allocations were failing. Some of structs have redundant fields which can be easily addressed.
Quoted 8 lines
For example please check below members in struct
lcore_params, 1.scenario_id2.lcore_id3.dma_name4.worker_id5.test_secsAlso, some of the parameters in the above struct is const for a thread (like
buf_size,kick_batchetc). So I was thinking may be it is better to splitlcore_paramsinto two portions and have only the dynamic part as volatile. But that is something we can take up later. I leave that to your judgement.
[Cheng] OK, I'll take a look and try to make some improvements. Thanks for your advice.
Quoted 120 lines
Quoted 11 lines
Thanks a lot,
ChengQuoted 7 lines
-----Original Message-----
From: Jiang, Cheng1
Sent: Friday, June 16, 2023 11:16 PM
To: Anoob Joseph <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia,Chenbo
Quoted 108 lines
Quoted 2 lines
<[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>;
Wang,
Quoted 1 line
YuanX <[email protected]>; He, Xingguang
Quoted 17 lines
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Hi Anoob,
Replies are inline.
Thanks,
ChengQuoted 7 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 6:53 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia,Chenbo
Quoted 4 lines
Quoted 2 lines
<[email protected]>; Amit Prakash Shukla <[email protected]>; Ma, WenwuX <[email protected]>;
Wang,
Quoted 1 line
YuanX <[email protected]>; He, Xingguang
Quoted 8 lines
Quoted 8 lines
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Hi Cheng,
I think there is a bug in EAL parsing. Please check the below diff and see if we should do something similar.
Quoted 11 lines
app/test-dma-perf/main.c append_eal_args(int argc, char **argv, const · line 436
Quoted 11 lines
app/test-dma-perf/main.c +0
-0index
d65655b87b..7fcaa5acf6100644app/test-dma-perf/main.c +0
-1@@ -432,7 +432,7 @@ append_eal_args(int argc, char **argv, const char *eal_args, char **new_argv) i++;continue ; } - strlcpy(new_argv[new_argc], argv[i],
sizeof(new_argv[new_argc]));Quoted 12 lines
Quoted 7 lines
app/test-dma-perf/main.c +2
-0+strlcpy (new_argv[new_argc], argv[i], +MAX_EAL_PARAM_LEN ); new_argc++; }Thanks,
Anoob[Cheng] yes there is an issue in it. And I have the same fix. I'll submit it later, thanks.
Quoted 1 line
Quoted 10 lines
Quoted 7 lines
Quoted 7 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 16, 2023 3:19 PM
To: Jiang, Cheng1 <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia,Chenbo
Quoted 2 lines
Quoted 2 lines
<[email protected]>; Amit Prakash Shukla
<[email protected]>; Ma, WenwuXQuoted 13 lines
Quoted 2 lines
Wang,
Quoted 1 line
YuanX <[email protected]>; He, Xingguang
Quoted 9 lines
Quoted 9 lines
Subject: RE: [EXT] [PATCH v6] app/dma-perf: introduce dma-perf application
Hi Cheng,
Quoted 2 lines
[Cheng] sure, no problem. Feel free to contact me if you have any more questions.
No further comments from my side. Can you submit next version?
Quoted 11 lines
[Cheng] Glad to know. Yes, the next version is almost ready, I'll submit it this weekend.
Quoted 2 lines
Quoted 2 lines
Top level, I think you may need to add a doc file for the new app.
[Cheng] sure, I was thinking about this, and I'll add a doc for this, thanks.
Quoted 3 lines
Quoted 3 lines
Few issues that we are also checking in our end, 1. The app is having significant memory footprint. Need to see where we can
improve.
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
2. EAL args passing doesn't seem to be working. Are you able to enable specific devices by making using of '-a' arg?
[Cheng] I'll take a look at the significant memory footprint issue later,
thanks.
Quoted 6 lines
As for the eal args process, yes there is an issue, Dengdui's comments also mentioned it, and I have fixed it in the v7 patch.
Quoted 3 lines
Quoted 3 lines
Thanks,
Anoob
PATCH v7 1/1 app/dma-perf: introduce dma-perf application
In reply to Cheng JiangThere are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are supported for now.
---
v7:
fixed some strcpy issues;
removed cache setup in calling rte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed some LONG_LINE warnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
app/meson.build | 1 + app/test-dma-perf/benchmark.c | 498 ++++++++++++++++++++++++++++ app/test-dma-perf/config.ini | 61 ++++ app/test-dma-perf/main.c | 594 ++++++++++++++++++++++++++++++++++ app/test-dma-perf/main.h | 69 ++++ app/test-dma-perf/meson.build | 17 + 6 files changed, 1240 insertions(+) create mode 100644 app/test-dma-perf/benchmark.c create mode 100644 app/test-dma-perf/config.ini create mode 100644 app/test-dma-perf/main.c create mode 100644 app/test-dma-perf/main.h create mode 100644 app/test-dma-perf/meson.build
app/meson.build +1 -0
@@ -19,6 +19,7 @@ apps = [ 'test-cmdline', 'test-compress-perf', 'test-crypto-perf', + 'test-dma-perf', 'test-eventdev', 'test-fib', 'test-flow-perf',
app/test-dma-perf/benchmark.c new file +498 -0
498 lines not on this page · raw message
app/test-dma-perf/config.ini new file +61 -0
61 lines not on this page · raw message
app/test-dma-perf/main.c new file +594 -0
594 lines not on this page · raw message
app/test-dma-perf/main.h new file +69 -0
69 lines not on this page · raw message
app/test-dma-perf/meson.build new file +17 -0
@@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019-2023 Intel Corporation + +# meson file, for building this app as part of a main DPDK build. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +deps += ['dmadev', 'mbuf', 'cfgfile'] + +sources = files( + 'main.c', + 'benchmark.c', +)
--
2.40.1
PATCH v8 1/1 app/dma-perf: introduce dma-perf application
In reply to Cheng JiangThere are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are supported for now.
---
v8:
fixed string copy issue in parse_lcore();
improved some data display format;
added doc in doc/guides/tools;
updated release notes;
v7:
fixed some strcpy issues;
removed cache setup in calling rte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed some LONG_LINE warnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removed worker_threads parameter from config.ini; improved the logs;
improved config file;
app/meson.build | 1 + app/test-dma-perf/benchmark.c | 498 +++++++++++++++++++++ app/test-dma-perf/config.ini | 61 +++ app/test-dma-perf/main.c | 594 +++++++++++++++++++++++++ app/test-dma-perf/main.h | 69 +++ app/test-dma-perf/meson.build | 17 + doc/guides/rel_notes/release_23_07.rst | 6 + doc/guides/tools/dmaperf.rst | 103 +++++ doc/guides/tools/index.rst | 1 + 9 files changed, 1350 insertions(+) create mode 100644 app/test-dma-perf/benchmark.c create mode 100644 app/test-dma-perf/config.ini create mode 100644 app/test-dma-perf/main.c create mode 100644 app/test-dma-perf/main.h create mode 100644 app/test-dma-perf/meson.build create mode 100644 doc/guides/tools/dmaperf.rst
app/meson.build +1 -0
@@ -19,6 +19,7 @@ apps = [ 'test-cmdline', 'test-compress-perf', 'test-crypto-perf', + 'test-dma-perf', 'test-eventdev', 'test-fib', 'test-flow-perf',
app/test-dma-perf/benchmark.c new file +498 -0
498 lines not on this page · raw message
app/test-dma-perf/config.ini new file +61 -0
61 lines not on this page · raw message
app/test-dma-perf/main.c new file +594 -0
594 lines not on this page · raw message
app/test-dma-perf/main.h new file +69 -0
69 lines not on this page · raw message
app/test-dma-perf/meson.build new file +17 -0
@@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019-2023 Intel Corporation + +# meson file, for building this app as part of a main DPDK build. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +deps += ['dmadev', 'mbuf', 'cfgfile'] + +sources = files( + 'main.c', + 'benchmark.c', +)
doc/guides/rel_notes/release_23_07.rst +6 -0
@@ -170,6 +170,12 @@ New Features See :doc:`../prog_guide/pdcp_lib` for more information. +* **Added DMA device performance test application.** + + Added an new application to test the performance of DMA device and CPU. + + See the :doc:`../tools/dmaperf` for more details. + Removed Items -------------
doc/guides/tools/dmaperf.rst new file +103 -0
103 lines not on this page · raw message
doc/guides/tools/index.rst +1 -0
@@ -23,3 +23,4 @@ DPDK Tools User Guides testregex testmldev dts + dmaperf
--
2.40.1
Hi Cheng,
Thanks for the new version. Please see inline.
Thanks,
Anoob
Quoted 78 lines
-----Original Message-----
From: Cheng Jiang <[email protected]>
Sent: Tuesday, June 20, 2023 12:24 PM
To: [email protected]; [email protected]; [email protected]; [email protected]; Amit Prakash Shukla <[email protected]>; Anoob Joseph <[email protected]>; [email protected]; [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Cheng Jiang <[email protected]>
Subject: [EXT] [PATCH v8] app/dma-perf: introduce dma-perf applicationExternal Email
----------------------------------------------------------------------
There are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre- set config file. Memory Copy performance test are supported for now.Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>---
v8:
fixed string copy issue inparse_lcore();
improved some data display format;
added doc in doc/guides/tools;
updated release notes;v7:
fixed some strcpy issues;
removed cache setup in callingrte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed someLONG_LINEwarnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removedworker_threadsparameter from config.ini; improved the logs;
improved config file;app/meson.build | 1 + app/test-dma-perf/benchmark.c | 498 +++++++++++++++++++++ app/test-dma-perf/config.ini | 61 +++ app/test-dma-perf/main.c | 594 +++++++++++++++++++++++++ app/test-dma-perf/main.h | 69 +++ app/test-dma-perf/meson.build | 17 + doc/guides/rel_notes/release_23_07.rst | 6 + doc/guides/tools/dmaperf.rst | 103 +++++ doc/guides/tools/index.rst | 1 + 9 files changed, 1350 insertions(+) create mode 100644 app/test-dma-perf/benchmark.c create mode 100644app/test-dma-perf/config.ini create mode 100644 app/test-dma-
perf/main.ccreate mode 100644app/test-dma-perf/main.hcreate mode 100644 app/test-dma-perf/meson.build create mode 100644doc/guides/tools/dmaperf.rst
<snip>
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +8
-0+/* Configuration of device. */ +static void +configure_dmadev_queue (uint32_t dev_id,uint32_t ring_size) { +uint16_t vchan =0 ; +struct rte_dma_info info; +struct rte_dma_conf dev_config = { .nb_vchans =1 }; +struct rte_dma_vchan_conf qconf = { + .direction =RTE_DMA_DIR_MEM_TO_MEM ,
app/test-dma-perf/benchmark.c line 150
Quoted 10 lines
app/test-dma-perf/benchmark.c new file +10
-0+ .nb_desc = ring_size + }; + +if (rte_dma_configure (dev_id, &dev_config) !=0 ) +rte_exit (EXIT_FAILURE ,"Error with dma configure.\n" ); + +if (rte_dma_vchan_setup (dev_id, vchan, &qconf) !=0 ) +rte_exit (EXIT_FAILURE ,"Error with queue configuration.\n" ); + +rte_dma_info_get (dev_id, &info);
[Anoob] This API can return errors. Better to add handling.
Quoted 12 lines
(quoted diff) +2
-0+ if (info.nb_vchans != 1) + rte_exit(EXIT_FAILURE, "Error, no configured queuesreported on device id. %u\n",
app/test-dma-perf/benchmark.c new file +9
-0+ dev_id); + +if (rte_dma_start (dev_id) !=0 ) +rte_exit (EXIT_FAILURE ,"Error with dma start.\n" ); } + +static int +config_dmadevs (struct test_configure *cfg) { +uint32_t ring_size = cfg->ring_size.cur; +struct lcore_dma_map_t *ldm = &cfg->lcore_dma_map;
app/test-dma-perf/benchmark.c line 173
Quoted 10 lines
app/test-dma-perf/benchmark.c new file +10
-0+uint32_t nb_workers = ldm->cnt; +uint32_t i; +int dev_id; +uint16_t nb_dmadevs =0 ; +char *dma_name; + +for (i =0 ; i < ldm->cnt; i++) { + dma_name = ldm->dma_names[i]; + dev_id =rte_dma_get_dev_id_by_name (dma_name); +if (dev_id ==-1 ) {
[Anoob] Can you check the above API definition? I think it returns not just -1 in case of errors.
Quoted 37 lines
(quoted diff) +1
-0+ fprintf(stderr, "Error: Fail to find DMA %s.\n",
dma_name);app/test-dma-perf/benchmark.c new file +11
-0+ goto end; + } + + ldm->dma_ids[i] = dev_id; +configure_dmadev_queue (dev_id, ring_size); + ++nb_dmadevs; + } + +end: +if (nb_dmadevs < nb_workers) { + printf("Not enough dmadevs (%u) for all workers (%u).\n",
nb_dmadevs,nb_workers);app/test-dma-perf/benchmark.c new file +22
-0+return -1 ; + } + +printf ("Number of used dmadevs: %u.\n" , nb_dmadevs); + +return 0 ; +} + +static inline void +do_dma_submit_and_poll (uint16_t dev_id,uint64_t *async_cnt, +volatile struct worker_info *worker_info) { +int ret; +uint16_t nr_cpl; + + ret =rte_dma_submit (dev_id,0 ); +if (ret <0 ) { +rte_dma_stop (dev_id); +rte_dma_close (dev_id); +rte_exit (EXIT_FAILURE ,"Error with dma submit.\n" ); + } + + nr_cpl = rte_dma_completed(dev_id, 0, MAX_DMA_CPL_NB, NULL,NULL);
app/test-dma-perf/benchmark.c line 214
Quoted 6 lines
app/test-dma-perf/benchmark.c new file +6
-0+ *async_cnt -= nr_cpl; + worker_info->total_cpl += nr_cpl; +} + +static inline int +do_dma_mem_copy (void *p)
[Anoob] Just curious, why not pass struct lcore_params *para itself? Is it because the pointer is volatile? If yes, then we can take an AI to split the struct into volatile and non-volatile parts.
Quoted 27 lines
app/test-dma-perf/benchmark.c new file +26
-0+{ +const uint16_t *para_idx = (uint16_t *)p; +volatile struct lcore_params *para = lcores_p[*para_idx].v_ptr; +volatile struct worker_info *worker_info = &(para->worker_info); +const uint16_t dev_id = para->dev_id; +const uint32_t nr_buf = para->nr_buf; +const uint16_t kick_batch = para->kick_batch; +const uint32_t buf_size = para->buf_size; +struct rte_mbuf **srcs = para->srcs; +struct rte_mbuf **dsts = para->dsts; +uint16_t nr_cpl; +uint64_t async_cnt =0 ; +uint32_t i; +uint32_t poll_cnt =0 ; +int ret; + + worker_info->stop_flag = false; + worker_info->ready_flag = true; + +while (!worker_info->start_flag) + ; + +while (1 ) { +for (i =0 ; i < nr_buf; i++) { +dma_copy: + ret = rte_dma_copy(dev_id, 0,
rte_pktmbuf_iova(srcs[i]),
app/test-dma-perf/benchmark.c line 241
Quoted 1 line
app/test-dma-perf/benchmark.c new file +1
-0+ rte_pktmbuf_iova(dsts[i]), buf_size, 0);
[Anoob] Do we need to use ' rte_mbuf_data_iova' here instead of 'rte_pktmbuf_iova'?
app/test-dma-perf/benchmark.c line 248
Quoted 8 lines
app/test-dma-perf/benchmark.c new file +3
-0+if (unlikely (ret <0 )) { +if (ret == -ENOSPC ) { + do_dma_submit_and_poll(dev_id,
&async_cnt,worker_info);app/test-dma-perf/benchmark.c new file +4
-0+ goto dma_copy; + }else { +/* Error exit */ +rte_dma_stop (dev_id);
[Anoob] Missing rte_dma_close() here. Also, may be introduce a static void function so that rte_exit call etc won't be part of the fastpath loop.
May be something like below and you can call it from here and "do_dma_submit_and_poll".
static voiderror_exit(int dev_id)
{
/* Error exit */
rte_dma_stop(dev_id);
rte_dma_close(dev_id);
rte_exit(EXIT_FAILURE, "DMA enqueue failed\n");
}Quoted 25 lines
(quoted diff) +1
-0+ rte_exit(EXIT_FAILURE, "DMAenqueue failed\n");
app/test-dma-perf/benchmark.c new file +6
-0+ } + } + async_cnt++; + +if ((async_cnt % kick_batch) ==0 ) + do_dma_submit_and_poll(dev_id,
&async_cnt,worker_info);app/test-dma-perf/benchmark.c new file +9
-0+ } + +if (worker_info->stop_flag) +break ; + } + +rte_dma_submit (dev_id,0 ); +while ((async_cnt >0 ) && (poll_cnt++ <POLL_MAX )) { + nr_cpl = rte_dma_completed(dev_id, 0,
MAX_DMA_CPL_NB, NULL, NULL);app/test-dma-perf/benchmark.c new file +6
-0+ async_cnt -= nr_cpl; + } + +return 0 ; +} +
<snip>
Quoted 41 lines
app/test-dma-perf/benchmark.c new file +12
-0+static int +setup_memory_env (struct test_configure *cfg,struct rte_mbuf ***srcs, +struct rte_mbuf ***dsts) +{ +unsigned int buf_size = cfg->buf_size.cur; +unsigned int nr_sockets; +uint32_t nr_buf = cfg->nr_buf; + + nr_sockets =rte_socket_count (); +if (cfg->src_numa_node >= nr_sockets || + cfg->dst_numa_node >= nr_sockets) { + printf("Error: Source or destination numa exceeds the acturenuma nodes.\n");
app/test-dma-perf/benchmark.c new file +28
-0+return -1 ; + } + + src_pool =rte_pktmbuf_pool_create ("Benchmark_DMA_SRC" , + nr_buf, +0 , +0 , + buf_size +RTE_PKTMBUF_HEADROOM , + cfg->src_numa_node); +if (src_pool ==NULL ) { +PRINT_ERR ("Error with source mempool creation.\n" ); +return -1 ; + } + + dst_pool =rte_pktmbuf_pool_create ("Benchmark_DMA_DST" , + nr_buf, +0 , +0 , + buf_size +RTE_PKTMBUF_HEADROOM , + cfg->dst_numa_node); +if (dst_pool ==NULL ) { +PRINT_ERR ("Error with destination mempool creation.\n" ); +return -1 ; + } + + *srcs =rte_malloc (NULL , nr_buf *sizeof (struct rte_mbuf *),0 ); +if (*srcs ==NULL ) { +printf ("Error: srcs malloc failed.\n" );
app/test-dma-perf/benchmark.c line 353
Quoted 10 lines
app/test-dma-perf/benchmark.c new file +10
-0+return -1 ; + } + + *dsts =rte_malloc (NULL , nr_buf *sizeof (struct rte_mbuf *),0 ); +if (*dsts ==NULL ) { +printf ("Error: dsts malloc failed.\n" ); +return -1 ; + } + +if (rte_mempool_get_bulk (src_pool, (void **)*srcs, nr_buf) !=0 ) {
[Anoob] Might be better to use 'rte_pktmbuf_alloc_bulk' since we use ' rte_mbuf_data_iova' in the datapath and it is desirable to initialize it properly as an mbuf. Same comment below as well.
<snip>
Review
Quoted 11 lines
app/test-dma-perf/benchmark.c new file +7
-0+ +for (i =0 ; i < nb_workers; i++) { +calc_result (buf_size, nr_buf, nb_workers, test_secs, + lcores_p[i].v_ptr->worker_info.test_cpl, + &memory, &avg_cycles, &bandwidth, &mops); + output_result(cfg->scenario_id, lcores_p[i].v_ptr->lcore_id, + lcores_p[i].v_ptr->dma_name,
avg_cycles,buf_size,(quoted diff) +1
-0+ nr_buf / nb_workers, memory,bandwidth, mops,
is_dma);(quoted diff) +1
-0+ }
[Anoob] Can you also print total_bandwidth & total_mops? It can be a simple aggregation in the above loop. Would help when we are dealing with larger number of queues but single hardware block.
app/test-dma-perf/benchmark.c line 466
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+ +out: +/* free mbufs used in the test */ +if (srcs)
[Anoob] DPDK coding guidelines recommend a usage like below,
(if (srcs != NULL)
Quoted 7 lines
app/test-dma-perf/benchmark.c new file +7
-0+rte_pktmbuf_free_bulk (srcs, nr_buf); +if (dsts) +rte_pktmbuf_free_bulk (dsts, nr_buf); + +/* free the points for the mbufs */ +rte_free (srcs); + srcs =NULL ;
app/test-dma-perf/benchmark.c line 483
Quoted 10 lines
app/test-dma-perf/benchmark.c new file +10
-0+rte_free (dsts); + dsts =NULL ; + +if (src_pool) { +rte_mempool_free (src_pool); + src_pool =NULL ; + } +if (dst_pool) { +rte_mempool_free (dst_pool); + src_pool =NULL ;
[Anoob] Should be dst_pool, right?
<snip>
Quoted 7 lines
app/test-dma-perf/main.h new file renamed from app/test-dma-perf/main.h +0
-0mode 100644 index 0000000000..215ac42673
app/test-dma-perf/main.h new file +2
-0@@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Intel Corporation
app/test-dma-perf/main.h line 12
Quoted 10 lines
app/test-dma-perf/main.h new file +10
-0+*/ + +#ifndef _MAIN_H_ +#define _MAIN_H_ + + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_dev.h> +#include <rte_dmadev.h>
[Anoob] Is the above include (rte_dmadev.h) required?
app/test-dma-perf/main.h line 16
Quoted 4 lines
app/test-dma-perf/main.h new file +4
-0+ +#ifndef __maybe_unused +#define __maybe_unused __rte_unused +#endif
[Anoob] Can you try to avoid this and use rte_unused or RTE_SET_USED in the cache_flush_buf() function?
Hi Anoob,
Replies are inline.
Thanks,
Cheng
Quoted 117 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 23, 2023 2:53 PM
To: Jiang, Cheng1 <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; [email protected]; Laatz, Kevin <[email protected]>; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>; Ling, WeiX <[email protected]>
Subject: RE: [EXT] [PATCH v8] app/dma-perf: introduce dma-perf applicationHi Cheng,
Thanks for the new version. Please see inline.
Thanks,
AnoobQuoted 14 lines
-----Original Message-----
From: Cheng Jiang <[email protected]>
Sent: Tuesday, June 20, 2023 12:24 PM
To: [email protected]; [email protected]; [email protected]; [email protected]; Amit Prakash Shukla <[email protected]>; Anoob Joseph <[email protected]>; [email protected]; [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Cheng Jiang <[email protected]>
Subject: [EXT] [PATCH v8] app/dma-perf: introduce dma-perf applicationQuoted 4 lines
External Email
----------------------------------------------------------------------
There are many high-performance DMA devices supported in DPDK now,and
Quoted 9 lines
these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre- set config file. Memory Copy performance test
are supported for now.
Quoted 51 lines
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>---
v8:
fixed string copy issue inparse_lcore();
improved some data display format;
added doc in doc/guides/tools;
updated release notes;v7:
fixed some strcpy issues;
removed cache setup in callingrte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed someLONG_LINEwarnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removedworker_threadsparameter from config.ini; improved the logs;
improved config file;app/meson.build | 1 + app/test-dma-perf/benchmark.c | 498 +++++++++++++++++++++ app/test-dma-perf/config.ini | 61 +++ app/test-dma-perf/main.c | 594 +++++++++++++++++++++++++ app/test-dma-perf/main.h | 69 +++ app/test-dma-perf/meson.build | 17 + doc/guides/rel_notes/release_23_07.rst | 6 + doc/guides/tools/dmaperf.rst | 103 +++++ doc/guides/tools/index.rst | 1 + 9 files changed, 1350 insertions(+) create mode 100644 app/test-dma-perf/benchmark.c create mode 100644app/test-dma-perf/config.ini create mode 100644 app/test-dma-
perf/main.ccreate mode 100644app/test-dma-perf/main.hcreate mode 100644 app/test-dma-perf/meson.build create mode 100644doc/guides/tools/dmaperf.rst<snip>
Quoted 12 lines
app/test-dma-perf/benchmark.c new file +12
-0+/* Configuration of device. */ +static void +configure_dmadev_queue (uint32_t dev_id,uint32_t ring_size) { +uint16_t vchan =0 ; +struct rte_dma_info info; +struct rte_dma_conf dev_config = { .nb_vchans =1 }; +struct rte_dma_vchan_conf qconf = { + .direction =RTE_DMA_DIR_MEM_TO_MEM , + .nb_desc = ring_size + }; + +if (rte_dma_configure (dev_id, &dev_config) !=0 )
Quoted 8 lines
app/test-dma-perf/benchmark.c line 150
Quoted 6 lines
app/test-dma-perf/benchmark.c new file +6
-0+rte_exit (EXIT_FAILURE ,"Error with dma configure.\n" ); + +if (rte_dma_vchan_setup (dev_id, vchan, &qconf) !=0 ) +rte_exit (EXIT_FAILURE ,"Error with queue configuration.\n" ); + +rte_dma_info_get (dev_id, &info);[Anoob] This API can return errors. Better to add handling.
[Cheng] Sure, I'll fix it in the next version.
Quoted 18 lines
Quoted 18 lines
(quoted diff) +2
-0+ if (info.nb_vchans != 1) + rte_exit(EXIT_FAILURE, "Error, no configured queuesreported on device id. %u\n",
app/test-dma-perf/benchmark.c new file +14
-0+ dev_id); + +if (rte_dma_start (dev_id) !=0 ) +rte_exit (EXIT_FAILURE ,"Error with dma start.\n" ); } + +static int +config_dmadevs (struct test_configure *cfg) { +uint32_t ring_size = cfg->ring_size.cur; +struct lcore_dma_map_t *ldm = &cfg->lcore_dma_map; +uint32_t nb_workers = ldm->cnt; +uint32_t i; +int dev_id; +uint16_t nb_dmadevs =0 ; +char *dma_name;
Quoted 8 lines
app/test-dma-perf/benchmark.c line 173
Quoted 5 lines
app/test-dma-perf/benchmark.c new file +5
-0+ +for (i =0 ; i < ldm->cnt; i++) { + dma_name = ldm->dma_names[i]; + dev_id =rte_dma_get_dev_id_by_name (dma_name); +if (dev_id ==-1 ) {[Anoob] Can you check the above API definition? I think it returns not just -1 in case of errors.
[Cheng] Yes, you are right, I'll fix it in the next version. Thanks a lot.
Quoted 40 lines
Quoted 40 lines
(quoted diff) +1
-0+ fprintf(stderr, "Error: Fail to find DMA %s.\n",
dma_name);app/test-dma-perf/benchmark.c new file +11
-0+ goto end; + } + + ldm->dma_ids[i] = dev_id; +configure_dmadev_queue (dev_id, ring_size); + ++nb_dmadevs; + } + +end: +if (nb_dmadevs < nb_workers) { + printf("Not enough dmadevs (%u) for all workers (%u).\n",
nb_dmadevs,nb_workers);app/test-dma-perf/benchmark.c new file +22
-0+return -1 ; + } + +printf ("Number of used dmadevs: %u.\n" , nb_dmadevs); + +return 0 ; +} + +static inline void +do_dma_submit_and_poll (uint16_t dev_id,uint64_t *async_cnt, +volatile struct worker_info *worker_info) { +int ret; +uint16_t nr_cpl; + + ret =rte_dma_submit (dev_id,0 ); +if (ret <0 ) { +rte_dma_stop (dev_id); +rte_dma_close (dev_id); +rte_exit (EXIT_FAILURE ,"Error with dma submit.\n" ); + } + + nr_cpl = rte_dma_completed(dev_id, 0, MAX_DMA_CPL_NB, NULL,NULL);
app/test-dma-perf/benchmark.c new file +2
-0+ *async_cnt -= nr_cpl; + worker_info->total_cpl += nr_cpl;
Quoted 8 lines
app/test-dma-perf/benchmark.c line 214
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+} + +static inline int +do_dma_mem_copy (void *p)[Anoob] Just curious, why not pass struct
lcore_params*para itself? Is it because the pointer is volatile? If yes, then we can take an AI to split the struct into volatile and non-volatile parts.
[Cheng] The reason I did it this way is because I want to launch this function on another core by spawning a new thread, and rte_eal_remote_launch() takes a void * as the parameter. That's why I passed void *p. Your suggestion to split the struct into volatile and non-volatile parts is quite reasonable. I am thinking about the best way to implement it. Thanks.
Quoted 24 lines
Quoted 24 lines
app/test-dma-perf/benchmark.c new file +23
-0+{ +const uint16_t *para_idx = (uint16_t *)p; +volatile struct lcore_params *para = lcores_p[*para_idx].v_ptr; +volatile struct worker_info *worker_info = &(para->worker_info); +const uint16_t dev_id = para->dev_id; +const uint32_t nr_buf = para->nr_buf; +const uint16_t kick_batch = para->kick_batch; +const uint32_t buf_size = para->buf_size; +struct rte_mbuf **srcs = para->srcs; +struct rte_mbuf **dsts = para->dsts; +uint16_t nr_cpl; +uint64_t async_cnt =0 ; +uint32_t i; +uint32_t poll_cnt =0 ; +int ret; + + worker_info->stop_flag = false; + worker_info->ready_flag = true; + +while (!worker_info->start_flag) + ; + +while (1 ) {
Quoted 8 lines
app/test-dma-perf/benchmark.c line 241
Quoted 5 lines
app/test-dma-perf/benchmark.c new file +3
-0+for (i =0 ; i < nr_buf; i++) { +dma_copy: + ret = rte_dma_copy(dev_id, 0,
rte_pktmbuf_iova(srcs[i]),app/test-dma-perf/benchmark.c new file +1
-0+ rte_pktmbuf_iova(dsts[i]), buf_size, 0);[Anoob] Do we need to use '
rte_mbuf_data_iova' here instead of 'rte_pktmbuf_iova'?
[Cheng] yes rte_mbuf_data_iova is more appropriate, I'll fix it in the next version. Thanks.
Quoted 16 lines
app/test-dma-perf/benchmark.c line 248
Quoted 9 lines
app/test-dma-perf/benchmark.c new file +3
-0+if (unlikely (ret <0 )) { +if (ret == -ENOSPC ) { + do_dma_submit_and_poll(dev_id,
&async_cnt,worker_info);app/test-dma-perf/benchmark.c new file +4
-0+ goto dma_copy; + }else { +/* Error exit */ +rte_dma_stop (dev_id);[Anoob] Missing
rte_dma_close()here. Also, may be introduce a static void function so thatrte_exitcall etc won't be part of the fastpath loop.May be something like below and you can call it from here and "
do_dma_submit_and_poll".
Quoted 8 lines
static void
error_exit(int dev_id){ /* Error exit */ rte_dma_stop(dev_id); rte_dma_close(dev_id); rte_exit(EXIT_FAILURE, "DMA enqueue failed\n"); }
[Cheng] I'm not so sure here. rte_dma_close() is called in the rte_exit(). Do we still call it explicitly before rte_exit()?
Quoted 75 lines
Quoted 25 lines
(quoted diff) +1
-0+ rte_exit(EXIT_FAILURE, "DMAenqueue failed\n");
app/test-dma-perf/benchmark.c new file +6
-0+ } + } + async_cnt++; + +if ((async_cnt % kick_batch) ==0 ) + do_dma_submit_and_poll(dev_id,
&async_cnt,worker_info);app/test-dma-perf/benchmark.c new file +9
-0+ } + +if (worker_info->stop_flag) +break ; + } + +rte_dma_submit (dev_id,0 ); +while ((async_cnt >0 ) && (poll_cnt++ <POLL_MAX )) { + nr_cpl = rte_dma_completed(dev_id, 0,
MAX_DMA_CPL_NB, NULL, NULL);app/test-dma-perf/benchmark.c new file +6
-0+ async_cnt -= nr_cpl; + } + +return 0 ; +} +<snip>
Quoted 47 lines
app/test-dma-perf/benchmark.c new file +12
-0+static int +setup_memory_env (struct test_configure *cfg,struct rte_mbuf ***srcs, +struct rte_mbuf ***dsts) +{ +unsigned int buf_size = cfg->buf_size.cur; +unsigned int nr_sockets; +uint32_t nr_buf = cfg->nr_buf; + + nr_sockets =rte_socket_count (); +if (cfg->src_numa_node >= nr_sockets || + cfg->dst_numa_node >= nr_sockets) { + printf("Error: Source or destination numa exceeds the acturenuma nodes.\n");
app/test-dma-perf/benchmark.c new file +34
-0+return -1 ; + } + + src_pool =rte_pktmbuf_pool_create ("Benchmark_DMA_SRC" , + nr_buf, +0 , +0 , + buf_size +RTE_PKTMBUF_HEADROOM , + cfg->src_numa_node); +if (src_pool ==NULL ) { +PRINT_ERR ("Error with source mempool creation.\n" ); +return -1 ; + } + + dst_pool =rte_pktmbuf_pool_create ("Benchmark_DMA_DST" , + nr_buf, +0 , +0 , + buf_size +RTE_PKTMBUF_HEADROOM , + cfg->dst_numa_node); +if (dst_pool ==NULL ) { +PRINT_ERR ("Error with destination mempool creation.\n" ); +return -1 ; + } + + *srcs =rte_malloc (NULL , nr_buf *sizeof (struct rte_mbuf *),0 ); +if (*srcs ==NULL ) { +printf ("Error: srcs malloc failed.\n" ); +return -1 ; + } + + *dsts =rte_malloc (NULL , nr_buf *sizeof (struct rte_mbuf *),0 ); +if (*dsts ==NULL ) { +printf ("Error: dsts malloc failed.\n" );
Quoted 9 lines
app/test-dma-perf/benchmark.c line 353
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+return -1 ; + } + +if (rte_mempool_get_bulk (src_pool, (void **)*srcs, nr_buf) !=0 ) {[Anoob] Might be better to use '
rte_pktmbuf_alloc_bulk' since we use 'rte_mbuf_data_iova' in the datapath and it is desirable to initialize it properly as an mbuf. Same comment below as well.
[Cheng] sure, I'll fix it in the next version.
Quoted 17 lines
<snip>
Review
Quoted 11 lines
app/test-dma-perf/benchmark.c new file +7
-0+ +for (i =0 ; i < nb_workers; i++) { +calc_result (buf_size, nr_buf, nb_workers, test_secs, + lcores_p[i].v_ptr->worker_info.test_cpl, + &memory, &avg_cycles, &bandwidth, &mops); + output_result(cfg->scenario_id, lcores_p[i].v_ptr->lcore_id, + lcores_p[i].v_ptr->dma_name,
avg_cycles,buf_size,(quoted diff) +1
-0+ nr_buf / nb_workers, memory,bandwidth, mops,
is_dma);(quoted diff) +1
-0+ }[Anoob] Can you also print
total_bandwidth&total_mops? It can be a simple aggregation in the above loop. Would help when we are dealing with larger number of queues but single hardware block.
[Cheng] sure, good point. I'll add it in the next version, thanks.
Quoted 9 lines
app/test-dma-perf/benchmark.c line 466
Quoted 5 lines
app/test-dma-perf/benchmark.c new file +4
-0+ +out: +/* free mbufs used in the test */ +if (srcs)[Anoob] DPDK coding guidelines recommend a usage like below,
(if (srcs != NULL)
[Cheng] sure, thanks. I'll fix it in the next version.
Quoted 11 lines
Quoted 11 lines
app/test-dma-perf/benchmark.c new file +11
-0+rte_pktmbuf_free_bulk (srcs, nr_buf); +if (dsts) +rte_pktmbuf_free_bulk (dsts, nr_buf); + +/* free the points for the mbufs */ +rte_free (srcs); + srcs =NULL ; +rte_free (dsts); + dsts =NULL ; + +if (src_pool) {
Quoted 8 lines
app/test-dma-perf/benchmark.c line 483
Quoted 6 lines
app/test-dma-perf/benchmark.c new file +6
-0+rte_mempool_free (src_pool); + src_pool =NULL ; + } +if (dst_pool) { +rte_mempool_free (dst_pool); + src_pool =NULL ;[Anoob] Should be
dst_pool, right?
[Cheng] yes, sorry for the miss. I'll fix it in the next version.
Quoted 13 lines
<snip>
Quoted 10 lines
app/test-dma-perf/main.h new renamed from app/test-dma-perf/main.h +0
-0file mode 100644 index 0000000000..215ac42673
app/test-dma-perf/main.h new file +5
-0@@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2023 Intel Corporation */ + +#ifndef _MAIN_H_ +#define _MAIN_H_
Quoted 8 lines
app/test-dma-perf/main.h line 12
Quoted 6 lines
app/test-dma-perf/main.h new file +6
-0+ + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_dev.h> +#include <rte_dmadev.h> [Anoob] Is the above include (
rte_dmadev.h) required?
[Cheng] you are right. It's not required. I'll remove it in the next version.
Quoted 8 lines
app/test-dma-perf/main.h line 16
Quoted 5 lines
app/test-dma-perf/main.h new file +4
-0+ +#ifndef __maybe_unused +#define __maybe_unused __rte_unused +#endif [Anoob] Can you try to avoid this and use
rte_unusedorRTE_SET_USEDin thecache_flush_buf()function?
[Cheng] sure, I'll fix it in the next version.
Hi Cheng,
Please see inline.
Thanks,
Anoob
Quoted 135 lines
-----Original Message-----
From: Jiang, Cheng1 <[email protected]>
Sent: Saturday, June 24, 2023 5:23 PM
To: Anoob Joseph <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; [email protected]; Laatz, Kevin <[email protected]>; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>; Ling, WeiX <[email protected]>
Subject: RE: [EXT] [PATCH v8] app/dma-perf: introduce dma-perf applicationHi Anoob,
Replies are inline.
Thanks,
ChengQuoted 7 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 23, 2023 2:53 PM
To: Jiang, Cheng1 <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>;Quoted 4 lines
Laatz, Kevin <[email protected]>; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; Wang,YuanX
Quoted 1 line
<[email protected]>; He, Xingguang <[email protected]>;
Ling,
Quoted 4 lines
WeiX <[email protected]>
Subject: RE: [EXT] [PATCH v8] app/dma-perf: introduce dma-perf applicationQuoted 12 lines
Hi Cheng,
Thanks for the new version. Please see inline.
Thanks,
AnoobQuoted 5 lines
-----Original Message-----
From: Cheng Jiang <[email protected]>
Sent: Tuesday, June 20, 2023 12:24 PM
To: [email protected]; [email protected]; [email protected]; [email protected]; Amit PrakashShukla
Quoted 6 lines
Quoted 6 lines
<[email protected]>; Anoob Joseph <[email protected]>; [email protected]; [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected];Quoted 74 lines
Quoted 9 lines
[email protected]; Cheng Jiang <[email protected]>
Subject: [EXT] [PATCH v8] app/dma-perf: introduce dma-perf applicationExternal Email
--------------------------------------------------------------------
-- There are many high-performance DMA devices supported in DPDK now,and
Quoted 10 lines
these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre- set config file. Memory Copy performance test
are supported for now.
Quoted 52 lines
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>---
v8:
fixed string copy issue inparse_lcore();
improved some data display format;
added doc in doc/guides/tools;
updated release notes;v7:
fixed some strcpy issues;
removed cache setup in callingrte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed someLONG_LINEwarnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removedworker_threadsparameter from config.ini; improved the logs;
improved config file;app/meson.build | 1 + app/test-dma-perf/benchmark.c | 498 +++++++++++++++++++++ app/test-dma-perf/config.ini | 61 +++ app/test-dma-perf/main.c | 594 +++++++++++++++++++++++++ app/test-dma-perf/main.h | 69 +++ app/test-dma-perf/meson.build | 17 + doc/guides/rel_notes/release_23_07.rst | 6 + doc/guides/tools/dmaperf.rst | 103 +++++ doc/guides/tools/index.rst | 1 + 9 files changed, 1350 insertions(+) create mode 100644
app/test-dma-perf/benchmark.ccreate mode 100644 app/test-dma-perf/config.ini create mode 100644 app/test-dma-perf/main.ccreate mode 100644app/test-dma-perf/main.hcreate mode
100644 app/test-dma-perf/meson.build create mode 100644doc/guides/tools/dmaperf.rst
<snip>
Quoted 49 lines
Quoted 40 lines
Quoted 40 lines
(quoted diff) +1
-0+ fprintf(stderr, "Error: Fail to find DMA %s.\n",
dma_name);app/test-dma-perf/benchmark.c new file +11
-0+ goto end; + } + + ldm->dma_ids[i] = dev_id; +configure_dmadev_queue (dev_id, ring_size); + ++nb_dmadevs; + } + +end: +if (nb_dmadevs < nb_workers) { + printf("Not enough dmadevs (%u) for all workers (%u).\n",
nb_dmadevs,nb_workers);app/test-dma-perf/benchmark.c new file +22
-0+return -1 ; + } + +printf ("Number of used dmadevs: %u.\n" , nb_dmadevs); + +return 0 ; +} + +static inline void +do_dma_submit_and_poll (uint16_t dev_id,uint64_t *async_cnt, +volatile struct worker_info *worker_info) { +int ret; +uint16_t nr_cpl; + + ret =rte_dma_submit (dev_id,0 ); +if (ret <0 ) { +rte_dma_stop (dev_id); +rte_dma_close (dev_id); +rte_exit (EXIT_FAILURE ,"Error with dma submit.\n" ); + } + + nr_cpl = rte_dma_completed(dev_id, 0, MAX_DMA_CPL_NB, NULL,NULL);
app/test-dma-perf/benchmark.c new file +2
-0+ *async_cnt -= nr_cpl; + worker_info->total_cpl += nr_cpl;Quoted 8 lines
app/test-dma-perf/benchmark.c line 214
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+} + +static inline int +do_dma_mem_copy (void *p)[Anoob] Just curious, why not pass struct
lcore_params*para itself? Is it because the pointer is volatile? If yes, then we can take an AI to split the struct into volatile and non-volatile parts.
Quoted 5 lines
[Cheng] The reason I did it this way is because I want to launch this function on another core by spawning a new thread, and
rte_eal_remote_launch()takes a void * as the parameter. That's why I passed void *p. Your suggestion to split the struct into volatile and non-volatile parts is quite reasonable. I am thinking about the best way to implement it. Thanks.
[Anoob] Instead of passing the address of index variable as void *, you can easily send lcore_params pointer, right?
Quoted 62 lines
Quoted 25 lines
Quoted 25 lines
app/test-dma-perf/benchmark.c new file +23
-0+{ +const uint16_t *para_idx = (uint16_t *)p; +volatile struct lcore_params *para = lcores_p[*para_idx].v_ptr; +volatile struct worker_info *worker_info = &(para->worker_info); +const uint16_t dev_id = para->dev_id; +const uint32_t nr_buf = para->nr_buf; +const uint16_t kick_batch = para->kick_batch; +const uint32_t buf_size = para->buf_size; +struct rte_mbuf **srcs = para->srcs; +struct rte_mbuf **dsts = para->dsts; +uint16_t nr_cpl; +uint64_t async_cnt =0 ; +uint32_t i; +uint32_t poll_cnt =0 ; +int ret; + + worker_info->stop_flag = false; + worker_info->ready_flag = true; + +while (!worker_info->start_flag) + ; + +while (1 ) {Quoted 8 lines
app/test-dma-perf/benchmark.c line 241
Quoted 5 lines
app/test-dma-perf/benchmark.c new file +3
-0+for (i =0 ; i < nr_buf; i++) { +dma_copy: + ret = rte_dma_copy(dev_id, 0,
rte_pktmbuf_iova(srcs[i]),app/test-dma-perf/benchmark.c new file +1
-0+ rte_pktmbuf_iova(dsts[i]), buf_size, 0);[Anoob] Do we need to use '
rte_mbuf_data_iova' here instead of 'rte_pktmbuf_iova'?[Cheng] yes
rte_mbuf_data_iovais more appropriate, I'll fix it in the next version. Thanks.Quoted 16 lines
app/test-dma-perf/benchmark.c line 248
Quoted 9 lines
app/test-dma-perf/benchmark.c new file +3
-0+if (unlikely (ret <0 )) { +if (ret == -ENOSPC ) { + do_dma_submit_and_poll(dev_id,
&async_cnt,worker_info);app/test-dma-perf/benchmark.c new file +4
-0+ goto dma_copy; + }else { +/* Error exit */ +rte_dma_stop (dev_id);[Anoob] Missing
rte_dma_close()here. Also, may be introduce a static void function so thatrte_exitcall etc won't be part of the fastpath loop.May be something like below and you can call it from here and "
do_dma_submit_and_poll".Quoted 8 lines
static void
error_exit(int dev_id){ /* Error exit */ rte_dma_stop(dev_id); rte_dma_close(dev_id); rte_exit(EXIT_FAILURE, "DMA enqueue failed\n"); }
Quoted 2 lines
[Cheng] I'm not so sure here.
rte_dma_close()is called in therte_exit(). Do we still call it explicitly beforerte_exit()?
[Anoob] In ' do_dma_submit_and_poll', there is rte_dma_close() before rte_exit(). I'm fine either way as long is it is consistent. Said that, I think it is better to call close() from app, rather than relying on rte_exit.
<snip>
Hi Anoob,
Replies are inline.
Thanks,
Cheng
Quoted 218 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Monday, June 26, 2023 1:42 PM
To: Jiang, Cheng1 <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>; [email protected]; Laatz, Kevin <[email protected]>; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>; Ling, WeiX <[email protected]>
Subject: RE: [EXT] [PATCH v8] app/dma-perf: introduce dma-perf applicationHi Cheng,
Please see inline.
Thanks,
AnoobQuoted 7 lines
-----Original Message-----
From: Jiang, Cheng1 <[email protected]>
Sent: Saturday, June 24, 2023 5:23 PM
To: Anoob Joseph <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; Amit Prakash Shukla <[email protected]>;Quoted 4 lines
Laatz, Kevin <[email protected]>; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; Wang,YuanX
Quoted 5 lines
<[email protected]>; He, Xingguang <[email protected]>; Ling, WeiX <[email protected]>
Subject: RE: [EXT] [PATCH v8] app/dma-perf: introduce dma-perf applicationQuoted 13 lines
Hi Anoob,
Replies are inline.
Thanks,
ChengQuoted 6 lines
-----Original Message-----
From: Anoob Joseph <[email protected]>
Sent: Friday, June 23, 2023 2:53 PM
To: Jiang, Cheng1 <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>;Amit
Quoted 108 lines
Quoted 1 line
Prakash Shukla <[email protected]>;
Quoted 4 lines
Laatz, Kevin <[email protected]>; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; Wang,YuanX
Quoted 1 line
<[email protected]>; He, Xingguang <[email protected]>;
Ling,
Quoted 4 lines
WeiX <[email protected]>
Subject: RE: [EXT] [PATCH v8] app/dma-perf: introduce dma-perf applicationQuoted 12 lines
Hi Cheng,
Thanks for the new version. Please see inline.
Thanks,
AnoobQuoted 5 lines
-----Original Message-----
From: Cheng Jiang <[email protected]>
Sent: Tuesday, June 20, 2023 12:24 PM
To: [email protected]; [email protected]; [email protected]; [email protected]; Amit PrakashShukla
Quoted 6 lines
Quoted 6 lines
<[email protected]>; Anoob Joseph <[email protected]>; [email protected]; [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected];Quoted 75 lines
Quoted 10 lines
[email protected]; Cheng Jiang <[email protected]>
Subject: [EXT] [PATCH v8] app/dma-perf: introduce dma-perf applicationExternal Email
------------------------------------------------------------------
--
-- There are many high-performance DMA devices supported in DPDK now,and
Quoted 10 lines
these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre- set config file. Memory Copy performance test
are supported for now.
Quoted 52 lines
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>---
v8:
fixed string copy issue inparse_lcore();
improved some data display format;
added doc in doc/guides/tools;
updated release notes;v7:
fixed some strcpy issues;
removed cache setup in callingrte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed someLONG_LINEwarnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removedworker_threadsparameter from config.ini; improved the logs;
improved config file;app/meson.build | 1 + app/test-dma-perf/benchmark.c | 498 +++++++++++++++++++++ app/test-dma-perf/config.ini | 61 +++ app/test-dma-perf/main.c | 594 +++++++++++++++++++++++++ app/test-dma-perf/main.h | 69 +++ app/test-dma-perf/meson.build | 17 + doc/guides/rel_notes/release_23_07.rst | 6 + doc/guides/tools/dmaperf.rst | 103 +++++ doc/guides/tools/index.rst | 1 + 9 files changed, 1350 insertions(+) create mode 100644
app/test-dma-perf/benchmark.ccreate mode 100644 app/test-dma-perf/config.ini create mode 100644 app/test-dma-perf/main.ccreate mode 100644app/test-dma-perf/main.hcreate mode
100644 app/test-dma-perf/meson.build create mode 100644doc/guides/tools/dmaperf.rst<snip>
Quoted 48 lines
Quoted 39 lines
Quoted 39 lines
(quoted diff) +1
-0+ fprintf(stderr, "Error: Fail to find DMA %s.\n",
dma_name);app/test-dma-perf/benchmark.c new file +11
-0+ goto end; + } + + ldm->dma_ids[i] = dev_id; +configure_dmadev_queue (dev_id, ring_size); + ++nb_dmadevs; + } + +end: +if (nb_dmadevs < nb_workers) { + printf("Not enough dmadevs (%u) for all workers (%u).\n",
nb_dmadevs,nb_workers);app/test-dma-perf/benchmark.c new file +22
-0+return -1 ; + } + +printf ("Number of used dmadevs: %u.\n" , nb_dmadevs); + +return 0 ; +} + +static inline void +do_dma_submit_and_poll (uint16_t dev_id,uint64_t *async_cnt, +volatile struct worker_info *worker_info) { +int ret; +uint16_t nr_cpl; + + ret =rte_dma_submit (dev_id,0 ); +if (ret <0 ) { +rte_dma_stop (dev_id); +rte_dma_close (dev_id); +rte_exit (EXIT_FAILURE ,"Error with dma submit.\n" ); + } + + nr_cpl = rte_dma_completed(dev_id, 0, MAX_DMA_CPL_NB, NULL,NULL);
app/test-dma-perf/benchmark.c new file +1
-0+ *async_cnt -= nr_cpl;Quoted 8 lines
app/test-dma-perf/benchmark.c line 214
Quoted 4 lines
app/test-dma-perf/benchmark.c new file +4
-0+ worker_info->total_cpl += nr_cpl; } + +static inline int +do_dma_mem_copy (void *p)[Anoob] Just curious, why not pass struct
lcore_params*para itself? Is it because the pointer is volatile? If yes, then we can take an AI to split the struct into volatile and non-volatile parts.Quoted 5 lines
[Cheng] The reason I did it this way is because I want to launch this function on another core by spawning a new thread, and
rte_eal_remote_launch()takes a void * as the parameter. That's why I passed void *p. Your suggestion to split the struct into volatile and non-volatile parts is quite reasonable. I am thinking about the best way to
Quoted 5 lines
implement it. Thanks.
[Anoob] Instead of passing the address of index variable as void *, you can easily send
lcore_paramspointer, right?
[Cheng] Yes, you are right. I can pass the lcore_params pointer. And I'll fix it in the next version. The new lcore_params will be non-volatile with volatile worker_info in it. This is more reasonable.
Quoted 63 lines
Quoted 34 lines
Quoted 25 lines
Quoted 25 lines
app/test-dma-perf/benchmark.c new file +23
-0+{ +const uint16_t *para_idx = (uint16_t *)p; +volatile struct lcore_params *para = lcores_p[*para_idx].v_ptr; +volatile struct worker_info *worker_info = &(para->worker_info); +const uint16_t dev_id = para->dev_id; +const uint32_t nr_buf = para->nr_buf; +const uint16_t kick_batch = para->kick_batch; +const uint32_t buf_size = para->buf_size; +struct rte_mbuf **srcs = para->srcs; +struct rte_mbuf **dsts = para->dsts; +uint16_t nr_cpl; +uint64_t async_cnt =0 ; +uint32_t i; +uint32_t poll_cnt =0 ; +int ret; + + worker_info->stop_flag = false; + worker_info->ready_flag = true; + +while (!worker_info->start_flag) + ; + +while (1 ) {Quoted 8 lines
app/test-dma-perf/benchmark.c line 241
Quoted 5 lines
app/test-dma-perf/benchmark.c new file +3
-0+for (i =0 ; i < nr_buf; i++) { +dma_copy: + ret = rte_dma_copy(dev_id, 0,
rte_pktmbuf_iova(srcs[i]),app/test-dma-perf/benchmark.c new file +1
-0+ rte_pktmbuf_iova(dsts[i]), buf_size, 0);[Anoob] Do we need to use '
rte_mbuf_data_iova' here instead of 'rte_pktmbuf_iova'?Quoted 15 lines
[Cheng] yes
rte_mbuf_data_iovais more appropriate, I'll fix it in the next version. Thanks.Quoted 12 lines
app/test-dma-perf/benchmark.c line 248
Quoted 9 lines
app/test-dma-perf/benchmark.c new file +3
-0+if (unlikely (ret <0 )) { +if (ret == -ENOSPC ) { + do_dma_submit_and_poll(dev_id,
&async_cnt,worker_info);app/test-dma-perf/benchmark.c new file +4
-0+ goto dma_copy; + }else { +/* Error exit */ +rte_dma_stop (dev_id);[Anoob] Missing
rte_dma_close()here. Also, may be introduce a static void function so thatrte_exitcall etc won't be part of the fastpathloop.
Quoted 13 lines
Quoted 12 lines
May be something like below and you can call it from here and "
do_dma_submit_and_poll".static void
error_exit(int dev_id){ /* Error exit */ rte_dma_stop(dev_id); rte_dma_close(dev_id); rte_exit(EXIT_FAILURE, "DMA enqueue failed\n"); }
Quoted 7 lines
Quoted 2 lines
[Cheng] I'm not so sure here.
rte_dma_close()is called in therte_exit(). Do we still call it explicitly beforerte_exit()?[Anoob] In '
do_dma_submit_and_poll', there isrte_dma_close()beforerte_exit(). I'm fine either way as long is it is consistent. Said that, I think it is better to callclose()from app, rather than relying onrte_exit.
[Cheng] sure, it makes sense to me that app should call rte_dma_close(), I'll fix it in the next version. Thanks.
Quoted 1 line
<snip>
PATCH v9 1/1 app/dma-perf: introduce dma-perf application
In reply to Cheng JiangThere are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are supported for now.
---
v9:
improved error handling;
improved lcore_params structure;
improved mbuf api calling;
improved exit process;
fixed some typos;
added scenario summary data display;
removed unnecessary include;
v8:
fixed string copy issue in parse_lcore(); improved some data display format;
added doc in doc/guides/tools;
updated release notes;
v7:
fixed some strcpy issues;
removed cache setup in calling rte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed some LONG_LINE warnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removed worker_threads parameter from config.ini; improved the logs;
improved config file;
app/meson.build | 1 + app/test-dma-perf/benchmark.c | 508 ++++++++++++++++++++ app/test-dma-perf/config.ini | 61 +++ app/test-dma-perf/main.c | 616 +++++++++++++++++++++++++ app/test-dma-perf/main.h | 64 +++ app/test-dma-perf/meson.build | 17 + doc/guides/rel_notes/release_23_07.rst | 6 + doc/guides/tools/dmaperf.rst | 103 +++++ doc/guides/tools/index.rst | 1 + 9 files changed, 1377 insertions(+) create mode 100644 app/test-dma-perf/benchmark.c create mode 100644 app/test-dma-perf/config.ini create mode 100644 app/test-dma-perf/main.c create mode 100644 app/test-dma-perf/main.h create mode 100644 app/test-dma-perf/meson.build create mode 100644 doc/guides/tools/dmaperf.rst
app/meson.build +1 -0
@@ -19,6 +19,7 @@ apps = [ 'test-cmdline', 'test-compress-perf', 'test-crypto-perf', + 'test-dma-perf', 'test-eventdev', 'test-fib', 'test-flow-perf',
app/test-dma-perf/benchmark.c new file +508 -0
508 lines not on this page · raw message
app/test-dma-perf/config.ini new file +61 -0
61 lines not on this page · raw message
app/test-dma-perf/main.c new file +616 -0
616 lines not on this page · raw message
app/test-dma-perf/main.h new file +64 -0
64 lines not on this page · raw message
app/test-dma-perf/meson.build new file +17 -0
@@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019-2023 Intel Corporation + +# meson file, for building this app as part of a main DPDK build. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +deps += ['dmadev', 'mbuf', 'cfgfile'] + +sources = files( + 'main.c', + 'benchmark.c', +)
doc/guides/rel_notes/release_23_07.rst +6 -0
@@ -170,6 +170,12 @@ New Features See :doc:`../prog_guide/pdcp_lib` for more information. +* **Added DMA device performance test application.** + + Added an new application to test the performance of DMA device and CPU. + + See the :doc:`../tools/dmaperf` for more details. + Removed Items -------------
doc/guides/tools/dmaperf.rst new file +103 -0
103 lines not on this page · raw message
doc/guides/tools/index.rst +1 -0
@@ -23,3 +23,4 @@ DPDK Tools User Guides testregex testmldev dts + dmaperf
--
2.40.1
PATCH v10 1/1 app/dma-perf: introduce dma-perf application
In reply to Cheng JiangThere are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are supported for now.
---
v10:
rebased code from 23.07-rc2;
v9:
improved error handling;
improved lcore_params structure;
improved mbuf api calling;
improved exit process;
fixed some typos;
added scenario summary data display;
removed unnecessary include;
v8:
fixed string copy issue in parse_lcore(); improved some data display format;
added doc in doc/guides/tools;
updated release notes;
v7:
fixed some strcpy issues;
removed cache setup in calling rte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed some LONG_LINE warnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removed worker_threads parameter from config.ini; improved the logs;
improved config file;
app/meson.build | 1 + app/test-dma-perf/benchmark.c | 508 ++++++++++++++++++++ app/test-dma-perf/config.ini | 61 +++ app/test-dma-perf/main.c | 616 +++++++++++++++++++++++++ app/test-dma-perf/main.h | 64 +++ app/test-dma-perf/meson.build | 17 + doc/guides/rel_notes/release_23_07.rst | 6 + doc/guides/tools/dmaperf.rst | 103 +++++ doc/guides/tools/index.rst | 1 + 9 files changed, 1377 insertions(+) create mode 100644 app/test-dma-perf/benchmark.c create mode 100644 app/test-dma-perf/config.ini create mode 100644 app/test-dma-perf/main.c create mode 100644 app/test-dma-perf/main.h create mode 100644 app/test-dma-perf/meson.build create mode 100644 doc/guides/tools/dmaperf.rst
app/meson.build +1 -0
@@ -19,6 +19,7 @@ apps = [ 'test-cmdline', 'test-compress-perf', 'test-crypto-perf', + 'test-dma-perf', 'test-eventdev', 'test-fib', 'test-flow-perf',
app/test-dma-perf/benchmark.c new file +508 -0
508 lines not on this page · raw message
app/test-dma-perf/config.ini new file +61 -0
61 lines not on this page · raw message
app/test-dma-perf/main.c new file +616 -0
616 lines not on this page · raw message
app/test-dma-perf/main.h new file +64 -0
64 lines not on this page · raw message
app/test-dma-perf/meson.build new file +17 -0
@@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019-2023 Intel Corporation + +# meson file, for building this app as part of a main DPDK build. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +deps += ['dmadev', 'mbuf', 'cfgfile'] + +sources = files( + 'main.c', + 'benchmark.c', +)
doc/guides/rel_notes/release_23_07.rst +6 -0
@@ -200,6 +200,12 @@ New Features Enhanced the GRO library to support TCP packets over IPv6 network. +* **Added DMA device performance test application.** + + Added an new application to test the performance of DMA device and CPU. + + See the :doc:`../tools/dmaperf` for more details. + Removed Items -------------
doc/guides/tools/dmaperf.rst new file +103 -0
103 lines not on this page · raw message
doc/guides/tools/index.rst +1 -0
@@ -23,3 +23,4 @@ DPDK Tools User Guides testregex testmldev dts + dmaperf
--
2.40.1
Quoted 65 lines
There are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre- set config file. Memory Copy performance test are supported for now.
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>---
v10:
rebased code from 23.07-rc2;
v9:
improved error handling;
improvedlcore_paramsstructure;
improved mbuf api calling;
improved exit process;
fixed some typos;
added scenario summary data display;
removed unnecessary include;
v8:
fixed string copy issue inparse_lcore(); improved some data display format;
added doc in doc/guides/tools;
updated release notes;
v7:
fixed some strcpy issues;
removed cache setup in callingrte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed someLONG_LINEwarnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removedworker_threadsparameter from config.ini; improved the logs;
improved config file;app/meson.build | 1 + app/test-dma-perf/benchmark.c | 508 ++++++++++++++++++++ app/test-dma-perf/config.ini | 61 +++ app/test-dma-perf/main.c | 616 +++++++++++++++++++++++++ app/test-dma-perf/main.h | 64 +++ app/test-dma-perf/meson.build | 17 + doc/guides/rel_notes/release_23_07.rst | 6 + doc/guides/tools/dmaperf.rst | 103 +++++ doc/guides/tools/index.rst | 1 + 9 files changed, 1377 insertions(+) create mode 100644 app/test-dma-perf/benchmark.c create mode 100644
Quoted 5 lines
app/test-dma-perf/config.ini create mode 100644 app/test-dma-
perf/main.ccreate mode 100644app/test-dma-perf/main.hcreate mode 100644 app/test-dma-perf/meson.build create mode 100644doc/guides/tools/dmaperf.rst
Thanks Cheng for addressing all the comments.
Quoted 79 lines
-----Original Message-----
From: Jiang, Cheng1 <[email protected]>
Sent: Wednesday, June 28, 2023 9:21 AM
To: [email protected]; Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; [email protected]; [email protected]; [email protected]; Laatz, Kevin <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>; Ling, WeiX <[email protected]>; Jiang, Cheng1 <[email protected]>
Subject: [PATCH v10] app/dma-perf: introduce dma-perf applicationThere are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre- set config file. Memory Copy performance test are supported for now.
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>---
v10:
rebased code from 23.07-rc2;
v9:
improved error handling;
improvedlcore_paramsstructure;
improved mbuf api calling;
improved exit process;
fixed some typos;
added scenario summary data display;
removed unnecessary include;
v8:
fixed string copy issue inparse_lcore(); improved some data display format;
added doc in doc/guides/tools;
updated release notes;
v7:
fixed some strcpy issues;
removed cache setup in callingrte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed someLONG_LINEwarnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removedworker_threadsparameter from config.ini; improved the logs;
improved config file;app/meson.build | 1 + app/test-dma-perf/benchmark.c | 508 ++++++++++++++++++++ app/test-dma-perf/config.ini | 61 +++ app/test-dma-perf/main.c | 616 +++++++++++++++++++++++++ app/test-dma-perf/main.h | 64 +++ app/test-dma-perf/meson.build | 17 + doc/guides/rel_notes/release_23_07.rst | 6 + doc/guides/tools/dmaperf.rst | 103 +++++ doc/guides/tools/index.rst | 1 + 9 files changed, 1377 insertions(+) create mode 100644 app/test-dma-perf/benchmark.c create mode 100644
Quoted 5 lines
app/test-dma-perf/config.ini create mode 100644 app/test-dma-
perf/main.ccreate mode 100644app/test-dma-perf/main.hcreate mode 100644 app/test-dma-perf/meson.build create mode 100644doc/guides/tools/dmaperf.rst
28/06/2023 03:20, Cheng Jiang:
Quoted 11 lines
There are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are supported for now.
Quoted 5 lines
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>
Who is going to be the maintainer for this new app? An entry in the file MAINTAINERS would be perfect.
Hi Thomas,
Quoted 28 lines
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Thursday, June 29, 2023 5:09 PM
To: Hu, Jiayu <[email protected]>; Wang, YuanX <[email protected]>; Jiang, Cheng1 <[email protected]>
Cc: Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; [email protected]; [email protected]; [email protected]; Laatz, Kevin <[email protected]>; [email protected]; [email protected]; [email protected]; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; He, Xingguang <[email protected]>; Ling, WeiX <[email protected]>
Subject: Re: [PATCH v10] app/dma-perf: introduce dma-perf application28/06/2023 03:20, Cheng Jiang:
Quoted 1 line
There are many high-performance DMA devices supported in DPDK now,
and
Quoted 9 lines
these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are
supported for now.
Quoted 1 line
Quoted 9 lines
Quoted 5 lines
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>Who is going to be the maintainer for this new app? An entry in the file
MAINTAINERSwould be perfect.
I can be the maintainer for this new app. Okay, I will add an entry to the MAINTAINERS file. Would you prefer I send a new version patch to add this entry, or send a separate patch?
Thanks,
Cheng
Quoted 1 line
29/06/2023 14:50, Jiang, Cheng1:
Quoted 21 lines
From: Thomas Monjalon <[email protected]>
Quoted 20 lines
28/06/2023 03:20, Cheng Jiang:
Quoted 1 line
There are many high-performance DMA devices supported in DPDK now,
and
Quoted 9 lines
these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are
supported for now.
Quoted 6 lines
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>
Quoted 6 lines
Quoted 3 lines
Who is going to be the maintainer for this new app? An entry in the file
MAINTAINERSwould be perfect.I can be the maintainer for this new app. Okay, I will add an entry to the
MAINTAINERSfile. Would you prefer I send a new version patch to add this entry, or send a separate patch?
It should be part of this patch.
Quoted 44 lines
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Thursday, June 29, 2023 9:19 PM
To: Hu, Jiayu <[email protected]>; Wang, YuanX <[email protected]>; Jiang, Cheng1 <[email protected]>
Cc: Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; [email protected]; [email protected]; [email protected]; Laatz, Kevin <[email protected]>; [email protected]; [email protected]; [email protected]; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; He, Xingguang <[email protected]>; Ling, WeiX <[email protected]>
Subject: Re: [PATCH v10] app/dma-perf: introduce dma-perf application29/06/2023 14:50, Jiang, Cheng1:
Quoted 3 lines
From: Thomas Monjalon <[email protected]>
Quoted 2 lines
28/06/2023 03:20, Cheng Jiang:
Quoted 1 line
There are many high-performance DMA devices supported in DPDK
now,
Quoted 23 lines
Quoted 22 lines
and
Quoted 10 lines
these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are
supported for now.
Quoted 6 lines
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>Who is going to be the maintainer for this new app? An entry in the file
MAINTAINERSwould be perfect.
Quoted 8 lines
Quoted 1 line
I can be the maintainer for this new app. Okay, I will add an entry to the
MAINTAINERSfile.Quoted 1 line
Would you prefer I send a new version patch to add this entry, or send a
separate patch?
It should be part of this patch.
Sure, I'll submit a new patch. Thanks.
Quoted 2 lines
28/06/2023 03:20, Cheng Jiang:
doc/guides/tools/index.rst DPDK Tools User Guides · line 26
Quoted 7 lines
doc/guides/tools/index.rst +1
-0@@ -23,3 +23,4 @@ DPDK Tools User Guides testregex testmldev dts + dmaperf
Would be better to have upper in the list (DTS should be last). I suggest before flow-perf as DMA is more low-level.
Hi Thomas,
Quoted 15 lines
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Thursday, June 29, 2023 5:38 PM
To: Jiang, Cheng1 <[email protected]>
Cc: Richardson, Bruce <[email protected]>; [email protected]; Xia, Chenbo <[email protected]>; [email protected]; [email protected]; [email protected]; Laatz, Kevin <[email protected]>; [email protected]; [email protected]; [email protected]; Hu, Jiayu <[email protected]>; Ding, Xuan <[email protected]>; Ma, WenwuX <[email protected]>; Wang, YuanX <[email protected]>; He, Xingguang <[email protected]>; Ling, WeiX <[email protected]>
Subject: Re: [PATCH v10] app/dma-perf: introduce dma-perf application28/06/2023 03:20, Cheng Jiang:
Quoted 11 lines
doc/guides/tools/index.rst DPDK Tools User Guides · line 26
Quoted 7 lines
doc/guides/tools/index.rst +1
-0@@ -23,3 +23,4 @@ DPDK Tools User Guides testregex testmldev dts + dmaperfWould be better to have upper in the list (DTS should be last). I suggest before flow-perf as DMA is more low-level.
Sure, I'll submit a new patch to fix it.
Thanks,
Cheng
PATCH v11 1/1 app/dma-perf: introduce dma-perf application
In reply to Cheng JiangThere are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are supported for now. This patch also updates the documentation and maintainer list for the application.
---
v11:
updated MAINTAINERS file;
fixed typo in doc/guides/tools/dmaperf.rst; changed the dmaperf order in doc/guides/tools/index.rst;
v10:
rebased code from 23.07-rc2;
v9:
improved error handling;
improved lcore_params structure;
improved mbuf api calling;
improved exit process;
fixed some typos;
added scenario summary data display;
removed unnecessary include;
v8:
fixed string copy issue in parse_lcore(); improved some data display format;
added doc in doc/guides/tools;
updated release notes;
v7:
fixed some strcpy issues;
removed cache setup in calling rte_pktmbuf_pool_create(); fixed some typos;
added some memory free and null set operations; improved result calculation;
v6:
improved code based on Anoob's comments;
fixed some code structure issues;
v5:
fixed some LONG_LINE warnings;
v4:
fixed inaccuracy of the memory footprint display;
v3:
fixed some typos;
v2:
added lcore/dmadev designation;
added error case process;
removed worker_threads parameter from config.ini; improved the logs;
improved config file;
MAINTAINERS | 5 + app/meson.build | 1 + app/test-dma-perf/benchmark.c | 508 ++++++++++++++++++++ app/test-dma-perf/config.ini | 61 +++ app/test-dma-perf/main.c | 616 +++++++++++++++++++++++++ app/test-dma-perf/main.h | 64 +++ app/test-dma-perf/meson.build | 17 + doc/guides/rel_notes/release_23_07.rst | 6 + doc/guides/tools/dmaperf.rst | 103 +++++ doc/guides/tools/index.rst | 1 + 10 files changed, 1382 insertions(+) create mode 100644 app/test-dma-perf/benchmark.c create mode 100644 app/test-dma-perf/config.ini create mode 100644 app/test-dma-perf/main.c create mode 100644 app/test-dma-perf/main.h create mode 100644 app/test-dma-perf/meson.build create mode 100644 doc/guides/tools/dmaperf.rst
MAINTAINERS +5 -0
@@ -1767,6 +1767,11 @@ T: git://dpdk.org/next/dpdk-next-net F: app/test-pmd/ F: doc/guides/testpmd_app_ug/ +DMA device performance tool +M: Cheng Jiang <[email protected]> +F: app/test-dma-perf/ +F: doc/guides/tools/dmaperf.rst + Flow performance tool M: Wisam Jaddo <[email protected]> F: app/test-flow-perf/
app/meson.build +1 -0
@@ -19,6 +19,7 @@ apps = [ 'test-cmdline', 'test-compress-perf', 'test-crypto-perf', + 'test-dma-perf', 'test-eventdev', 'test-fib', 'test-flow-perf',
app/test-dma-perf/benchmark.c new file +508 -0
508 lines not on this page · raw message
app/test-dma-perf/config.ini new file +61 -0
61 lines not on this page · raw message
app/test-dma-perf/main.c new file +616 -0
616 lines not on this page · raw message
app/test-dma-perf/main.h new file +64 -0
64 lines not on this page · raw message
app/test-dma-perf/meson.build new file +17 -0
@@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2019-2023 Intel Corporation + +# meson file, for building this app as part of a main DPDK build. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +deps += ['dmadev', 'mbuf', 'cfgfile'] + +sources = files( + 'main.c', + 'benchmark.c', +)
doc/guides/rel_notes/release_23_07.rst +6 -0
@@ -200,6 +200,12 @@ New Features Enhanced the GRO library to support TCP packets over IPv6 network. +* **Added DMA device performance test application.** + + Added an new application to test the performance of DMA device and CPU. + + See the :doc:`../tools/dmaperf` for more details. + Removed Items -------------
doc/guides/tools/dmaperf.rst new file +103 -0
103 lines not on this page · raw message
doc/guides/tools/index.rst +1 -0
@@ -14,6 +14,7 @@ DPDK Tools User Guides pmdinfo dumpcap pdump + dmaperf flow-perf securityperf testbbdev
--
2.40.1
Quoted 21 lines · Cheng Jiang
There are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are supported for now. This patch also updates the documentation and maintainer list for the application.
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>Acked-by Anoob Joseph <[email protected]>Tested-by Anoob Joseph <[email protected]>Tested-by Wei Ling <[email protected]>
...
Quoted 3 lines
.
Quoted 22 lines
Quoted 13 lines
There are many high-performance DMA devices supported in DPDK now, and these DMA devices can also be integrated into other modules of DPDK as accelerators, such as Vhost. Before integrating DMA into applications, developers need to know the performance of these DMA devices in various scenarios and the performance of CPUs in the same scenario, such as different buffer lengths. Only in this way can we know the target performance of the application accelerated by using them. This patch introduces a high-performance testing tool, which supports comparing the performance of CPU and DMA in different scenarios automatically with a pre-set config file. Memory Copy performance test are supported for now. This patch also updates the documentation and maintainer list for the application.
Quoted 8 lines
Signed-off-by Cheng Jiang <[email protected]>Signed-off-by Jiayu Hu <[email protected]>Signed-off-by Yuan Wang <[email protected]>Acked-by Morten Brørup <[email protected]>Acked-by Chenbo Xia <[email protected]>Acked-by Anoob Joseph <[email protected]>Tested-by Anoob Joseph <[email protected]>Tested-by Wei Ling <[email protected]>
Quoted 1 line
Acked-by Chengwen Feng <[email protected]>
Applied with few improvements in the documentation, thanks.