Skip to contentSkip to conversation

dmadev: introduce DMA device library

PATCH Superseded Chengwen Feng 17 participants 339 messages Jul 2, 2021 – Oct 17, 2021 Page 4 of 4

Subsystems (3)
lore
Share & tools
lore
#255 lore raw

v21 cover support dmadev

In reply to Chengwen Feng

This patch set contains seven patch for new add dmadev.

Chengwen Feng (7):
dmadev: introduce DMA device library public APIs
dmadev: introduce DMA device library internal header
dmadev: introduce DMA device library PMD header
dmadev: introduce DMA device library implementation
doc: add DMA device library guide dma/skeleton: introduce skeleton dmadev driver app/test: add dmadev API test

---
v21:
* add comment for reserved fields of struct rte_dmadev. v20:
* delete unnecessary and duplicate include header files.
* the conf_sz parameter is added to the configure and vchan-setup callbacks of the PMD, this is mainly used to enhance ABI compatibility.
* the rte_dmadev structure field is rearranged to reserve more space for I/O functions.
* fix some ambiguous and unnecessary comments.
* fix the potential memory leak of ut.
* redefine skeldma_init_once to skeldma_count.
* suppress rte_dmadev error output when execute ut.
v19:
* squash maintainer patch to patch #1.
v18:
* RTE_DMA_STATUS_* add BUS_READ/WRITE_ERR, PAGE_FAULT.
* rte_dmadev dataplane API add judge dev_started when debug enable.
* rte_dmadev_start/vchan_setup add judge device configured.
* rte_dmadev_dump support format capability name.
* optimized the comments of rte_dmadev.
* fix skeldma_copy always return zero when enqueue successful.
* log encapsulation macro add newline characters.
* test_dmadev_api support rte_dmadev_dump() ut.

 MAINTAINERS                            |    7 +
 app/test/meson.build                   |    4 +
 app/test/test_dmadev.c                 |   43 +
 app/test/test_dmadev_api.c             |  543 ++++++++++++
 config/rte_config.h                    |    3 +
 doc/api/doxy-api-index.md              |    1 +
 doc/api/doxy-api.conf.in               |    1 +
 doc/guides/prog_guide/dmadev.rst       |  125 +++
 doc/guides/prog_guide/img/dmadev.svg   |  283 +++++++
 doc/guides/prog_guide/index.rst        |    1 +
 doc/guides/rel_notes/release_21_11.rst |    5 +
 drivers/dma/meson.build                |   11 +
 drivers/dma/skeleton/meson.build       |    7 +
 drivers/dma/skeleton/skeleton_dmadev.c |  594 ++++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |   61 ++
 drivers/dma/skeleton/version.map       |    3 +
 drivers/meson.build                    |    1 +
 lib/dmadev/meson.build                 |    7 +
 lib/dmadev/rte_dmadev.c                |  607 ++++++++++++++
 lib/dmadev/rte_dmadev.h                | 1047 ++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  187 +++++
 lib/dmadev/rte_dmadev_pmd.h            |   72 ++
 lib/dmadev/version.map                 |   35 +
 lib/meson.build                        |    1 +
 24 files changed, 3649 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
Signature · 1 line
2.33.0
#256 lore raw

PATCH v21 1/7 dmadev: introduce DMA device library public APIs

The 'dmadevice' is a generic type of DMA device.

This patch introduce the 'dmadevice' public APIs which expose generic operations that can enable configuration and I/O with the DMA devices.

Maintainers update is also included in this patch.

---
 MAINTAINERS                            |   4 +
 doc/api/doxy-api-index.md              |   1 +
 doc/api/doxy-api.conf.in               |   1 +
 doc/guides/rel_notes/release_21_11.rst |   5 +
 lib/dmadev/meson.build                 |   4 +
 lib/dmadev/rte_dmadev.h                | 951 +++++++++++++++++++++++++
 lib/dmadev/version.map                 |  24 +
 lib/meson.build                        |   1 +
 8 files changed, 991 insertions(+)
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/version.map
MAINTAINERS +4 -0
index 1e0d303394..9885cc56b7 100644
@@ -496,6 +496,10 @@ F: drivers/raw/skeleton/
 F: app/test/test_rawdev.c
 F: doc/guides/prog_guide/rawdev.rst
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <[email protected]>
+F: lib/dmadev/
+
 
 Memory Pool Drivers
 -------------------
doc/api/doxy-api-index.md +1 -0
index 1992107a03..ce08250639 100644
@@ -27,6 +27,7 @@ The public API headers are grouped by topics:
   [event_timer_adapter]    (@ref rte_event_timer_adapter.h),
   [event_crypto_adapter]   (@ref rte_event_crypto_adapter.h),
   [rawdev]             (@ref rte_rawdev.h),
+  [dmadev]             (@ref rte_dmadev.h),
   [metrics]            (@ref rte_metrics.h),
   [bitrate]            (@ref rte_bitrate.h),
   [latency]            (@ref rte_latencystats.h),
doc/api/doxy-api.conf.in +1 -0
index 325a0195c6..a44a92b5fe 100644
@@ -34,6 +34,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/cmdline \
                           @TOPDIR@/lib/compressdev \
                           @TOPDIR@/lib/cryptodev \
+                          @TOPDIR@/lib/dmadev \
                           @TOPDIR@/lib/distributor \
                           @TOPDIR@/lib/efd \
                           @TOPDIR@/lib/ethdev \
doc/guides/rel_notes/release_21_11.rst +5 -0
index 675b573834..3562822b3d 100644
@@ -62,6 +62,11 @@ New Features
   * Added bus-level parsing of the devargs syntax.
   * Kept compatibility with the legacy syntax as parsing fallback.
 
+* **Added dmadev library support.**
+
+  The dmadev library provides a DMA device framework for management and
+  provision of hardware and software DMA devices.
+
 
 Removed Items
 -------------
lib/dmadev/meson.build new file +4 -0
new file mode 100644
index 0000000000..6d5bd85373
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited.
+
+headers = files('rte_dmadev.h')
lib/dmadev/rte_dmadev.h new file +951 -0

951 lines not on this page · raw message

lib/dmadev/version.map new file +24 -0
new file mode 100644
index 0000000000..2e37882364
@@ -0,0 +1,24 @@
+EXPERIMENTAL {
+	global:
+
+	rte_dmadev_close;
+	rte_dmadev_completed;
+	rte_dmadev_completed_status;
+	rte_dmadev_configure;
+	rte_dmadev_copy;
+	rte_dmadev_copy_sg;
+	rte_dmadev_count;
+	rte_dmadev_dump;
+	rte_dmadev_fill;
+	rte_dmadev_get_dev_id;
+	rte_dmadev_info_get;
+	rte_dmadev_is_valid_dev;
+	rte_dmadev_start;
+	rte_dmadev_stats_get;
+	rte_dmadev_stats_reset;
+	rte_dmadev_stop;
+	rte_dmadev_submit;
+	rte_dmadev_vchan_setup;
+
+	local: *;
+};
lib/meson.build +1 -0
index 1673ca4323..a542c238d2 100644
@@ -44,6 +44,7 @@ libraries = [
         'power',
         'pdump',
         'rawdev',
+        'dmadev',
         'regexdev',
         'rib',
         'reorder',
Signature · 1 line
2.33.0
#257 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

Hi,

I am having a surface look at the API.
I hope we can do better than previous libs.

07/09/2021 14:56, Chengwen Feng:

MAINTAINERS F: drivers/raw/skeleton/ · line 501

Quoted 9 lines
MAINTAINERS +3 -0
@@ -496,6 +496,10 @@ F: drivers/raw/skeleton/
 F: app/test/test_rawdev.c
 F: doc/guides/prog_guide/rawdev.rst
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <[email protected]>
+F: lib/dmadev/

I think it should before (preferably) or after eventdev, but let's keep rawdev as the last one.

Then please apply the same order in other non-alphabetical lists (doc, meson, etc).

MAINTAINERS F: drivers/raw/skeleton/

Quoted 6 lines
MAINTAINERS +6 -0
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited.
+ * Copyright(c) 2021 Intel Corporation.
+ * Copyright(c) 2021 Marvell International Ltd.
+ * Copyright(c) 2021 SmartShare Systems.
+ */

No need for final dot in copyright lines.

MAINTAINERS

Quoted 2 lines
MAINTAINERS +2 -0
+#ifndef _RTE_DMADEV_H_
+#define _RTE_DMADEV_H_

No need for surrounding underscores.

MAINTAINERS

Quoted 5 lines
MAINTAINERS +5 -0
+
+/**
+ * @file rte_dmadev.h
+ *
+ * RTE DMA (Direct Memory Access) device APIs.

RTE has no meaning when used in a sentence.
And given it is a DPDK library, you don't really need to specify. I would also remove the final "s" as the library is one interface.

Quoted 15 lines

+ *
+ * The DMA framework is built on the following model:
+ *

+ *     ---------------   ---------------       ---------------
+ *     | virtual DMA |   | virtual DMA |       | virtual DMA |
+ *     | channel     |   | channel     |       | channel     |
+ *     ---------------   ---------------       ---------------
+ *            |                |                      |
+ *            ------------------                      |
+ *                     |                              |
+ *               ------------                    ------------
+ *               |  dmadev  |                    |  dmadev  |
+ *               ------------                    ------------
+ *                     |                              |
+ *            ------------------               ------------------
Quoted 8 lines
+ *            | HW-DMA-channel |               | HW-DMA-channel |
+ *            ------------------               ------------------
+ *                     |                              |
+ *                     --------------------------------
+ *                                     |
+ *                           ---------------------
+ *                           | HW-DMA-Controller |
+ *                           ---------------------

You don't hyphens between the words I think.

Quoted 9 lines

+ *
+ * The DMA controller could have multiple HW-DMA-channels (aka. HW-DMA-queues),
+ * each HW-DMA-channel should be represented by a dmadev.
+ *
+ * The dmadev could create multiple virtual DMA channels, each virtual DMA
+ * channel represents a different transfer context. The DMA operation request
+ * must be submitted to the virtual DMA channel. e.g. Application could create
+ * virtual DMA channel 0 for memory-to-memory transfer scenario, and create
+ * virtual DMA channel 1 for memory-to-device transfer scenario.

What is the benefit of virtual channels compared to have separate dmadevs for the same HW channel?

Quoted 3 lines

+ *
+ * The dmadev are dynamically allocated by rte_dmadev_pmd_allocate() during the
+ * PCI/SoC device probing phase performed at EAL initialization time. And could

Not clear what is this phase. Do you mean bus probing?

Quoted 2 lines

+ * be released by rte_dmadev_pmd_release() during the PCI/SoC device removing
+ * phase.

Again what is this phase?
I think freeing should be done only via the "close" function.

Quoted 3 lines

+ *
+ * This framework uses 'uint16_t dev_id' as the device identifier of a dmadev,
+ * and 'uint16_t vchan' as the virtual DMA channel identifier in one dmadev.

I think it is better to use signed int16_t so you can express "none" in the API, which can simplify some functions and error management.

Quoted 8 lines

+ *
+ * The functions exported by the dmadev API to setup a device designated by its
+ * device identifier must be invoked in the following order:

+ *     - rte_dmadev_configure()
+ *     - rte_dmadev_vchan_setup()
+ *     - rte_dmadev_start()

+ *
+ * Then, the application can invoke dataplane APIs to process jobs.

You mean "dataplane functions".

Quoted 5 lines

+ *
+ * If the application wants to change the configuration (i.e. invoke
+ * rte_dmadev_configure() or rte_dmadev_vchan_setup()), it must invoke
+ * rte_dmadev_stop() first to stop the device and then do the reconfiguration
+ * before invoking rte_dmadev_start() again. The dataplane APIs should not be

again, APIs -> functions

Quoted 11 lines

+ * invoked when the device is stopped.
+ *
+ * Finally, an application can close a dmadev by invoking the
+ * rte_dmadev_close() function.
+ *
+ * The dataplane APIs include two parts:
+ * The first part is the submission of operation requests:

+ *     - rte_dmadev_copy()
+ *     - rte_dmadev_copy_sg()
+ *     - rte_dmadev_fill()
+ *     - rte_dmadev_submit()
Quoted 7 lines

+ *
+ * These APIs could work with different virtual DMA channels which have
+ * different contexts.
+ *
+ * The first three APIs are used to submit the operation request to the virtual
+ * DMA channel, if the submission is successful, an uint16_t ring_idx is
+ * returned, otherwise a negative number is returned.

unsigned or negative? looks weird.

Quoted 4 lines

+ *
+ * The last API was used to issue doorbell to hardware, and also there are flags
+ * (@see RTE_DMA_OP_FLAG_SUBMIT) parameter of the first three APIs could do the
+ * same work.

I don't understand this sentence.
You mean rte_dmadev_submit function?
Why past tense "was"?
Why having a redundant function?

Quoted 9 lines

+ *
+ * The second part is to obtain the result of requests:
+ * - rte_dmadev_completed()
+ * - return the number of operation requests completed successfully.
+ * - rte_dmadev_completed_status()
+ * - return the number of operation requests completed.
+ *
+ * @note The two completed APIs also support return the last completed
+ * operation's ring_idx.

Not sure why this note here.

Quoted 5 lines

+ * @note If the dmadev works in silent mode (@see RTE_DMADEV_CAPA_SILENT),
+ * application does not invoke the above two completed APIs.
+ *
+ * About the ring_idx which enqueue APIs (e.g. rte_dmadev_copy()
+ * rte_dmadev_fill()) returned, the rules are as follows:

I feel a word is missing above.

Quoted 20 lines

+ * - ring_idx for each virtual DMA channel are independent.
+ * - For a virtual DMA channel, the ring_idx is monotonically incremented,
+ * when it reach UINT16_MAX, it wraps back to zero.
+ * - This ring_idx can be used by applications to track per-operation
+ * metadata in an application-defined circular ring.
+ * - The initial ring_idx of a virtual DMA channel is zero, after the
+ * device is stopped, the ring_idx needs to be reset to zero.
+ *
+ * One example:

+ *     - step-1: start one dmadev
+ *     - step-2: enqueue a copy operation, the ring_idx return is 0
+ *     - step-3: enqueue a copy operation again, the ring_idx return is 1
+ *     - ...
+ *     - step-101: stop the dmadev
+ *     - step-102: start the dmadev
+ *     - step-103: enqueue a copy operation, the ring_idx return is 0
+ *     - ...
+ *     - step-x+0: enqueue a fill operation, the ring_idx return is 65535
+ *     - step-x+1: enqueue a copy operation, the ring_idx return is 0
+ *     - ...
Quoted 3 lines

+ *
+ * The DMA operation address used in enqueue APIs (i.e. rte_dmadev_copy(),
+ * rte_dmadev_copy_sg(), rte_dmadev_fill()) defined as rte_iova_t type. The

"is" defined

Quoted 1 line

+ * dmadev supports two types of address: memory address and device address.

Please try to start new sentences on a new line.

Quoted 4 lines

+ *
+ * - memory address: the source and destination address of the memory-to-memory
+ * transfer type, or the source address of the memory-to-device transfer type,
+ * or the destination address of the device-to-memory transfer type.

Quoted 8 lines

+ * @note If the device support SVA (@see RTE_DMADEV_CAPA_SVA), the memory
+ * address can be any VA address, otherwise it must be an IOVA address.
+ *
+ * - device address: the source and destination address of the device-to-device
+ * transfer type, or the source address of the device-to-memory transfer type,
+ * or the destination address of the memory-to-device transfer type.
+ *
+ * By default, all the functions of the dmadev API exported by a PMD are

What do you mean "by default"? Is there some exceptions?

Quoted 7 lines

+ * lock-free functions which assume to not be invoked in parallel on different
+ * logical cores to work on the same target dmadev object.
+ * @note Different virtual DMA channels on the same dmadev *DO NOT* support
+ * parallel invocation because these virtual DMA channels share the same
+ * HW-DMA-channel.
+ *
+ */

[...]

Quoted 1 line

+#define RTE_DMADEV_NAME_MAX_LEN RTE_DEV_NAME_MAX_LEN

Why not using RTE_DEV_NAME_MAX_LEN directly?
If you keep it, it should be commented, explaining whether it takes '\0' into account or not.

MAINTAINERS

Quoted 3 lines
MAINTAINERS +3 -0
+__rte_experimental
+bool
+rte_dmadev_is_valid_dev(uint16_t dev_id);

I would suggest dropping the final "_dev" in the function name.

MAINTAINERS

Quoted 2 lines
MAINTAINERS +2 -0
+uint16_t
+rte_dmadev_count(void);

It would be safer to name it rte_dmadev_count_avail in case we need new kind of device count later.

MAINTAINERS

Quoted 2 lines
MAINTAINERS +2 -0
+
+/* Enumerates DMA device capabilities. */

You should group them with a doxygen group syntax.
See https://patches.dpdk.org/project/dpdk/patch/[email protected]/

Quoted 1 line

+#define RTE_DMADEV_CAPA_MEM_TO_MEM (1ull << 0)

Please use RTE_BIT macro (32 or 64).

Quoted 4 lines

+/**< DMA device support memory-to-memory transfer.
+ *
+ * @see struct rte_dmadev_info::dev_capa
+ */

It is preferred to have documentation before the item.

[...]

MAINTAINERS

Quoted 5 lines
MAINTAINERS +5 -0
+/**
+ * A structure used to retrieve the information of a DMA device.
+ */
+struct rte_dmadev_info {
+	struct rte_device *device; /**< Generic Device information. */

Please do not expose this.

Quoted 6 lines
MAINTAINERS +6 -0
+	uint64_t dev_capa; /**< Device capabilities (RTE_DMADEV_CAPA_*). */
+	uint16_t max_vchans;
+	/**< Maximum number of virtual DMA channels supported. */
+	uint16_t max_desc;
+	/**< Maximum allowed number of virtual DMA channel descriptors. */
+	uint16_t min_desc;

MAINTAINERS

Quoted 10 lines
MAINTAINERS +10 -0
+	/**< Minimum allowed number of virtual DMA channel descriptors. */
+	uint16_t max_sges;
+	/**< Maximum number of source or destination scatter-gather entry
+	 * supported.
+	 * If the device does not support COPY_SG capability, this value can be
+	 * zero.
+	 * If the device supports COPY_SG capability, then rte_dmadev_copy_sg()
+	 * parameter nb_src/nb_dst should not exceed this value.
+	 */
+	uint16_t nb_vchans; /**< Number of virtual DMA channel configured. */

What about adding NUMA node?

    /* Local NUMA memory ID. -1 if unknown. */
    int16_t numa_node;

MAINTAINERS

Quoted 4 lines
MAINTAINERS +3 -0
+};
 
+int
+rte_dmadev_info_get(uint16_t dev_id, struct rte_dmadev_info *dev_info);

In .h files, the return type should not be on a separate line.

MAINTAINERS

Quoted 4 lines
MAINTAINERS +4 -0
+
+/**
+ * A structure used to configure a DMA device.
+ */

You should mention where it is used with @see.

Quoted 15 lines
MAINTAINERS +15 -0
+struct rte_dmadev_conf {
+	uint16_t nb_vchans;
+	/**< The number of virtual DMA channels to set up for the DMA device.
+	 * This value cannot be greater than the field 'max_vchans' of struct
+	 * rte_dmadev_info which get from rte_dmadev_info_get().
+	 */
+	bool enable_silent;
+	/**< Indicates whether to enable silent mode.
+	 * false-default mode, true-silent mode.
+	 * This value can be set to true only when the SILENT capability is
+	 * supported.
+	 *
+	 * @see RTE_DMADEV_CAPA_SILENT
+	 */
+};

[...]

Quoted 1 line

+#define RTE_DMADEV_ALL_VCHAN 0xFFFFu

Please do not add this kind of constant without a doxygen comment.

It seems you don't manage the maximum number of devices. It is fixed in the .c file:

	struct rte_dmadev rte_dmadevices[RTE_DMADEV_MAX_DEVS];

Instead I would suggest a more dynamic approach with an init function, so the application can extend it before calling rte_eal_init. Please see how it is implemented here:
https://patches.dpdk.org/project/dpdk/patch/[email protected]/

Above patch could also inspire you to start some docs in this patch.

This series add one file per patch.
Instead it would be better to have groups of features per patch, meaning the implementation and the driver interface should be in the same patch.
You can split like this:
1/ device allocation
2/ configuration and start/stop
3/ dataplane functions

I would suggest 2 more patches:
4/ event notification
see https://patches.dpdk.org/project/dpdk/patch/[email protected]/
5/ multi-process
see https://patches.dpdk.org/project/dpdk/patch/[email protected]/

Thanks for the work

#258 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

Quoted 5 lines · Thomas Monjalon

Hi,

I am having a surface look at the API.
I hope we can do better than previous libs.

A few bits of feedback on your comments and the API below.

/Bruce

Quoted 137 lines

07/09/2021 14:56, Chengwen Feng:

MAINTAINERS F: drivers/raw/skeleton/ · line 501

Quoted 9 lines
MAINTAINERS +3 -0
@@ -496,6 +496,10 @@ F: drivers/raw/skeleton/
 F: app/test/test_rawdev.c
 F: doc/guides/prog_guide/rawdev.rst
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <[email protected]>
+F: lib/dmadev/

I think it should before (preferably) or after eventdev, but let's keep rawdev as the last one.

Then please apply the same order in other non-alphabetical lists (doc, meson, etc).

MAINTAINERS F: drivers/raw/skeleton/

Quoted 6 lines
MAINTAINERS +6 -0
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited.
+ * Copyright(c) 2021 Intel Corporation.
+ * Copyright(c) 2021 Marvell International Ltd.
+ * Copyright(c) 2021 SmartShare Systems.
+ */

No need for final dot in copyright lines.

MAINTAINERS

Quoted 2 lines
MAINTAINERS +2 -0
+#ifndef _RTE_DMADEV_H_
+#define _RTE_DMADEV_H_

No need for surrounding underscores.

MAINTAINERS

Quoted 5 lines
MAINTAINERS +5 -0
+
+/**
+ * @file rte_dmadev.h
+ *
+ * RTE DMA (Direct Memory Access) device APIs.

RTE has no meaning when used in a sentence.
And given it is a DPDK library, you don't really need to specify. I would also remove the final "s" as the library is one interface.

Quoted 15 lines

+ *
+ * The DMA framework is built on the following model:
+ *

+ *     ---------------   ---------------       ---------------
+ *     | virtual DMA |   | virtual DMA |       | virtual DMA |
+ *     | channel     |   | channel     |       | channel     |
+ *     ---------------   ---------------       ---------------
+ *            |                |                      |
+ *            ------------------                      |
+ *                     |                              |
+ *               ------------                    ------------
+ *               |  dmadev  |                    |  dmadev  |
+ *               ------------                    ------------
+ *                     |                              |
+ *            ------------------               ------------------
Quoted 8 lines
+ *            | HW-DMA-channel |               | HW-DMA-channel |
+ *            ------------------               ------------------
+ *                     |                              |
+ *                     --------------------------------
+ *                                     |
+ *                           ---------------------
+ *                           | HW-DMA-Controller |
+ *                           ---------------------

You don't hyphens between the words I think.

Quoted 9 lines

+ *
+ * The DMA controller could have multiple HW-DMA-channels (aka. HW-DMA-queues),
+ * each HW-DMA-channel should be represented by a dmadev.
+ *
+ * The dmadev could create multiple virtual DMA channels, each virtual DMA
+ * channel represents a different transfer context. The DMA operation request
+ * must be submitted to the virtual DMA channel. e.g. Application could create
+ * virtual DMA channel 0 for memory-to-memory transfer scenario, and create
+ * virtual DMA channel 1 for memory-to-device transfer scenario.

What is the benefit of virtual channels compared to have separate dmadevs for the same HW channel?

Quoted 3 lines

+ *
+ * The dmadev are dynamically allocated by rte_dmadev_pmd_allocate() during the
+ * PCI/SoC device probing phase performed at EAL initialization time. And could

Not clear what is this phase. Do you mean bus probing?

Quoted 2 lines

+ * be released by rte_dmadev_pmd_release() during the PCI/SoC device removing
+ * phase.

Again what is this phase?
I think freeing should be done only via the "close" function.

Quoted 3 lines

+ *
+ * This framework uses 'uint16_t dev_id' as the device identifier of a dmadev,
+ * and 'uint16_t vchan' as the virtual DMA channel identifier in one dmadev.

I think it is better to use signed int16_t so you can express "none" in the API, which can simplify some functions and error management.

Quoted 8 lines

+ *
+ * The functions exported by the dmadev API to setup a device designated by its
+ * device identifier must be invoked in the following order:

+ *     - rte_dmadev_configure()
+ *     - rte_dmadev_vchan_setup()
+ *     - rte_dmadev_start()

+ *
+ * Then, the application can invoke dataplane APIs to process jobs.

You mean "dataplane functions".

Quoted 5 lines

+ *
+ * If the application wants to change the configuration (i.e. invoke
+ * rte_dmadev_configure() or rte_dmadev_vchan_setup()), it must invoke
+ * rte_dmadev_stop() first to stop the device and then do the reconfiguration
+ * before invoking rte_dmadev_start() again. The dataplane APIs should not be

again, APIs -> functions

Quoted 11 lines

+ * invoked when the device is stopped.
+ *
+ * Finally, an application can close a dmadev by invoking the
+ * rte_dmadev_close() function.
+ *
+ * The dataplane APIs include two parts:
+ * The first part is the submission of operation requests:

+ *     - rte_dmadev_copy()
+ *     - rte_dmadev_copy_sg()
+ *     - rte_dmadev_fill()
+ *     - rte_dmadev_submit()
Quoted 7 lines

+ *
+ * These APIs could work with different virtual DMA channels which have
+ * different contexts.
+ *
+ * The first three APIs are used to submit the operation request to the virtual
+ * DMA channel, if the submission is successful, an uint16_t ring_idx is
+ * returned, otherwise a negative number is returned.

Quoted 2 lines

unsigned or negative? looks weird.

May be, but it works well. We could perhaps rephase to make it less weird though:
"if the submission is successful, a positive ring_idx <= UINT16_MAX is
returned, otherwise a negative number is returned."

Quoted 10 lines
Quoted 4 lines

+ *
+ * The last API was used to issue doorbell to hardware, and also there are flags
+ * (@see RTE_DMA_OP_FLAG_SUBMIT) parameter of the first three APIs could do the
+ * same work.

I don't understand this sentence.
You mean rte_dmadev_submit function?
Why past tense "was"?
Why having a redundant function?

Just because there are two ways to do something does not mean that one of them is redundant, as both may be more suitable for different situations. When enqueuing a set of jobs to the device, having a separate submit outside a loop makes for clearer code than having a check for the last iteration inside the loop to set a special submit flag. However, for cases where one item alone is to be submitted or there is a small set of jobs to be submitted sequentially, having a submit flag provides a lower-overhead way of doing the submission while still keeping the code clean.

Quoted 79 lines
Quoted 9 lines

+ *
+ * The second part is to obtain the result of requests:
+ * - rte_dmadev_completed()
+ * - return the number of operation requests completed successfully.
+ * - rte_dmadev_completed_status()
+ * - return the number of operation requests completed.
+ *
+ * @note The two completed APIs also support return the last completed
+ * operation's ring_idx.

Not sure why this note here.

Quoted 5 lines

+ * @note If the dmadev works in silent mode (@see RTE_DMADEV_CAPA_SILENT),
+ * application does not invoke the above two completed APIs.
+ *
+ * About the ring_idx which enqueue APIs (e.g. rte_dmadev_copy()
+ * rte_dmadev_fill()) returned, the rules are as follows:

I feel a word is missing above.

Quoted 20 lines

+ * - ring_idx for each virtual DMA channel are independent.
+ * - For a virtual DMA channel, the ring_idx is monotonically incremented,
+ * when it reach UINT16_MAX, it wraps back to zero.
+ * - This ring_idx can be used by applications to track per-operation
+ * metadata in an application-defined circular ring.
+ * - The initial ring_idx of a virtual DMA channel is zero, after the
+ * device is stopped, the ring_idx needs to be reset to zero.
+ *
+ * One example:

+ *     - step-1: start one dmadev
+ *     - step-2: enqueue a copy operation, the ring_idx return is 0
+ *     - step-3: enqueue a copy operation again, the ring_idx return is 1
+ *     - ...
+ *     - step-101: stop the dmadev
+ *     - step-102: start the dmadev
+ *     - step-103: enqueue a copy operation, the ring_idx return is 0
+ *     - ...
+ *     - step-x+0: enqueue a fill operation, the ring_idx return is 65535
+ *     - step-x+1: enqueue a copy operation, the ring_idx return is 0
+ *     - ...
Quoted 3 lines

+ *
+ * The DMA operation address used in enqueue APIs (i.e. rte_dmadev_copy(),
+ * rte_dmadev_copy_sg(), rte_dmadev_fill()) defined as rte_iova_t type. The

"is" defined

Quoted 1 line

+ * dmadev supports two types of address: memory address and device address.

Please try to start new sentences on a new line.

Quoted 4 lines

+ *
+ * - memory address: the source and destination address of the memory-to-memory
+ * transfer type, or the source address of the memory-to-device transfer type,
+ * or the destination address of the device-to-memory transfer type.

Quoted 8 lines

+ * @note If the device support SVA (@see RTE_DMADEV_CAPA_SVA), the memory
+ * address can be any VA address, otherwise it must be an IOVA address.
+ *
+ * - device address: the source and destination address of the device-to-device
+ * transfer type, or the source address of the device-to-memory transfer type,
+ * or the destination address of the memory-to-device transfer type.
+ *
+ * By default, all the functions of the dmadev API exported by a PMD are

What do you mean "by default"? Is there some exceptions?

Quoted 7 lines

+ * lock-free functions which assume to not be invoked in parallel on different
+ * logical cores to work on the same target dmadev object.
+ * @note Different virtual DMA channels on the same dmadev *DO NOT* support
+ * parallel invocation because these virtual DMA channels share the same
+ * HW-DMA-channel.
+ *
+ */

[...]

Quoted 1 line

+#define RTE_DMADEV_NAME_MAX_LEN RTE_DEV_NAME_MAX_LEN

Why not using RTE_DEV_NAME_MAX_LEN directly?
If you keep it, it should be commented, explaining whether it takes '\0' into account or not.

Quoted 6 lines

MAINTAINERS

Quoted 3 lines
MAINTAINERS +3 -0
+__rte_experimental
+bool
+rte_dmadev_is_valid_dev(uint16_t dev_id);

I would suggest dropping the final "_dev" in the function name.

The alternative, which I would support, is replacing "rte_dmadev" with "rte_dma" across the API. This would then become "rte_dma_is_valid_dev" which is clearer, since the dev is not part of the standard prefix. It also would fit in with a possible future function of "rte_dma_is_valid_vchan" for instance.

Quoted 7 lines

MAINTAINERS

Quoted 3 lines
MAINTAINERS +2 -0
+uint16_t
+rte_dmadev_count(void);

It would be safer to name it rte_dmadev_count_avail in case we need new kind of device count later.

If we change "dmadev" to "dma" this could then be
"rte_dma_count_avail_dev".

Quoted 46 lines

MAINTAINERS

Quoted 2 lines
MAINTAINERS +2 -0
+
+/* Enumerates DMA device capabilities. */

You should group them with a doxygen group syntax.
See https://patches.dpdk.org/project/dpdk/patch/[email protected]/

Quoted 1 line

+#define RTE_DMADEV_CAPA_MEM_TO_MEM (1ull << 0)

Please use RTE_BIT macro (32 or 64).

Quoted 4 lines

+/**< DMA device support memory-to-memory transfer.
+ *
+ * @see struct rte_dmadev_info::dev_capa
+ */

It is preferred to have documentation before the item.

[...]

MAINTAINERS

Quoted 5 lines
MAINTAINERS +5 -0
+/**
+ * A structure used to retrieve the information of a DMA device.
+ */
+struct rte_dmadev_info {
+	struct rte_device *device; /**< Generic Device information. */

Please do not expose this.

Quoted 6 lines
MAINTAINERS +6 -0
+	uint64_t dev_capa; /**< Device capabilities (RTE_DMADEV_CAPA_*). */
+	uint16_t max_vchans;
+	/**< Maximum number of virtual DMA channels supported. */
+	uint16_t max_desc;
+	/**< Maximum allowed number of virtual DMA channel descriptors. */
+	uint16_t min_desc;

MAINTAINERS

Quoted 10 lines
MAINTAINERS +10 -0
+	/**< Minimum allowed number of virtual DMA channel descriptors. */
+	uint16_t max_sges;
+	/**< Maximum number of source or destination scatter-gather entry
+	 * supported.
+	 * If the device does not support COPY_SG capability, this value can be
+	 * zero.
+	 * If the device supports COPY_SG capability, then rte_dmadev_copy_sg()
+	 * parameter nb_src/nb_dst should not exceed this value.
+	 */
+	uint16_t nb_vchans; /**< Number of virtual DMA channel configured. */

What about adding NUMA node?

Quoted 3 lines
    /* Local NUMA memory ID. -1 if unknown. */
    int16_t numa_node;

That was omitted as it could be got through the device structure. If device is removed, we need to ensure all fields needed from device, such as numa node, are made available here.

Quoted 5 lines
Quoted 4 lines
MAINTAINERS +3 -0
+};
 
+int
+rte_dmadev_info_get(uint16_t dev_id, struct rte_dmadev_info *dev_info);

<snip>

#259 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

09/09/2021 13:18, Bruce Richardson:

Quoted 6 lines · Thomas Monjalon
Quoted 5 lines

07/09/2021 14:56, Chengwen Feng:

Quoted 3 lines

+ * The first three APIs are used to submit the operation request to the virtual
+ * DMA channel, if the submission is successful, an uint16_t ring_idx is
+ * returned, otherwise a negative number is returned.

Quoted 6 lines
Quoted 1 line

unsigned or negative? looks weird.

May be, but it works well. We could perhaps rephase to make it less weird though:
"if the submission is successful, a positive ring_idx <= UINT16_MAX is
returned, otherwise a negative number is returned."

I am advocating for int16_t,
it makes a lot of things simpler.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

+ *
+ * The last API was used to issue doorbell to hardware, and also there are flags
+ * (@see RTE_DMA_OP_FLAG_SUBMIT) parameter of the first three APIs could do the
+ * same work.

Quoted 8 lines
Quoted 5 lines

I don't understand this sentence.
You mean rte_dmadev_submit function?
Why past tense "was"?
Why having a redundant function?

Just because there are two ways to do something does not mean that one of them is redundant, as both may be more suitable for different situations.

I agree.

Quoted 6 lines

When enqueuing a set of jobs to the device, having a separate submit outside a loop makes for clearer code than having a check for the last iteration inside the loop to set a special submit flag. However, for cases where one item alone is to be submitted or there is a small set of jobs to be submitted sequentially, having a submit flag provides a lower-overhead way of doing the submission while still keeping the code clean.

This kind of explanation may be missing in doxygen?

Quoted 11 lines
Quoted 5 lines

lib/dmadev/rte_dmadev.h line 189

Quoted 2 lines
lib/dmadev/rte_dmadev.h new file +2 -0
+bool
+rte_dmadev_is_valid_dev(uint16_t dev_id);

I would suggest dropping the final "_dev" in the function name.

The alternative, which I would support, is replacing "rte_dmadev" with "rte_dma" across the API. This would then become "rte_dma_is_valid_dev" which is clearer, since the dev is not part of the standard prefix. It also would fit in with a possible future function of "rte_dma_is_valid_vchan" for instance.

Yes
The question is whether it would make sense to reserver rte_dma_ prefix for some DMA functions which would be outside of dmadev lib? If you think that all DMA functions will be in dmadev,
then yes we can shorten the prefix to rte_dma_.

Quoted 9 lines
Quoted 6 lines

lib/dmadev/rte_dmadev.h line 203

Quoted 2 lines
lib/dmadev/rte_dmadev.h new file +2 -0
+uint16_t
+rte_dmadev_count(void);

It would be safer to name it rte_dmadev_count_avail in case we need new kind of device count later.

If we change "dmadev" to "dma" this could then be
"rte_dma_count_avail_dev".

Yes

Quoted 27 lines
Quoted 27 lines

lib/dmadev/rte_dmadev.h line 275

Quoted 5 lines
lib/dmadev/rte_dmadev.h new file +5 -0
+/**
+ * A structure used to retrieve the information of a DMA device.
+ */
+struct rte_dmadev_info {
+	struct rte_device *device; /**< Generic Device information. */

Please do not expose this.

Quoted 6 lines
lib/dmadev/rte_dmadev.h new file +6 -0
+	uint64_t dev_capa; /**< Device capabilities (RTE_DMADEV_CAPA_*). */
+	uint16_t max_vchans;
+	/**< Maximum number of virtual DMA channels supported. */
+	uint16_t max_desc;
+	/**< Maximum allowed number of virtual DMA channel descriptors. */
+	uint16_t min_desc;

lib/dmadev/rte_dmadev.h line 291

Quoted 10 lines
lib/dmadev/rte_dmadev.h new file +10 -0
+	/**< Minimum allowed number of virtual DMA channel descriptors. */
+	uint16_t max_sges;
+	/**< Maximum number of source or destination scatter-gather entry
+	 * supported.
+	 * If the device does not support COPY_SG capability, this value can be
+	 * zero.
+	 * If the device supports COPY_SG capability, then rte_dmadev_copy_sg()
+	 * parameter nb_src/nb_dst should not exceed this value.
+	 */
+	uint16_t nb_vchans; /**< Number of virtual DMA channel configured. */

What about adding NUMA node?

Quoted 7 lines
Quoted 3 lines
    /* Local NUMA memory ID. -1 if unknown. */
    int16_t numa_node;

That was omitted as it could be got through the device structure. If device is removed, we need to ensure all fields needed from device, such as numa node, are made available here.

Ah yes, forgot about rte_device :)
Yes please remove rte_device from this struct.

#260 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

Quoted 9 lines · Thomas Monjalon

09/09/2021 13:18, Bruce Richardson:

Quoted 8 lines · Thomas Monjalon
Quoted 6 lines

07/09/2021 14:56, Chengwen Feng:

Quoted 3 lines

+ * The first three APIs are used to submit the operation request to the virtual
+ * DMA channel, if the submission is successful, an uint16_t ring_idx is
+ * returned, otherwise a negative number is returned.

unsigned or negative? looks weird.

Quoted 8 lines
Quoted 4 lines

May be, but it works well. We could perhaps rephase to make it less weird though:
"if the submission is successful, a positive ring_idx <= UINT16_MAX is
returned, otherwise a negative number is returned."

I am advocating for int16_t,
it makes a lot of things simpler.

No, it doesn't work as you can't have wrap-around of the IDs once you use signed values - and that impacts both the end app and the internals of the drivers. Let's keep it as-is otherwise it will have massive impacts - including potential perf impacts.

Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

+ *
+ * The last API was used to issue doorbell to hardware, and also there are flags
+ * (@see RTE_DMA_OP_FLAG_SUBMIT) parameter of the first three APIs could do the
+ * same work.

Quoted 8 lines
Quoted 5 lines

I don't understand this sentence.
You mean rte_dmadev_submit function?
Why past tense "was"?
Why having a redundant function?

Just because there are two ways to do something does not mean that one of them is redundant, as both may be more suitable for different situations.

I agree.

Quoted 9 lines
Quoted 6 lines

When enqueuing a set of jobs to the device, having a separate submit outside a loop makes for clearer code than having a check for the last iteration inside the loop to set a special submit flag. However, for cases where one item alone is to be submitted or there is a small set of jobs to be submitted sequentially, having a submit flag provides a lower-overhead way of doing the submission while still keeping the code clean.

This kind of explanation may be missing in doxygen?

It can be added, sure.

Quoted 12 lines
Quoted 11 lines
Quoted 5 lines

lib/dmadev/rte_dmadev.h line 189

Quoted 2 lines
lib/dmadev/rte_dmadev.h new file +2 -0
+bool
+rte_dmadev_is_valid_dev(uint16_t dev_id);

I would suggest dropping the final "_dev" in the function name.

The alternative, which I would support, is replacing "rte_dmadev" with "rte_dma" across the API. This would then become "rte_dma_is_valid_dev" which is clearer, since the dev is not part of the standard prefix. It also would fit in with a possible future function of "rte_dma_is_valid_vchan" for instance.

Quoted 6 lines

Yes
The question is whether it would make sense to reserver rte_dma_ prefix for some DMA functions which would be outside of dmadev lib? If you think that all DMA functions will be in dmadev,
then yes we can shorten the prefix to rte_dma_.

Well, any DPDK dma functions which are not in dma library should have the prefix of the library they are in e.g. rte_eal_dma_*, rte_pci_dma_* Therefore, I don't think name conflicts should be an issue, and I like having less typing to do in function names (and I believe Morten was strongly proposing this previously too)

Quoted 50 lines
Quoted 9 lines
Quoted 6 lines

lib/dmadev/rte_dmadev.h line 203

Quoted 2 lines
lib/dmadev/rte_dmadev.h new file +2 -0
+uint16_t
+rte_dmadev_count(void);

It would be safer to name it rte_dmadev_count_avail in case we need new kind of device count later.

If we change "dmadev" to "dma" this could then be
"rte_dma_count_avail_dev".

Yes

Quoted 27 lines
Quoted 27 lines

lib/dmadev/rte_dmadev.h line 275

Quoted 5 lines
lib/dmadev/rte_dmadev.h new file +5 -0
+/**
+ * A structure used to retrieve the information of a DMA device.
+ */
+struct rte_dmadev_info {
+	struct rte_device *device; /**< Generic Device information. */

Please do not expose this.

Quoted 6 lines
lib/dmadev/rte_dmadev.h new file +6 -0
+	uint64_t dev_capa; /**< Device capabilities (RTE_DMADEV_CAPA_*). */
+	uint16_t max_vchans;
+	/**< Maximum number of virtual DMA channels supported. */
+	uint16_t max_desc;
+	/**< Maximum allowed number of virtual DMA channel descriptors. */
+	uint16_t min_desc;

lib/dmadev/rte_dmadev.h line 291

Quoted 10 lines
lib/dmadev/rte_dmadev.h new file +10 -0
+	/**< Minimum allowed number of virtual DMA channel descriptors. */
+	uint16_t max_sges;
+	/**< Maximum number of source or destination scatter-gather entry
+	 * supported.
+	 * If the device does not support COPY_SG capability, this value can be
+	 * zero.
+	 * If the device supports COPY_SG capability, then rte_dmadev_copy_sg()
+	 * parameter nb_src/nb_dst should not exceed this value.
+	 */
+	uint16_t nb_vchans; /**< Number of virtual DMA channel configured. */

What about adding NUMA node?

Quoted 7 lines
Quoted 3 lines
    /* Local NUMA memory ID. -1 if unknown. */
    int16_t numa_node;

That was omitted as it could be got through the device structure. If device is removed, we need to ensure all fields needed from device, such as numa node, are made available here.

Ah yes, forgot about rte_device :)
Yes please remove rte_device from this struct.

#261 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

Quoted 71 lines · Bruce Richardson
Quoted 9 lines · Thomas Monjalon

09/09/2021 13:18, Bruce Richardson:

Quoted 8 lines · Thomas Monjalon
Quoted 6 lines

07/09/2021 14:56, Chengwen Feng:

Quoted 3 lines

+ * The first three APIs are used to submit the operation request to the virtual
+ * DMA channel, if the submission is successful, an uint16_t ring_idx is
+ * returned, otherwise a negative number is returned.

unsigned or negative? looks weird.

Quoted 8 lines
Quoted 4 lines

May be, but it works well. We could perhaps rephase to make it less weird though:
"if the submission is successful, a positive ring_idx <= UINT16_MAX is
returned, otherwise a negative number is returned."

I am advocating for int16_t,
it makes a lot of things simpler.

No, it doesn't work as you can't have wrap-around of the IDs once you use signed values - and that impacts both the end app and the internals of the drivers. Let's keep it as-is otherwise it will have massive impacts - including potential perf impacts.

Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

+ *
+ * The last API was used to issue doorbell to hardware, and also there are flags
+ * (@see RTE_DMA_OP_FLAG_SUBMIT) parameter of the first three APIs could do the
+ * same work.

Quoted 8 lines
Quoted 5 lines

I don't understand this sentence.
You mean rte_dmadev_submit function?
Why past tense "was"?
Why having a redundant function?

Just because there are two ways to do something does not mean that one of them is redundant, as both may be more suitable for different situations.

I agree.

Quoted 9 lines
Quoted 6 lines

When enqueuing a set of jobs to the device, having a separate submit outside a loop makes for clearer code than having a check for the last iteration inside the loop to set a special submit flag. However, for cases where one item alone is to be submitted or there is a small set of jobs to be submitted sequentially, having a submit flag provides a lower-overhead way of doing the submission while still keeping the code clean.

This kind of explanation may be missing in doxygen?

It can be added, sure.

Quoted 12 lines
Quoted 11 lines
Quoted 5 lines

lib/dmadev/rte_dmadev.h line 189

Quoted 2 lines
lib/dmadev/rte_dmadev.h new file +2 -0
+bool
+rte_dmadev_is_valid_dev(uint16_t dev_id);

I would suggest dropping the final "_dev" in the function name.

The alternative, which I would support, is replacing "rte_dmadev" with "rte_dma" across the API. This would then become "rte_dma_is_valid_dev" which is clearer, since the dev is not part of the standard prefix. It also would fit in with a possible future function of "rte_dma_is_valid_vchan" for instance.

Quoted 6 lines

Yes
The question is whether it would make sense to reserver rte_dma_ prefix for some DMA functions which would be outside of dmadev lib? If you think that all DMA functions will be in dmadev,
then yes we can shorten the prefix to rte_dma_.

Quoted 5 lines

Well, any DPDK dma functions which are not in dma library should have the prefix of the library they are in e.g. rte_eal_dma_*, rte_pci_dma_* Therefore, I don't think name conflicts should be an issue, and I like having less typing to do in function names (and I believe Morten was strongly proposing this previously too)

The dmadev is rather short, if change I prefer all public API with rte_dma_ prefix, and don't have rte_dma_dev_ prefix for such start/stop/close. (ps: the rte_eth_ also have rte_eth_dev_close which is painful for OCD).

Also should the filename(e.g. rte_dmadev.h) and directory-name(lib/dmadev) also change ?

Quoted 53 lines
Quoted 44 lines
Quoted 10 lines
Quoted 7 lines

lib/dmadev/rte_dmadev.h line 203

Quoted 3 lines
lib/dmadev/rte_dmadev.h new file +2 -0
+uint16_t
+rte_dmadev_count(void);

It would be safer to name it rte_dmadev_count_avail in case we need new kind of device count later.

If we change "dmadev" to "dma" this could then be
"rte_dma_count_avail_dev".

Yes

Quoted 31 lines
Quoted 30 lines

lib/dmadev/rte_dmadev.h line 275

Quoted 5 lines
lib/dmadev/rte_dmadev.h new file +5 -0
+/**
+ * A structure used to retrieve the information of a DMA device.
+ */
+struct rte_dmadev_info {
+	struct rte_device *device; /**< Generic Device information. */

Please do not expose this.

Quoted 6 lines
lib/dmadev/rte_dmadev.h new file +6 -0
+	uint64_t dev_capa; /**< Device capabilities (RTE_DMADEV_CAPA_*). */
+	uint16_t max_vchans;
+	/**< Maximum number of virtual DMA channels supported. */
+	uint16_t max_desc;
+	/**< Maximum allowed number of virtual DMA channel descriptors. */
+	uint16_t min_desc;

lib/dmadev/rte_dmadev.h line 291

Quoted 10 lines
lib/dmadev/rte_dmadev.h new file +10 -0
+	/**< Minimum allowed number of virtual DMA channel descriptors. */
+	uint16_t max_sges;
+	/**< Maximum number of source or destination scatter-gather entry
+	 * supported.
+	 * If the device does not support COPY_SG capability, this value can be
+	 * zero.
+	 * If the device supports COPY_SG capability, then rte_dmadev_copy_sg()
+	 * parameter nb_src/nb_dst should not exceed this value.
+	 */
+	uint16_t nb_vchans; /**< Number of virtual DMA channel configured. */

What about adding NUMA node?

    /* Local NUMA memory ID. -1 if unknown. */
    int16_t numa_node;
Quoted 7 lines
Quoted 3 lines

That was omitted as it could be got through the device structure. If device is removed, we need to ensure all fields needed from device, such as numa node, are made available here.

Ah yes, forgot about rte_device :)
Yes please remove rte_device from this struct.

.

#262 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

09/09/2021 15:54, fengchengwen:

Quoted 16 lines · Bruce Richardson
Quoted 15 lines · Thomas Monjalon
Quoted 14 lines

09/09/2021 13:18, Bruce Richardson:

Quoted 12 lines · Thomas Monjalon
Quoted 6 lines

07/09/2021 14:56, Chengwen Feng:

Quoted 3 lines

+ * The first three APIs are used to submit the operation request to the virtual
+ * DMA channel, if the submission is successful, an uint16_t ring_idx is
+ * returned, otherwise a negative number is returned.

unsigned or negative? looks weird.

May be, but it works well. We could perhaps rephase to make it less weird though:
"if the submission is successful, a positive ring_idx <= UINT16_MAX is
returned, otherwise a negative number is returned."

Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

I am advocating for int16_t,
it makes a lot of things simpler.

No, it doesn't work as you can't have wrap-around of the IDs once you use signed values - and that impacts both the end app and the internals of the drivers. Let's keep it as-is otherwise it will have massive impacts - including potential perf impacts.

Not sure to understand what you mean.
Please could you explain what does not work and what is the perf impact? I guess you want unsigned index for rings, then OK.
For device ID however, I believe signed integer is useful.

[...]

Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
Quoted 10 lines
Quoted 4 lines

lib/dmadev/rte_dmadev.h line 189

Quoted 2 lines
lib/dmadev/rte_dmadev.h new file +2 -0
+bool
+rte_dmadev_is_valid_dev(uint16_t dev_id);

I would suggest dropping the final "_dev" in the function name.

The alternative, which I would support, is replacing "rte_dmadev" with "rte_dma" across the API. This would then become "rte_dma_is_valid_dev" which is clearer, since the dev is not part of the standard prefix. It also would fit in with a possible future function of "rte_dma_is_valid_vchan" for instance.

Yes
The question is whether it would make sense to reserver rte_dma_ prefix for some DMA functions which would be outside of dmadev lib? If you think that all DMA functions will be in dmadev,
then yes we can shorten the prefix to rte_dma_.

Quoted 2 lines
Quoted 2 lines

Well, any DPDK dma functions which are not in dma library should have the prefix of the library they are in e.g. rte_eal_dma_*, rte_pci_dma_*

Quite often, we skip the eal_ prefix, that's why I was thinking about a possible namespace conflict. Anyway it could be managed.

Quoted 7 lines
Quoted 3 lines

Therefore, I don't think name conflicts should be an issue, and I like having less typing to do in function names (and I believe Morten was strongly proposing this previously too)

The dmadev is rather short, if change I prefer all public API with rte_dma_ prefix, and don't have rte_dma_dev_ prefix for such start/stop/close. (ps: the rte_eth_ also have rte_eth_dev_close which is painful for OCD).

Yes OK for rte_dma_ prefix everywhere.

Quoted 1 line

Also should the filename(e.g. rte_dmadev.h) and directory-name(lib/dmadev) also change ?

I believe it's better to keep dmadev as name of the lib and filename, so it's consistent with other device classes.
What are the other opinions?

#263 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

Quoted 20 lines · Thomas Monjalon

09/09/2021 15:54, fengchengwen:

Quoted 19 lines · Bruce Richardson
Quoted 18 lines · Thomas Monjalon
Quoted 16 lines

09/09/2021 13:18, Bruce Richardson:

Quoted 6 lines · Thomas Monjalon
Quoted 5 lines

07/09/2021 14:56, Chengwen Feng:

Quoted 3 lines

+ * The first three APIs are used to submit the operation request to the virtual
+ * DMA channel, if the submission is successful, an uint16_t ring_idx is
+ * returned, otherwise a negative number is returned.

Quoted 6 lines
Quoted 1 line

unsigned or negative? looks weird.

May be, but it works well. We could perhaps rephase to make it less weird though:
"if the submission is successful, a positive ring_idx <= UINT16_MAX is
returned, otherwise a negative number is returned."

I am advocating for int16_t,
it makes a lot of things simpler.

Quoted 8 lines
Quoted 4 lines
Quoted 4 lines

No, it doesn't work as you can't have wrap-around of the IDs once you use signed values - and that impacts both the end app and the internals of the drivers. Let's keep it as-is otherwise it will have massive impacts - including potential perf impacts.

Not sure to understand what you mean.
Please could you explain what does not work and what is the perf impact? I guess you want unsigned index for rings, then OK.

Yes, that is it.

Quoted 1 line

For device ID however, I believe signed integer is useful.

No objection to that.

Quoted 34 lines

[...]

Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
Quoted 10 lines
Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+bool
+rte_dmadev_is_valid_dev(uint16_t dev_id);

I would suggest dropping the final "_dev" in the function name.

The alternative, which I would support, is replacing "rte_dmadev" with "rte_dma" across the API. This would then become "rte_dma_is_valid_dev" which is clearer, since the dev is not part of the standard prefix. It also would fit in with a possible future function of "rte_dma_is_valid_vchan" for instance.

Yes
The question is whether it would make sense to reserver rte_dma_ prefix for some DMA functions which would be outside of dmadev lib? If you think that all DMA functions will be in dmadev,
then yes we can shorten the prefix to rte_dma_.

Quoted 2 lines
Quoted 2 lines

Well, any DPDK dma functions which are not in dma library should have the prefix of the library they are in e.g. rte_eal_dma_*, rte_pci_dma_*

Quite often, we skip the eal_ prefix, that's why I was thinking about a possible namespace conflict. Anyway it could be managed.

Quoted 7 lines
Quoted 3 lines

Therefore, I don't think name conflicts should be an issue, and I like having less typing to do in function names (and I believe Morten was strongly proposing this previously too)

The dmadev is rather short, if change I prefer all public API with rte_dma_ prefix, and don't have rte_dma_dev_ prefix for such start/stop/close. (ps: the rte_eth_ also have rte_eth_dev_close which is painful for OCD).

Quoted 7 lines

Yes OK for rte_dma_ prefix everywhere.

Quoted 1 line

Also should the filename(e.g. rte_dmadev.h) and directory-name(lib/dmadev) also change ?

I believe it's better to keep dmadev as name of the lib and filename, so it's consistent with other device classes.
What are the other opinions?

Definitely keep. It's one thing to have additional characters in the header name, another to have them in the APIs.

/Bruce

#264 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

In reply to fengchengwen
Quoted 78 lines · fengchengwen
Quoted 71 lines · Bruce Richardson
Quoted 9 lines · Thomas Monjalon

09/09/2021 13:18, Bruce Richardson:

Quoted 8 lines · Thomas Monjalon
Quoted 6 lines

07/09/2021 14:56, Chengwen Feng:

Quoted 3 lines

+ * The first three APIs are used to submit the operation request to the virtual
+ * DMA channel, if the submission is successful, an uint16_t ring_idx is
+ * returned, otherwise a negative number is returned.

unsigned or negative? looks weird.

Quoted 8 lines
Quoted 4 lines

May be, but it works well. We could perhaps rephase to make it less weird though:
"if the submission is successful, a positive ring_idx <= UINT16_MAX is
returned, otherwise a negative number is returned."

I am advocating for int16_t,
it makes a lot of things simpler.

No, it doesn't work as you can't have wrap-around of the IDs once you use signed values - and that impacts both the end app and the internals of the drivers. Let's keep it as-is otherwise it will have massive impacts - including potential perf impacts.

Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

+ *
+ * The last API was used to issue doorbell to hardware, and also there are flags
+ * (@see RTE_DMA_OP_FLAG_SUBMIT) parameter of the first three APIs could do the
+ * same work.

Quoted 8 lines
Quoted 5 lines

I don't understand this sentence. You mean rte_dmadev_submit function? Why past tense "was"?
Why having a redundant function?

Just because there are two ways to do something does not mean that one of them is redundant, as both may be more suitable for different situations.

I agree.

Quoted 9 lines
Quoted 6 lines

When enqueuing a set of jobs to the device, having a separate submit outside a loop makes for clearer code than having a check for the last iteration inside the loop to set a special submit flag. However, for cases where one item alone is to be submitted or there is a small set of jobs to be submitted sequentially, having a submit flag provides a lower-overhead way of doing the submission while still keeping the code clean.

This kind of explanation may be missing in doxygen?

It can be added, sure.

Quoted 12 lines
Quoted 11 lines
Quoted 5 lines

lib/dmadev/rte_dmadev.h line 189

Quoted 2 lines
lib/dmadev/rte_dmadev.h new file +2 -0
+bool
+rte_dmadev_is_valid_dev(uint16_t dev_id);

I would suggest dropping the final "_dev" in the function name.

The alternative, which I would support, is replacing "rte_dmadev" with "rte_dma" across the API. This would then become "rte_dma_is_valid_dev" which is clearer, since the dev is not part of the standard prefix. It also would fit in with a possible future function of "rte_dma_is_valid_vchan" for instance.

Quoted 6 lines

Yes
The question is whether it would make sense to reserver rte_dma_ prefix for some DMA functions which would be outside of dmadev lib? If you think that all DMA functions will be in dmadev,
then yes we can shorten the prefix to rte_dma_.

Quoted 5 lines

Well, any DPDK dma functions which are not in dma library should have the prefix of the library they are in e.g. rte_eal_dma_*, rte_pci_dma_* Therefore, I don't think name conflicts should be an issue, and I like having less typing to do in function names (and I believe Morten was strongly proposing this previously too)

Quoted 3 lines

The dmadev is rather short, if change I prefer all public API with rte_dma_ prefix, and don't have rte_dma_dev_ prefix for such start/stop/close. (ps: the rte_eth_ also have rte_eth_dev_close which is painful for OCD).

I agree that having rte_dma_dev_* is unpleasant naming for those functions, so if we use rte_dma_ as prefix, any dev should be at the end instead:
i.e. rte_dma_stop_dev, rte_dma_start_dev, rte_dma_close_dev, etc.

Quoted 3 lines

Also should the filename(e.g. rte_dmadev.h) and directory-name(lib/dmadev) also change ?

I would keep those names intact.

/Bruce

#265 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

Quoted 9 lines

From: dev [mailto:[email protected]] On Behalf Of Bruce Richardson
Sent: Thursday, 9 September 2021 16.29

Quoted 5 lines · fengchengwen
Quoted 4 lines · Bruce Richardson
Quoted 3 lines · Thomas Monjalon

09/09/2021 13:18, Bruce Richardson:

Quoted 2 lines · Thomas Monjalon
Quoted 1 line

07/09/2021 14:56, Chengwen Feng:

[snip]

Quoted 35 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+bool
+rte_dmadev_is_valid_dev(uint16_t dev_id);

I would suggest dropping the final "_dev" in the function name.

The alternative, which I would support, is replacing "rte_dmadev"

with

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"rte_dma" across the API. This would then become

"rte_dma_is_valid_dev"

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

which is clearer, since the dev is not part of the standard

prefix. It also

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

would fit in with a possible future function of

"rte_dma_is_valid_vchan"

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line

for instance.

Yes
The question is whether it would make sense to reserver rte_dma_

prefix

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

for some DMA functions which would be outside of dmadev lib? If you think that all DMA functions will be in dmadev, then yes we can shorten the prefix to rte_dma_.

Well, any DPDK dma functions which are not in dma library should

have the

Quoted 2 lines
Quoted 2 lines

prefix of the library they are in e.g. rte_eal_dma_*, rte_pci_dma_* Therefore, I don't think name conflicts should be an issue, and I

like

Quoted 1 line
Quoted 1 line

having less typing to do in function names (and I believe Morten

was

Quoted 3 lines
Quoted 1 line

strongly proposing this previously too)

The dmadev is rather short, if change I prefer all public API with

rte_dma_ prefix,

Quoted 9 lines
Quoted 1 line

and don't have rte_dma_dev_ prefix for such start/stop/close. (ps:

the rte_eth_ also

Quoted 1 line

have rte_eth_dev_close which is painful for OCD).

I agree that having rte_dma_dev_* is unpleasant naming for those functions,
so if we use rte_dma_ as prefix, any dev should be at the end instead:
i.e. rte_dma_stop_dev, rte_dma_start_dev, rte_dma_close_dev, etc.

I agree about using rte_dma_ as general prefix.

But I disagree about rte_dma_<action>_<object>() function names, such as rte_dma_stop_dev().

We should follow the convention of rte_dma_<object>_<action>(), like in the ethdev library, e.g. rte_eth_dev_get(), rte_eth_fec_get_capability().

Or simply rte_dma_<action>(), if the object is obvious and can be omitted.

I.e. rte_dma_dev_stop() or rte_dma_stop().

Quoted 5 lines
Quoted 2 lines

Also should the filename(e.g. rte_dmadev.h) and directory-

name(lib/dmadev) also change ?

Quoted 1 line

I would keep those names intact.

Keep intact, as Bruce suggests. This also aligns with the ethdev library.

Quoted 2 lines

/Bruce

#266 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

In reply to Thomas Monjalon

Thanks for reviewing, mostly OK, a few inlined.

Quoted 82 lines · Thomas Monjalon

Hi,

I am having a surface look at the API.
I hope we can do better than previous libs.

07/09/2021 14:56, Chengwen Feng:

MAINTAINERS F: drivers/raw/skeleton/ · line 501

Quoted 9 lines
MAINTAINERS +3 -0
@@ -496,6 +496,10 @@ F: drivers/raw/skeleton/
 F: app/test/test_rawdev.c
 F: doc/guides/prog_guide/rawdev.rst
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <[email protected]>
+F: lib/dmadev/

I think it should before (preferably) or after eventdev, but let's keep rawdev as the last one.

Then please apply the same order in other non-alphabetical lists (doc, meson, etc).

MAINTAINERS F: drivers/raw/skeleton/

Quoted 6 lines
MAINTAINERS +6 -0
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited.
+ * Copyright(c) 2021 Intel Corporation.
+ * Copyright(c) 2021 Marvell International Ltd.
+ * Copyright(c) 2021 SmartShare Systems.
+ */

No need for final dot in copyright lines.

MAINTAINERS

Quoted 2 lines
MAINTAINERS +2 -0
+#ifndef _RTE_DMADEV_H_
+#define _RTE_DMADEV_H_

No need for surrounding underscores.

MAINTAINERS

Quoted 5 lines
MAINTAINERS +5 -0
+
+/**
+ * @file rte_dmadev.h
+ *
+ * RTE DMA (Direct Memory Access) device APIs.

RTE has no meaning when used in a sentence.
And given it is a DPDK library, you don't really need to specify. I would also remove the final "s" as the library is one interface.

Quoted 15 lines

+ *
+ * The DMA framework is built on the following model:
+ *

+ *     ---------------   ---------------       ---------------
+ *     | virtual DMA |   | virtual DMA |       | virtual DMA |
+ *     | channel     |   | channel     |       | channel     |
+ *     ---------------   ---------------       ---------------
+ *            |                |                      |
+ *            ------------------                      |
+ *                     |                              |
+ *               ------------                    ------------
+ *               |  dmadev  |                    |  dmadev  |
+ *               ------------                    ------------
+ *                     |                              |
+ *            ------------------               ------------------
Quoted 8 lines
+ *            | HW-DMA-channel |               | HW-DMA-channel |
+ *            ------------------               ------------------
+ *                     |                              |
+ *                     --------------------------------
+ *                                     |
+ *                           ---------------------
+ *                           | HW-DMA-Controller |
+ *                           ---------------------

You don't hyphens between the words I think.

Quoted 9 lines

+ *
+ * The DMA controller could have multiple HW-DMA-channels (aka. HW-DMA-queues),
+ * each HW-DMA-channel should be represented by a dmadev.
+ *
+ * The dmadev could create multiple virtual DMA channels, each virtual DMA
+ * channel represents a different transfer context. The DMA operation request
+ * must be submitted to the virtual DMA channel. e.g. Application could create
+ * virtual DMA channel 0 for memory-to-memory transfer scenario, and create
+ * virtual DMA channel 1 for memory-to-device transfer scenario.

Quoted 2 lines

What is the benefit of virtual channels compared to have separate dmadevs for the same HW channel?

This design is from the previous discussion [1]. If a dmadev is created for each virtual channel, there are many associations between the dmadevs. For example, channel operations of some devices need to interact with the kernel, the corresponding kernel operation handles need to be shared among multiple dmadevs. It's going to get more complicated.

[1] https://lore.kernel.org/dpdk-dev/[email protected]/

Quoted 6 lines
Quoted 4 lines

+ *
+ * The dmadev are dynamically allocated by rte_dmadev_pmd_allocate() during the
+ * PCI/SoC device probing phase performed at EAL initialization time. And could

Not clear what is this phase. Do you mean bus probing?

yes, it's bus probing

Quoted 6 lines
Quoted 3 lines

+ * be released by rte_dmadev_pmd_release() during the PCI/SoC device removing
+ * phase.

Again what is this phase?
I think freeing should be done only via the "close" function.

OK
The allocate/release will be reconstructed with reference to rte_eth_dev_pci_generic_probe.

Quoted 70 lines
Quoted 4 lines

+ *
+ * This framework uses 'uint16_t dev_id' as the device identifier of a dmadev,
+ * and 'uint16_t vchan' as the virtual DMA channel identifier in one dmadev.

I think it is better to use signed int16_t so you can express "none" in the API, which can simplify some functions and error management.

Quoted 8 lines

+ *
+ * The functions exported by the dmadev API to setup a device designated by its
+ * device identifier must be invoked in the following order:

+ *     - rte_dmadev_configure()
+ *     - rte_dmadev_vchan_setup()
+ *     - rte_dmadev_start()

+ *
+ * Then, the application can invoke dataplane APIs to process jobs.

You mean "dataplane functions".

Quoted 5 lines

+ *
+ * If the application wants to change the configuration (i.e. invoke
+ * rte_dmadev_configure() or rte_dmadev_vchan_setup()), it must invoke
+ * rte_dmadev_stop() first to stop the device and then do the reconfiguration
+ * before invoking rte_dmadev_start() again. The dataplane APIs should not be

again, APIs -> functions

Quoted 11 lines

+ * invoked when the device is stopped.
+ *
+ * Finally, an application can close a dmadev by invoking the
+ * rte_dmadev_close() function.
+ *
+ * The dataplane APIs include two parts:
+ * The first part is the submission of operation requests:

+ *     - rte_dmadev_copy()
+ *     - rte_dmadev_copy_sg()
+ *     - rte_dmadev_fill()
+ *     - rte_dmadev_submit()
Quoted 7 lines

+ *
+ * These APIs could work with different virtual DMA channels which have
+ * different contexts.
+ *
+ * The first three APIs are used to submit the operation request to the virtual
+ * DMA channel, if the submission is successful, an uint16_t ring_idx is
+ * returned, otherwise a negative number is returned.

unsigned or negative? looks weird.

Quoted 4 lines

+ *
+ * The last API was used to issue doorbell to hardware, and also there are flags
+ * (@see RTE_DMA_OP_FLAG_SUBMIT) parameter of the first three APIs could do the
+ * same work.

I don't understand this sentence.
You mean rte_dmadev_submit function?
Why past tense "was"?
Why having a redundant function?

Quoted 9 lines

+ *
+ * The second part is to obtain the result of requests:
+ * - rte_dmadev_completed()
+ * - return the number of operation requests completed successfully.
+ * - rte_dmadev_completed_status()
+ * - return the number of operation requests completed.
+ *
+ * @note The two completed APIs also support return the last completed
+ * operation's ring_idx.

Not sure why this note here.

Quoted 7 lines
Quoted 5 lines

+ * @note If the dmadev works in silent mode (@see RTE_DMADEV_CAPA_SILENT),
+ * application does not invoke the above two completed APIs.
+ *
+ * About the ring_idx which enqueue APIs (e.g. rte_dmadev_copy()
+ * rte_dmadev_fill()) returned, the rules are as follows:

I feel a word is missing above.

Can you point it out specifically ?
PS: I specifically examined by access https://www.nounplus.net/grammarcheck/ and found it prompts the 'enqueue' to 'enqueues or enqueued'.

Quoted 81 lines
Quoted 21 lines

+ * - ring_idx for each virtual DMA channel are independent.
+ * - For a virtual DMA channel, the ring_idx is monotonically incremented,
+ * when it reach UINT16_MAX, it wraps back to zero.
+ * - This ring_idx can be used by applications to track per-operation
+ * metadata in an application-defined circular ring.
+ * - The initial ring_idx of a virtual DMA channel is zero, after the
+ * device is stopped, the ring_idx needs to be reset to zero.
+ *
+ * One example:

+ *     - step-1: start one dmadev
+ *     - step-2: enqueue a copy operation, the ring_idx return is 0
+ *     - step-3: enqueue a copy operation again, the ring_idx return is 1
+ *     - ...
+ *     - step-101: stop the dmadev
+ *     - step-102: start the dmadev
+ *     - step-103: enqueue a copy operation, the ring_idx return is 0
+ *     - ...
+ *     - step-x+0: enqueue a fill operation, the ring_idx return is 65535
+ *     - step-x+1: enqueue a copy operation, the ring_idx return is 0
+ *     - ...
Quoted 3 lines

+ *
+ * The DMA operation address used in enqueue APIs (i.e. rte_dmadev_copy(),
+ * rte_dmadev_copy_sg(), rte_dmadev_fill()) defined as rte_iova_t type. The

"is" defined

Quoted 1 line

+ * dmadev supports two types of address: memory address and device address.

Please try to start new sentences on a new line.

Quoted 4 lines

+ *
+ * - memory address: the source and destination address of the memory-to-memory
+ * transfer type, or the source address of the memory-to-device transfer type,
+ * or the destination address of the device-to-memory transfer type.

Quoted 8 lines

+ * @note If the device support SVA (@see RTE_DMADEV_CAPA_SVA), the memory
+ * address can be any VA address, otherwise it must be an IOVA address.
+ *
+ * - device address: the source and destination address of the device-to-device
+ * transfer type, or the source address of the device-to-memory transfer type,
+ * or the destination address of the memory-to-device transfer type.
+ *
+ * By default, all the functions of the dmadev API exported by a PMD are

What do you mean "by default"? Is there some exceptions?

Quoted 7 lines

+ * lock-free functions which assume to not be invoked in parallel on different
+ * logical cores to work on the same target dmadev object.
+ * @note Different virtual DMA channels on the same dmadev *DO NOT* support
+ * parallel invocation because these virtual DMA channels share the same
+ * HW-DMA-channel.
+ *
+ */

[...]

Quoted 1 line

+#define RTE_DMADEV_NAME_MAX_LEN RTE_DEV_NAME_MAX_LEN

Why not using RTE_DEV_NAME_MAX_LEN directly?
If you keep it, it should be commented, explaining whether it takes '\0' into account or not.

MAINTAINERS

Quoted 3 lines
MAINTAINERS +3 -0
+__rte_experimental
+bool
+rte_dmadev_is_valid_dev(uint16_t dev_id);

I would suggest dropping the final "_dev" in the function name.

MAINTAINERS

Quoted 2 lines
MAINTAINERS +2 -0
+uint16_t
+rte_dmadev_count(void);

It would be safer to name it rte_dmadev_count_avail in case we need new kind of device count later.

MAINTAINERS

Quoted 2 lines
MAINTAINERS +2 -0
+
+/* Enumerates DMA device capabilities. */

You should group them with a doxygen group syntax.
See https://patches.dpdk.org/project/dpdk/patch/[email protected]/

Quoted 1 line

+#define RTE_DMADEV_CAPA_MEM_TO_MEM (1ull << 0)

Quoted 8 lines

Please use RTE_BIT macro (32 or 64).

Quoted 4 lines

+/**< DMA device support memory-to-memory transfer.
+ *
+ * @see struct rte_dmadev_info::dev_capa
+ */

It is preferred to have documentation before the item.

Is this particularly strong?
I use postfix comment style for whole doxygen comments. I think it's better to maintain a unified style.

Quoted 89 lines

[...]

MAINTAINERS

Quoted 5 lines
MAINTAINERS +5 -0
+/**
+ * A structure used to retrieve the information of a DMA device.
+ */
+struct rte_dmadev_info {
+	struct rte_device *device; /**< Generic Device information. */

Please do not expose this.

Quoted 6 lines
MAINTAINERS +6 -0
+	uint64_t dev_capa; /**< Device capabilities (RTE_DMADEV_CAPA_*). */
+	uint16_t max_vchans;
+	/**< Maximum number of virtual DMA channels supported. */
+	uint16_t max_desc;
+	/**< Maximum allowed number of virtual DMA channel descriptors. */
+	uint16_t min_desc;

MAINTAINERS

Quoted 10 lines
MAINTAINERS +10 -0
+	/**< Minimum allowed number of virtual DMA channel descriptors. */
+	uint16_t max_sges;
+	/**< Maximum number of source or destination scatter-gather entry
+	 * supported.
+	 * If the device does not support COPY_SG capability, this value can be
+	 * zero.
+	 * If the device supports COPY_SG capability, then rte_dmadev_copy_sg()
+	 * parameter nb_src/nb_dst should not exceed this value.
+	 */
+	uint16_t nb_vchans; /**< Number of virtual DMA channel configured. */

What about adding NUMA node?

    /* Local NUMA memory ID. -1 if unknown. */
    int16_t numa_node;

MAINTAINERS

Quoted 4 lines
MAINTAINERS +3 -0
+};
 
+int
+rte_dmadev_info_get(uint16_t dev_id, struct rte_dmadev_info *dev_info);

In .h files, the return type should not be on a separate line.

MAINTAINERS

Quoted 4 lines
MAINTAINERS +4 -0
+
+/**
+ * A structure used to configure a DMA device.
+ */

You should mention where it is used with @see.

Quoted 15 lines
MAINTAINERS +15 -0
+struct rte_dmadev_conf {
+	uint16_t nb_vchans;
+	/**< The number of virtual DMA channels to set up for the DMA device.
+	 * This value cannot be greater than the field 'max_vchans' of struct
+	 * rte_dmadev_info which get from rte_dmadev_info_get().
+	 */
+	bool enable_silent;
+	/**< Indicates whether to enable silent mode.
+	 * false-default mode, true-silent mode.
+	 * This value can be set to true only when the SILENT capability is
+	 * supported.
+	 *
+	 * @see RTE_DMADEV_CAPA_SILENT
+	 */
+};

[...]

Quoted 1 line

+#define RTE_DMADEV_ALL_VCHAN 0xFFFFu

Please do not add this kind of constant without a doxygen comment.

It seems you don't manage the maximum number of devices. It is fixed in the .c file:

	struct rte_dmadev rte_dmadevices[RTE_DMADEV_MAX_DEVS];

Instead I would suggest a more dynamic approach with an init function, so the application can extend it before calling rte_eal_init. Please see how it is implemented here:
https://patches.dpdk.org/project/dpdk/patch/[email protected]/

Above patch could also inspire you to start some docs in this patch.

This series add one file per patch.
Instead it would be better to have groups of features per patch, meaning the implementation and the driver interface should be in the same patch.
You can split like this:
1/ device allocation
2/ configuration and start/stop
3/ dataplane functions

Quoted 5 lines

I would suggest 2 more patches:
4/ event notification
see https://patches.dpdk.org/project/dpdk/patch/[email protected]/
5/ multi-process
see https://patches.dpdk.org/project/dpdk/patch/[email protected]/

The split mode you recommend is better.
But is this particularly strong ? Because many acked-by and reviewed-by base on stand-alone file. Does this division mean that a new acked/reviewed ?

Quoted 7 lines

Thanks for the work

.

#267 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

09/09/2021 15:33, fengchengwen:

Quoted 14 lines · Thomas Monjalon
Quoted 10 lines

07/09/2021 14:56, Chengwen Feng:

Quoted 8 lines

+ * The DMA controller could have multiple HW-DMA-channels (aka. HW-DMA-queues),
+ * each HW-DMA-channel should be represented by a dmadev.
+ *
+ * The dmadev could create multiple virtual DMA channels, each virtual DMA
+ * channel represents a different transfer context. The DMA operation request
+ * must be submitted to the virtual DMA channel. e.g. Application could create
+ * virtual DMA channel 0 for memory-to-memory transfer scenario, and create
+ * virtual DMA channel 1 for memory-to-device transfer scenario.

Quoted 2 lines

What is the benefit of virtual channels compared to have separate dmadevs for the same HW channel?

Quoted 7 lines

This design is from the previous discussion [1]. If a dmadev is created for each virtual channel, there are many associations between the dmadevs. For example, channel operations of some devices need to interact with the kernel, the corresponding kernel operation handles need to be shared among multiple dmadevs. It's going to get more complicated.

[1] https://lore.kernel.org/dpdk-dev/[email protected]/

OK thanks for the explanation.

[...]

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

+ * be released by rte_dmadev_pmd_release() during the PCI/SoC device removing
+ * phase.

Again what is this phase?
I think freeing should be done only via the "close" function.

OK
The allocate/release will be reconstructed with reference to rte_eth_dev_pci_generic_probe.

You shouldn't always mimic ethdev, there can be some misconceptions :) I think you don't need PCI specific helper.

[...]

Quoted 11 lines
Quoted 7 lines
Quoted 5 lines

+ * @note If the dmadev works in silent mode (@see RTE_DMADEV_CAPA_SILENT),
+ * application does not invoke the above two completed APIs.
+ *
+ * About the ring_idx which enqueue APIs (e.g. rte_dmadev_copy()
+ * rte_dmadev_fill()) returned, the rules are as follows:

I feel a word is missing above.

Can you point it out specifically ?
PS: I specifically examined by access https://www.nounplus.net/grammarcheck/ and found it prompts the 'enqueue' to 'enqueues or enqueued'.

After second read, I think it is a tense problem.
What about "returned" -> "return" ?

[...]

Quoted 10 lines
Quoted 6 lines
Quoted 4 lines

+/**< DMA device support memory-to-memory transfer.
+ *
+ * @see struct rte_dmadev_info::dev_capa
+ */

It is preferred to have documentation before the item.

Is this particularly strong?
I use postfix comment style for whole doxygen comments. I think it's better to maintain a unified style.

In general prefix comment is preferred.
Postfix comments are OK for short comments on the same line.

[...]

Quoted 9 lines
Quoted 9 lines

This series add one file per patch.
Instead it would be better to have groups of features per patch, meaning the implementation and the driver interface should be in the same patch.
You can split like this:
1/ device allocation
2/ configuration and start/stop
3/ dataplane functions

Quoted 8 lines
Quoted 5 lines

I would suggest 2 more patches:
4/ event notification
see https://patches.dpdk.org/project/dpdk/patch/[email protected]/
5/ multi-process
see https://patches.dpdk.org/project/dpdk/patch/[email protected]/

The split mode you recommend is better.
But is this particularly strong ?

Yes, that's really better.

Quoted 2 lines

Because many acked-by and reviewed-by base on stand-alone file. Does this division mean that a new acked/reviewed ?

You can keep the acks which are commong to the first 4 patches I guess and ask for re-ack to others.

#268 lore raw

Re: [dpdk-dev] [PATCH v21 1/7] dmadev: introduce DMA device library public APIs

In reply to Thomas Monjalon

Hi Thomas,

Most fixed in V22, some please see inline comment

Thanks.

Quoted 15 lines · Thomas Monjalon

Hi,

I am having a surface look at the API.
I hope we can do better than previous libs.

07/09/2021 14:56, Chengwen Feng:

Quoted 9 lines
MAINTAINERS +3 -0
@@ -496,6 +496,10 @@ F: drivers/raw/skeleton/
 F: app/test/test_rawdev.c
 F: doc/guides/prog_guide/rawdev.rst
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <[email protected]>
+F: lib/dmadev/

[snip]

Quoted 5 lines

MAINTAINERS

Quoted 2 lines
MAINTAINERS +2 -0
+
+/* Enumerates DMA device capabilities. */

You should group them with a doxygen group syntax.
See https://patches.dpdk.org/project/dpdk/patch/[email protected]/

Because RTE_DMADEV_CAPA_* has multiple lines of comments, the effect is not good when using group syntax.

Also consider using enum to define, but its value is uint64_t, and enumeration is generally of the int type.

So it stays the same here.

Quoted 10 lines
Quoted 2 lines

+#define RTE_DMADEV_CAPA_MEM_TO_MEM (1ull << 0)

Please use RTE_BIT macro (32 or 64).

Quoted 4 lines

+/**< DMA device support memory-to-memory transfer.
+ *
+ * @see struct rte_dmadev_info::dev_capa
+ */

[snip]

Quoted 10 lines

This series add one file per patch.
Instead it would be better to have groups of features per patch, meaning the implementation and the driver interface should be in the same patch.
You can split like this:
1/ device allocation
2/ configuration and start/stop
3/ dataplane functions

Quoted 6 lines

I would suggest 2 more patches:
4/ event notification
see https://patches.dpdk.org/project/dpdk/patch/[email protected]/
5/ multi-process
see https://patches.dpdk.org/project/dpdk/patch/[email protected]/

The multi-process have many modify for device allocation, because the coupling between the two is deep, I combines them to one patch.

Quoted 6 lines

Thanks for the work

.

#269 lore raw

PATCH v21 2/7 dmadev: introduce DMA device library internal header

In reply to Chengwen Feng

This patch introduce DMA device library internal header, which contains internal data types that are used by the DMA devices in order to expose their ops to the class.

---
 lib/dmadev/meson.build       |   1 +
 lib/dmadev/rte_dmadev_core.h | 185 +++++++++++++++++++++++++++++++++++
 2 files changed, 186 insertions(+)
 create mode 100644 lib/dmadev/rte_dmadev_core.h
lib/dmadev/meson.build +1 -0
index 6d5bd85373..f421ec1909 100644
@@ -2,3 +2,4 @@
 # Copyright(c) 2021 HiSilicon Limited.
 
 headers = files('rte_dmadev.h')
+indirect_headers += files('rte_dmadev_core.h')
lib/dmadev/rte_dmadev_core.h new file +185 -0

185 lines not on this page · raw message

Signature · 1 line
2.33.0
#270 lore raw

PATCH v21 3/7 dmadev: introduce DMA device library PMD header

In reply to Chengwen Feng

This patch introduce DMA device library PMD header which was driver facing APIs for a DMA device.

---
 lib/dmadev/meson.build      |  1 +
 lib/dmadev/rte_dmadev.h     |  2 ++
 lib/dmadev/rte_dmadev_pmd.h | 72 +++++++++++++++++++++++++++++++++++++
 lib/dmadev/version.map      | 10 ++++++
 4 files changed, 85 insertions(+)
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
lib/dmadev/meson.build +1 -0
index f421ec1909..833baf7d54 100644
@@ -3,3 +3,4 @@
 
 headers = files('rte_dmadev.h')
 indirect_headers += files('rte_dmadev_core.h')
+driver_sdk_headers += files('rte_dmadev_pmd.h')
lib/dmadev/rte_dmadev.h +2 -0
index 76d71615eb..c8dd0009f5 100644
@@ -730,6 +730,8 @@ struct rte_dmadev_sge {
 	uint32_t length; /**< The DMA operation length. */
 };
 
+#include "rte_dmadev_core.h"
+
 /* DMA flags to augment operation preparation. */
 #define RTE_DMA_OP_FLAG_FENCE	(1ull << 0)
 /**< DMA fence flag.
lib/dmadev/rte_dmadev_pmd.h new file +72 -0

72 lines not on this page · raw message

lib/dmadev/version.map +10 -0
index 2e37882364..d027eeac97 100644
@@ -22,3 +22,13 @@ EXPERIMENTAL {
 
 	local: *;
 };
+
+INTERNAL {
+        global:
+
+	rte_dmadev_get_device_by_name;
+	rte_dmadev_pmd_allocate;
+	rte_dmadev_pmd_release;
+
+	local: *;
+};
Signature · 1 line
2.33.0
#271 lore raw

PATCH v21 4/7 dmadev: introduce DMA device library implementation

In reply to Chengwen Feng

This patch introduce DMA device library implementation which includes configuration and I/O with the DMA devices.

---
 config/rte_config.h          |   3 +
 lib/dmadev/meson.build       |   1 +
 lib/dmadev/rte_dmadev.c      | 607 +++++++++++++++++++++++++++++++++++
 lib/dmadev/rte_dmadev.h      | 118 ++++++-
 lib/dmadev/rte_dmadev_core.h |   2 +
 lib/dmadev/version.map       |   1 +
 6 files changed, 720 insertions(+), 12 deletions(-)
 create mode 100644 lib/dmadev/rte_dmadev.c
config/rte_config.h +3 -0
index 590903c07d..331a431819 100644
@@ -81,6 +81,9 @@
 /* rawdev defines */
 #define RTE_RAWDEV_MAX_DEVS 64
 
+/* dmadev defines */
+#define RTE_DMADEV_MAX_DEVS 64
+
 /* ip_fragmentation defines */
 #define RTE_LIBRTE_IP_FRAG_MAX_FRAG 4
 #undef RTE_LIBRTE_IP_FRAG_TBL_STAT
lib/dmadev/meson.build +1 -0
index 833baf7d54..d2fc85e8c7 100644
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2021 HiSilicon Limited.
 
+sources = files('rte_dmadev.c')
 headers = files('rte_dmadev.h')
 indirect_headers += files('rte_dmadev_core.h')
 driver_sdk_headers += files('rte_dmadev_pmd.h')
lib/dmadev/rte_dmadev.c new file +607 -0

607 lines not on this page · raw message

lib/dmadev/rte_dmadev.h +106 -12

161 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h +2 -0
index cbf5e88621..398bdf6391 100644
@@ -182,4 +182,6 @@ struct rte_dmadev {
 	uint64_t reserved[2]; /**< Reserved for future fields. */
 } __rte_cache_aligned;
 
+extern struct rte_dmadev rte_dmadevices[];
+
 #endif /* _RTE_DMADEV_CORE_H_ */
lib/dmadev/version.map +1 -0
index d027eeac97..80be592713 100644
@@ -26,6 +26,7 @@ EXPERIMENTAL {
 INTERNAL {
         global:
 
+	rte_dmadevices;
 	rte_dmadev_get_device_by_name;
 	rte_dmadev_pmd_allocate;
 	rte_dmadev_pmd_release;
Signature · 1 line
2.33.0
#272 lore raw

Re: [dpdk-dev] [PATCH v21 4/7] dmadev: introduce DMA device library implementation

<snip>

Hi Chengwen,

While testing the IOAT driver I realised that we hadn't implemented the new RTE_DMADEV_ALL_VCHAN flag for stats. Rather than every driver that only supports 1 vchan enabling support for this flag it would probably be better to catch it in the library as shown below.

Thanks,
Conor.

Quoted 5 lines
lib/dmadev/rte_dmadev.c new file +5 -0
+int
+rte_dmadev_stats_get(uint16_t dev_id, uint16_t vchan,
+		     struct rte_dmadev_stats *stats)
+{
+	const struct rte_dmadev *dev = &rte_dmadevices[dev_id];

lib/dmadev/rte_dmadev.c line 506

Quoted 10 lines
lib/dmadev/rte_dmadev.c new file +10 -0
+
+	RTE_DMADEV_VALID_DEV_ID_OR_ERR_RET(dev_id, -EINVAL);
+	if (stats == NULL)
+		return -EINVAL;
+	if (vchan >= dev->data->dev_conf.nb_vchans &&
+	    vchan != RTE_DMADEV_ALL_VCHAN) {
+		RTE_DMADEV_LOG(ERR,
+			"Device %u vchan %u out of range", dev_id, vchan);
+		return -EINVAL;
+	}
	if (vchan == RTE_DMADEV_ALL_VCHAN && dev->data->dev_conf.nb_vchans == 1)
		vchan = 0;
Quoted 10 lines
(quoted diff) +2 -0
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->stats_get, -

ENOTSUP);

lib/dmadev/rte_dmadev.c new file +7 -0
+	memset(stats, 0, sizeof(struct rte_dmadev_stats));
+	return (*dev->dev_ops->stats_get)(dev, vchan, stats,
+					  sizeof(struct rte_dmadev_stats));
+}
+
+int
+rte_dmadev_stats_reset(uint16_t dev_id, uint16_t vchan)

lib/dmadev/rte_dmadev.c line 525

Quoted 10 lines
lib/dmadev/rte_dmadev.c new file +10 -0
+{
+	struct rte_dmadev *dev = &rte_dmadevices[dev_id];
+
+	RTE_DMADEV_VALID_DEV_ID_OR_ERR_RET(dev_id, -EINVAL);
+	if (vchan >= dev->data->dev_conf.nb_vchans &&
+	    vchan != RTE_DMADEV_ALL_VCHAN) {
+		RTE_DMADEV_LOG(ERR,
+			"Device %u vchan %u out of range", dev_id, vchan);
+		return -EINVAL;
+	}
	if (vchan == RTE_DMADEV_ALL_VCHAN && dev->data->dev_conf.nb_vchans == 1)
		vchan = 0;
Quoted 5 lines
(quoted diff) +2 -0
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->stats_reset, -

ENOTSUP);

lib/dmadev/rte_dmadev.c new file +2 -0
+	return (*dev->dev_ops->stats_reset)(dev, vchan);
+}

<snip>

#273 lore raw

Re: [dpdk-dev] [PATCH v21 4/7] dmadev: introduce DMA device library implementation

For the dmadev which only support one vchan, it's OK to ignore vchan parameters. I don't think that logic needs to be added.

Quoted 58 lines · Conor

<snip>

Hi Chengwen,

While testing the IOAT driver I realised that we hadn't implemented the new RTE_DMADEV_ALL_VCHAN flag for stats. Rather than every driver that only supports 1 vchan enabling support for this flag it would probably be better to catch it in the library as shown below.

Thanks,
Conor.

Quoted 5 lines
lib/dmadev/rte_dmadev.c new file +5 -0
+int
+rte_dmadev_stats_get(uint16_t dev_id, uint16_t vchan,
+		     struct rte_dmadev_stats *stats)
+{
+	const struct rte_dmadev *dev = &rte_dmadevices[dev_id];

lib/dmadev/rte_dmadev.c line 506

Quoted 10 lines
lib/dmadev/rte_dmadev.c new file +10 -0
+
+	RTE_DMADEV_VALID_DEV_ID_OR_ERR_RET(dev_id, -EINVAL);
+	if (stats == NULL)
+		return -EINVAL;
+	if (vchan >= dev->data->dev_conf.nb_vchans &&
+	    vchan != RTE_DMADEV_ALL_VCHAN) {
+		RTE_DMADEV_LOG(ERR,
+			"Device %u vchan %u out of range", dev_id, vchan);
+		return -EINVAL;
+	}
	if (vchan == RTE_DMADEV_ALL_VCHAN && dev->data->dev_conf.nb_vchans == 1)
		vchan = 0;
Quoted 10 lines
(quoted diff) +2 -0
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->stats_get, -

ENOTSUP);

lib/dmadev/rte_dmadev.c new file +7 -0
+	memset(stats, 0, sizeof(struct rte_dmadev_stats));
+	return (*dev->dev_ops->stats_get)(dev, vchan, stats,
+					  sizeof(struct rte_dmadev_stats));
+}
+
+int
+rte_dmadev_stats_reset(uint16_t dev_id, uint16_t vchan)

lib/dmadev/rte_dmadev.c line 525

Quoted 10 lines
lib/dmadev/rte_dmadev.c new file +10 -0
+{
+	struct rte_dmadev *dev = &rte_dmadevices[dev_id];
+
+	RTE_DMADEV_VALID_DEV_ID_OR_ERR_RET(dev_id, -EINVAL);
+	if (vchan >= dev->data->dev_conf.nb_vchans &&
+	    vchan != RTE_DMADEV_ALL_VCHAN) {
+		RTE_DMADEV_LOG(ERR,
+			"Device %u vchan %u out of range", dev_id, vchan);
+		return -EINVAL;
+	}
	if (vchan == RTE_DMADEV_ALL_VCHAN && dev->data->dev_conf.nb_vchans == 1)
		vchan = 0;
Quoted 5 lines
(quoted diff) +2 -0
+
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->stats_reset, -

ENOTSUP);

lib/dmadev/rte_dmadev.c new file +2 -0
+	return (*dev->dev_ops->stats_reset)(dev, vchan);
+}

<snip>

#274 lore raw

Re: [dpdk-dev] [PATCH v21 4/7] dmadev: introduce DMA device library implementation

In reply to Chengwen Feng
Quoted 18 lines · Chengwen Feng

This patch introduce DMA device library implementation which includes configuration and I/O with the DMA devices.

---
  config/rte_config.h          |   3 +
  lib/dmadev/meson.build       |   1 +
  lib/dmadev/rte_dmadev.c      | 607 +++++++++++++++++++++++++++++++++++
  lib/dmadev/rte_dmadev.h      | 118 ++++++-
  lib/dmadev/rte_dmadev_core.h |   2 +
  lib/dmadev/version.map       |   1 +
  6 files changed, 720 insertions(+), 12 deletions(-)

create mode 100644 lib/dmadev/rte_dmadev.c

[snip]

Quoted 16 lines

/**
* @warning

lib/dmadev/rte_dmadev.h +6 -2
@@ -941,10 +1018,27 @@ rte_dmadev_completed(uint16_t dev_id, uint16_t vchan, const uint16_t nb_cpls,
   *   status array are also set.
   */
  __rte_experimental
-uint16_t
+static inline uint16_t
  rte_dmadev_completed_status(uint16_t dev_id, uint16_t vchan,
  			    const uint16_t nb_cpls, uint16_t *last_idx,
-			    enum rte_dma_status_code *status);
+			    enum rte_dma_status_code *status)
+{
+	struct rte_dmadev *dev = &rte_dmadevices[dev_id];
+	uint16_t idx;
+

lib/dmadev/rte_dmadev.h rte_dmadev_completed(uint16_t dev_id, uint16_t vchan, const uint16_t nb_cpls, · line 1038

Quoted 10 lines
lib/dmadev/rte_dmadev.h +10 -0
+#ifdef RTE_DMADEV_DEBUG
+	if (!rte_dmadev_is_valid_dev(dev_id) || !dev->data->dev_started ||
+	    vchan >= dev->data->dev_conf.nb_vchans ||
+	    nb_cpls == 0 || status == NULL)
+		return 0;
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->completed_status, 0);
+#endif
+
+	if (last_idx == NULL)
+		last_idx = &idx;

Hi Chengwen,

An internal coverity scan on the IDXD dmadev driver patches flagged a potential null pointer dereference when using completed_status().

IMO it is a false positive for the driver code since it should be checked at the library API level, however the check is also not present in the library.

For the v22, can you add the NULL pointer check for status here, like you have for last_idx, please?

/Kevin

Quoted 7 lines
lib/dmadev/rte_dmadev.h +3 -0
+
+	return (*dev->completed_status)(dev, vchan, nb_cpls, last_idx, status);
+}
 
  #ifdef __cplusplus
  }
#275 lore raw

Re: [dpdk-dev] [PATCH v21 4/7] dmadev: introduce DMA device library implementation

Quoted 50 lines · Kevin Laatz
Quoted 18 lines · Chengwen Feng

This patch introduce DMA device library implementation which includes configuration and I/O with the DMA devices.

---
  config/rte_config.h          |   3 +
  lib/dmadev/meson.build       |   1 +
  lib/dmadev/rte_dmadev.c      | 607 +++++++++++++++++++++++++++++++++++
  lib/dmadev/rte_dmadev.h      | 118 ++++++-
  lib/dmadev/rte_dmadev_core.h |   2 +
  lib/dmadev/version.map       |   1 +
  6 files changed, 720 insertions(+), 12 deletions(-)

create mode 100644 lib/dmadev/rte_dmadev.c

[snip]

Quoted 16 lines

/**
* @warning

lib/dmadev/rte_dmadev.h +6 -2
@@ -941,10 +1018,27 @@ rte_dmadev_completed(uint16_t dev_id, uint16_t vchan, const uint16_t nb_cpls,
   *   status array are also set.
   */
  __rte_experimental
-uint16_t
+static inline uint16_t
  rte_dmadev_completed_status(uint16_t dev_id, uint16_t vchan,
  			    const uint16_t nb_cpls, uint16_t *last_idx,
-			    enum rte_dma_status_code *status);
+			    enum rte_dma_status_code *status)
+{
+	struct rte_dmadev *dev = &rte_dmadevices[dev_id];
+	uint16_t idx;
+

lib/dmadev/rte_dmadev.h rte_dmadev_completed(uint16_t dev_id, uint16_t vchan, const uint16_t nb_cpls, · line 1038

Quoted 10 lines
lib/dmadev/rte_dmadev.h +10 -0
+#ifdef RTE_DMADEV_DEBUG
+	if (!rte_dmadev_is_valid_dev(dev_id) || !dev->data->dev_started ||
+	    vchan >= dev->data->dev_conf.nb_vchans ||
+	    nb_cpls == 0 || status == NULL)
+		return 0;
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->completed_status, 0);
+#endif
+
+	if (last_idx == NULL)
+		last_idx = &idx;

Hi Chengwen,

Quoted 9 lines

An internal coverity scan on the IDXD dmadev driver patches flagged a potential null pointer dereference when using completed_status().

IMO it is a false positive for the driver code since it should be checked at the library API level, however the check is also not present in the library.

For the v22, can you add the NULL pointer check for status here, like you have for last_idx, please?

I think the check would have to be different than that for last_idx, since the status pointer is a pointer to an array, rather than a single value - which procludes a simple replacement in the wrapper function that the compiler can inline away if unnecessary.
It's probably best to add it as a check in the debug block, with an error-return if status is NULL.

/Bruce

#276 lore raw

Re: [dpdk-dev] [PATCH v21 4/7] dmadev: introduce DMA device library implementation

Quoted 59 lines · Bruce Richardson
Quoted 49 lines · Kevin Laatz
Quoted 18 lines · Chengwen Feng

This patch introduce DMA device library implementation which includes configuration and I/O with the DMA devices.

---
   config/rte_config.h          |   3 +
   lib/dmadev/meson.build       |   1 +
   lib/dmadev/rte_dmadev.c      | 607 +++++++++++++++++++++++++++++++++++
   lib/dmadev/rte_dmadev.h      | 118 ++++++-
   lib/dmadev/rte_dmadev_core.h |   2 +
   lib/dmadev/version.map       |   1 +
   6 files changed, 720 insertions(+), 12 deletions(-)

create mode 100644 lib/dmadev/rte_dmadev.c

[snip]

Quoted 16 lines

/**
* @warning

lib/dmadev/rte_dmadev.h +6 -2
@@ -941,10 +1018,27 @@ rte_dmadev_completed(uint16_t dev_id, uint16_t vchan, const uint16_t nb_cpls,
    *   status array are also set.
    */
   __rte_experimental
-uint16_t
+static inline uint16_t
   rte_dmadev_completed_status(uint16_t dev_id, uint16_t vchan,
   			    const uint16_t nb_cpls, uint16_t *last_idx,
-			    enum rte_dma_status_code *status);
+			    enum rte_dma_status_code *status)
+{
+	struct rte_dmadev *dev = &rte_dmadevices[dev_id];
+	uint16_t idx;
+

lib/dmadev/rte_dmadev.h rte_dmadev_completed(uint16_t dev_id, uint16_t vchan, const uint16_t nb_cpls, · line 1038

Quoted 10 lines
lib/dmadev/rte_dmadev.h +10 -0
+#ifdef RTE_DMADEV_DEBUG
+	if (!rte_dmadev_is_valid_dev(dev_id) || !dev->data->dev_started ||
+	    vchan >= dev->data->dev_conf.nb_vchans ||
+	    nb_cpls == 0 || status == NULL)
+		return 0;
+	RTE_FUNC_PTR_OR_ERR_RET(*dev->completed_status, 0);
+#endif
+
+	if (last_idx == NULL)
+		last_idx = &idx;

Hi Chengwen,

Quoted 9 lines

An internal coverity scan on the IDXD dmadev driver patches flagged a potential null pointer dereference when using completed_status().

IMO it is a false positive for the driver code since it should be checked at the library API level, however the check is also not present in the library.

For the v22, can you add the NULL pointer check for status here, like you have for last_idx, please?

Quoted 6 lines

I think the check would have to be different than that for last_idx, since the status pointer is a pointer to an array, rather than a single value - which procludes a simple replacement in the wrapper function that the compiler can inline away if unnecessary.
It's probably best to add it as a check in the debug block, with an error-return if status is NULL.

+1

/Kevin

#277 lore raw

PATCH v21 5/7 doc: add DMA device library guide

In reply to Chengwen Feng

This patch adds dmadev library guide.

---
 MAINTAINERS                          |   1 +
 doc/guides/prog_guide/dmadev.rst     | 125 ++++++++++++
 doc/guides/prog_guide/img/dmadev.svg | 283 +++++++++++++++++++++++++++
 doc/guides/prog_guide/index.rst      |   1 +
 4 files changed, 410 insertions(+)
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
MAINTAINERS +1 -0
index 9885cc56b7..e237e9406b 100644
@@ -499,6 +499,7 @@ F: doc/guides/prog_guide/rawdev.rst
 DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
+F: doc/guides/prog_guide/dmadev.rst
 
 
 Memory Pool Drivers
doc/guides/prog_guide/dmadev.rst new file +125 -0

125 lines not on this page · raw message

doc/guides/prog_guide/img/dmadev.svg new file +283 -0

283 lines not on this page · raw message

doc/guides/prog_guide/index.rst +1 -0
index 2dce507f46..0abea06b24 100644
@@ -29,6 +29,7 @@ Programmer's Guide
     regexdev
     rte_security
     rawdev
+    dmadev
     link_bonding_poll_mode_drv_lib
     timer_lib
     hash_lib
Signature · 1 line
2.33.0
#278 lore raw

PATCH v21 6/7 dma/skeleton: introduce skeleton dmadev driver

In reply to Chengwen Feng

Skeleton dmadevice driver, on the lines of rawdev skeleton, is for showcasing of the dmadev library.

Design of skeleton involves a virtual device which is plugged into VDEV bus on initialization.

Also, enable compilation of dmadev skeleton drivers.

---
 MAINTAINERS                            |   1 +
 drivers/dma/meson.build                |  11 +
 drivers/dma/skeleton/meson.build       |   7 +
 drivers/dma/skeleton/skeleton_dmadev.c | 594 +++++++++++++++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |  61 +++
 drivers/dma/skeleton/version.map       |   3 +
 drivers/meson.build                    |   1 +
 7 files changed, 678 insertions(+)
 create mode 100644 drivers/dma/meson.build
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
MAINTAINERS +1 -0
index e237e9406b..2b505ce71e 100644
@@ -499,6 +499,7 @@ F: doc/guides/prog_guide/rawdev.rst
 DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
+F: drivers/dma/skeleton/
 F: doc/guides/prog_guide/dmadev.rst
 
 
drivers/dma/meson.build new file +11 -0
new file mode 100644
index 0000000000..0c2c34cd00
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited.
+
+if is_windows
+    subdir_done()
+endif
+
+drivers = [
+        'skeleton',
+]
+std_deps = ['dmadev']
drivers/dma/skeleton/meson.build new file +7 -0
new file mode 100644
index 0000000000..27509b1668
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited.
+
+deps += ['dmadev', 'kvargs', 'ring', 'bus_vdev']
+sources = files(
+        'skeleton_dmadev.c',
+)
drivers/dma/skeleton/skeleton_dmadev.c new file +594 -0

594 lines not on this page · raw message

drivers/dma/skeleton/skeleton_dmadev.h new file +61 -0

61 lines not on this page · raw message

drivers/dma/skeleton/version.map new file +3 -0
new file mode 100644
index 0000000000..c2e0723b4c
@@ -0,0 +1,3 @@
+DPDK_22 {
+	local: *;
+};
drivers/meson.build +1 -0
index d9e331ec85..a390787d6a 100644
@@ -18,6 +18,7 @@ subdirs = [
         'vdpa',           # depends on common, bus and mempool.
         'event',          # depends on common, bus, mempool and net.
         'baseband',       # depends on common and bus.
+        'dma',            # depends on common and bus.
 ]
 
 if meson.is_cross_build()
Signature · 1 line
2.33.0
#279 lore raw

PATCH v21 7/7 app/test: add dmadev API test

In reply to Chengwen Feng

This patch add dmadev API test which based on 'dma_skeleton' vdev. The test cases could be executed using 'dmadev_autotest' command in test framework.

---
 MAINTAINERS                |   1 +
 app/test/meson.build       |   4 +
 app/test/test_dmadev.c     |  43 +++
 app/test/test_dmadev_api.c | 543 +++++++++++++++++++++++++++++++++++++
 4 files changed, 591 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
MAINTAINERS +1 -0
index 2b505ce71e..a19a3cb53c 100644
@@ -500,6 +500,7 @@ DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
 F: drivers/dma/skeleton/
+F: app/test/test_dmadev*
 F: doc/guides/prog_guide/dmadev.rst
 
 
app/test/meson.build +4 -0
index a7611686ad..9027eba3a4 100644
@@ -43,6 +43,8 @@ test_sources = files(
         'test_debug.c',
         'test_distributor.c',
         'test_distributor_perf.c',
+        'test_dmadev.c',
+        'test_dmadev_api.c',
         'test_eal_flags.c',
         'test_eal_fs.c',
         'test_efd.c',
@@ -162,6 +164,7 @@ test_deps = [
         'cmdline',
         'cryptodev',
         'distributor',
+        'dmadev',
         'efd',
         'ethdev',
         'eventdev',
@@ -333,6 +336,7 @@ driver_test_names = [
         'cryptodev_sw_mvsam_autotest',
         'cryptodev_sw_snow3g_autotest',
         'cryptodev_sw_zuc_autotest',
+        'dmadev_autotest',
         'eventdev_selftest_octeontx',
         'eventdev_selftest_sw',
         'rawdev_autotest',
app/test/test_dmadev.c new file +43 -0

43 lines not on this page · raw message

app/test/test_dmadev_api.c new file +543 -0

543 lines not on this page · raw message

Signature · 1 line
2.33.0
#280 lore raw

v22 cover support dmadev

In reply to Chengwen Feng

This patch set contains five patch for new add dmadev.

Chengwen Feng (5):
dmadev: introduce DMA device library
dmadev: add control plane function support
dmadev: add data plane function support
dma/skeleton: introduce skeleton dmadev driver
app/test: add dmadev API test

---
v22:
* function prefix change from rte_dmadev_* to rte_dma_*.
* change to prefix comment in most scenarios.
* dmadev dev_id use int16_t type.
* fix typo.
* organize patchsets in incremental mode.
v21:
* add comment for reserved fields of struct rte_dmadev. v20:
* delete unnecessary and duplicate include header files.
* the conf_sz parameter is added to the configure and vchan-setup callbacks of the PMD, this is mainly used to enhance ABI compatibility.
* the rte_dmadev structure field is rearranged to reserve more space for I/O functions.
* fix some ambiguous and unnecessary comments.
* fix the potential memory leak of ut.
* redefine skeldma_init_once to skeldma_count.
* suppress rte_dmadev error output when execute ut.

 MAINTAINERS                            |    7 +
 app/test/meson.build                   |    4 +
 app/test/test_dmadev.c                 |   43 +
 app/test/test_dmadev_api.c             |  539 ++++++++++++
 config/rte_config.h                    |    3 +
 doc/api/doxy-api-index.md              |    1 +
 doc/api/doxy-api.conf.in               |    1 +
 doc/guides/dmadevs/index.rst           |   12 +
 doc/guides/index.rst                   |    1 +
 doc/guides/prog_guide/dmadev.rst       |  125 +++
 doc/guides/prog_guide/img/dmadev.svg   |  283 +++++++
 doc/guides/prog_guide/index.rst        |    1 +
 doc/guides/rel_notes/release_21_11.rst |    5 +
 drivers/dma/meson.build                |    6 +
 drivers/dma/skeleton/meson.build       |    7 +
 drivers/dma/skeleton/skeleton_dmadev.c |  571 +++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |   61 ++
 drivers/dma/skeleton/version.map       |    3 +
 drivers/meson.build                    |    1 +
 lib/dmadev/meson.build                 |    7 +
 lib/dmadev/rte_dmadev.c                |  718 ++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 1073 ++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  184 ++++
 lib/dmadev/rte_dmadev_pmd.h            |   60 ++
 lib/dmadev/version.map                 |   35 +
 lib/meson.build                        |    1 +
 26 files changed, 3752 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
Signature · 1 line
2.33.0
#281 lore raw

PATCH v22 1/5 dmadev: introduce DMA device library

The 'dmadevice' is a generic type of DMA device.

This patch introduce the 'dmadevice' device allocation APIs and it's multi-process support.

The infrastructure is prepared to welcome drivers in drivers/dma/

---
 MAINTAINERS                            |   5 +
 config/rte_config.h                    |   3 +
 doc/api/doxy-api-index.md              |   1 +
 doc/api/doxy-api.conf.in               |   1 +
 doc/guides/dmadevs/index.rst           |  12 +
 doc/guides/index.rst                   |   1 +
 doc/guides/prog_guide/dmadev.rst       |  62 +++++
 doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++
 doc/guides/prog_guide/index.rst        |   1 +
 doc/guides/rel_notes/release_21_11.rst |   4 +
 drivers/dma/meson.build                |   4 +
 drivers/meson.build                    |   1 +
 lib/dmadev/meson.build                 |   7 +
 lib/dmadev/rte_dmadev.c                | 358 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 133 +++++++++
 lib/dmadev/rte_dmadev_core.h           |  83 ++++++
 lib/dmadev/rte_dmadev_pmd.h            |  60 +++++
 lib/dmadev/version.map                 |  20 ++
 lib/meson.build                        |   1 +
 19 files changed, 1040 insertions(+)
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
MAINTAINERS +5 -0
index 1e0d303394..8af9522a5a 100644
@@ -452,6 +452,11 @@ F: app/test-regex/
 F: doc/guides/prog_guide/regexdev.rst
 F: doc/guides/regexdevs/features/default.ini
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <[email protected]>
+F: lib/dmadev/
+F: doc/guides/prog_guide/dmadev.rst
+
 Eventdev API
 M: Jerin Jacob <[email protected]>
 T: git://dpdk.org/next/dpdk-next-eventdev
config/rte_config.h +3 -0
index 590903c07d..6e397a62ab 100644
@@ -70,6 +70,9 @@
 /* regexdev defines */
 #define RTE_MAX_REGEXDEV_DEVS 32
 
+/* dmadev defines */
+#define RTE_DMADEV_DEFAULT_MAX_DEVS 64
+
 /* eventdev defines */
 #define RTE_EVENT_MAX_DEVS 16
 #define RTE_EVENT_MAX_QUEUES_PER_DEV 255
doc/api/doxy-api-index.md +1 -0
index 1992107a03..2939050431 100644
@@ -21,6 +21,7 @@ The public API headers are grouped by topics:
   [compressdev]        (@ref rte_compressdev.h),
   [compress]           (@ref rte_comp.h),
   [regexdev]           (@ref rte_regexdev.h),
+  [dmadev]             (@ref rte_dmadev.h),
   [eventdev]           (@ref rte_eventdev.h),
   [event_eth_rx_adapter]   (@ref rte_event_eth_rx_adapter.h),
   [event_eth_tx_adapter]   (@ref rte_event_eth_tx_adapter.h),
doc/api/doxy-api.conf.in +1 -0
index 325a0195c6..109ec1f682 100644
@@ -35,6 +35,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/compressdev \
                           @TOPDIR@/lib/cryptodev \
                           @TOPDIR@/lib/distributor \
+                          @TOPDIR@/lib/dmadev \
                           @TOPDIR@/lib/efd \
                           @TOPDIR@/lib/ethdev \
                           @TOPDIR@/lib/eventdev \
doc/guides/dmadevs/index.rst new file +12 -0
new file mode 100644
index 0000000000..0bce29d766
@@ -0,0 +1,12 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright 2021 HiSilicon Limited
+
+DMA Device Drivers
+==================
+
+The following are a list of DMA device drivers, which can be used from
+an application through DMA API.
+
+.. toctree::
+   :maxdepth: 2
+   :numbered:
doc/guides/index.rst +1 -0
index 857f0363d3..919825992e 100644
@@ -21,6 +21,7 @@ DPDK documentation
    compressdevs/index
    vdpadevs/index
    regexdevs/index
+   dmadevs/index
    eventdevs/index
    rawdevs/index
    mempool/index
doc/guides/prog_guide/dmadev.rst new file +62 -0

62 lines not on this page · raw message

doc/guides/prog_guide/img/dmadev.svg new file +283 -0

283 lines not on this page · raw message

doc/guides/prog_guide/index.rst +1 -0
index 2dce507f46..89af28dacb 100644
@@ -27,6 +27,7 @@ Programmer's Guide
     cryptodev_lib
     compressdev
     regexdev
+    dmadev
     rte_security
     rawdev
     link_bonding_poll_mode_drv_lib
doc/guides/rel_notes/release_21_11.rst +4 -0
index 43d367bcad..5a85198e0d 100644
@@ -87,6 +87,10 @@ New Features
   Added command-line options to specify total number of processes and
   current process ID. Each process owns subset of Rx and Tx queues.
 
+* **Introduced dmadev library with:**
+
+  * Device allocation and it's multi-process support.
+
 
 Removed Items
 -------------
drivers/dma/meson.build new file +4 -0
new file mode 100644
index 0000000000..a24c56d8ff
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2021 HiSilicon Limited
+
+drivers = []
drivers/meson.build +1 -0
index d9e331ec85..a390787d6a 100644
@@ -18,6 +18,7 @@ subdirs = [
         'vdpa',           # depends on common, bus and mempool.
         'event',          # depends on common, bus, mempool and net.
         'baseband',       # depends on common and bus.
+        'dma',            # depends on common and bus.
 ]
 
 if meson.is_cross_build()
lib/dmadev/meson.build new file +7 -0
new file mode 100644
index 0000000000..d2fc85e8c7
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited.
+
+sources = files('rte_dmadev.c')
+headers = files('rte_dmadev.h')
+indirect_headers += files('rte_dmadev_core.h')
+driver_sdk_headers += files('rte_dmadev_pmd.h')
lib/dmadev/rte_dmadev.c new file +358 -0

358 lines not on this page · raw message

lib/dmadev/rte_dmadev.h new file +133 -0

133 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h new file +83 -0

83 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h new file +60 -0

60 lines not on this page · raw message

lib/dmadev/version.map new file +20 -0
new file mode 100644
index 0000000000..56ea0332cb
@@ -0,0 +1,20 @@
+EXPERIMENTAL {
+	global:
+
+	rte_dma_count_avail;
+	rte_dma_dev_max;
+	rte_dma_get_dev_id;
+	rte_dma_is_valid;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	rte_dma_devices;
+	rte_dma_pmd_allocate;
+	rte_dma_pmd_release;
+
+	local: *;
+};
lib/meson.build +1 -0
index 1673ca4323..3dd920f5c5 100644
@@ -45,6 +45,7 @@ libraries = [
         'pdump',
         'rawdev',
         'regexdev',
+        'dmadev',
         'rib',
         'reorder',
         'sched',
Signature · 1 line
2.33.0
#282 lore raw

PATCH v22 2/5 dmadev: add control plane function support

In reply to Chengwen Feng

This patch add control plane functions for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  41 +++
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.c                | 360 +++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 479 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  60 ++++
 lib/dmadev/version.map                 |   9 +
 6 files changed, 950 insertions(+)
doc/guides/prog_guide/dmadev.rst +41 -0
index c1c7579107..c724fefcb0 100644
@@ -60,3 +60,44 @@ identifiers:
 
 - A device name used to designate the DMA device in console messages, for
   administration or debugging purposes.
+
+
+Device Configuration
+~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_configure API is used to configure a DMA device.
+
+.. code-block:: c
+
+   int rte_dma_configure(int16_t dev_id,
+                         const struct rte_dma_conf *dev_conf);
+
+The ``rte_dma_conf`` structure is used to pass the configuration parameters
+for the DMA device for example the number of virtual DMA channels to set up,
+indication of whether to enable silent mode.
+
+
+Configuration of Virtual DMA Channels
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_vchan_setup API is used to configure a virtual DMA channel.
+
+.. code-block:: c
+
+   int rte_dma_vchan_setup(int16_t dev_id, uint16_t vchan,
+                           const struct rte_dma_vchan_conf *conf);
+
+The ``rte_dma_vchan_conf`` structure is used to pass the configuration
+parameters for the virtual DMA channel for example transfer direction, number of
+descriptor for the virtual DMA channel, source device access port parameter,
+destination device access port parameter.
+
+
+Device Features and Capabilities
+--------------------------------
+
+DMA devices may support different feature sets. The ``rte_dma_info_get`` API
+can be used to get the device info and supported features.
+
+Silent mode is a special device capability which does not require the
+application to invoke dequeue APIs.
doc/guides/rel_notes/release_21_11.rst +1 -0
index 5a85198e0d..eb0315aaca 100644
@@ -90,6 +90,7 @@ New Features
 * **Introduced dmadev library with:**
 
   * Device allocation and it's multi-process support.
+  * Control plane functions.
 
 
 Removed Items
lib/dmadev/rte_dmadev.c +360 -0

369 lines not on this page · raw message

lib/dmadev/rte_dmadev.h +479 -0

491 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h +60 -0

84 lines not on this page · raw message

lib/dmadev/version.map +9 -0
index 56ea0332cb..6b7939b10f 100644
@@ -1,10 +1,19 @@
 EXPERIMENTAL {
 	global:
 
+	rte_dma_close;
+	rte_dma_configure;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
+	rte_dma_dump;
 	rte_dma_get_dev_id;
+	rte_dma_info_get;
 	rte_dma_is_valid;
+	rte_dma_start;
+	rte_dma_stats_get;
+	rte_dma_stats_reset;
+	rte_dma_stop;
+	rte_dma_vchan_setup;
 
 	local: *;
 };
Signature · 1 line
2.33.0
#283 lore raw

PATCH v22 3/5 dmadev: add data plane function support

In reply to Chengwen Feng

This patch add data plane functions for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  22 ++
 doc/guides/rel_notes/release_21_11.rst |   2 +-
 lib/dmadev/rte_dmadev.h                | 461 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  41 +++
 lib/dmadev/version.map                 |   6 +
 5 files changed, 531 insertions(+), 1 deletion(-)
doc/guides/prog_guide/dmadev.rst +22 -0
index c724fefcb0..44eea23202 100644
@@ -101,3 +101,25 @@ can be used to get the device info and supported features.
 
 Silent mode is a special device capability which does not require the
 application to invoke dequeue APIs.
+
+
+Enqueue / Dequeue APIs
+~~~~~~~~~~~~~~~~~~~~~~
+
+Enqueue APIs such as ``rte_dma_copy`` and ``rte_dma_fill`` can be used to
+enqueue operations to hardware. If an enqueue is successful, a ``ring_idx`` is
+returned. This ``ring_idx`` can be used by applications to track per operation
+metadata in an application-defined circular ring.
+
+The ``rte_dma_submit`` API is used to issue doorbell to hardware.
+Alternatively the ``RTE_DMA_OP_FLAG_SUBMIT`` flag can be passed to the enqueue
+APIs to also issue the doorbell to hardware.
+
+There are two dequeue APIs ``rte_dma_completed`` and
+``rte_dma_completed_status``, these are used to obtain the results of the
+enqueue requests. ``rte_dma_completed`` will return the number of successfully
+completed operations. ``rte_dma_completed_status`` will return the number of
+completed operations along with the status of each operation (filled into the
+``status`` array passed by user). These two APIs can also return the last
+completed operation's ``ring_idx`` which could help user track operations within
+their own application-defined rings.
doc/guides/rel_notes/release_21_11.rst +1 -1
index eb0315aaca..a71853b9c3 100644
@@ -90,7 +90,7 @@ New Features
 * **Introduced dmadev library with:**
 
   * Device allocation and it's multi-process support.
-  * Control plane functions.
+  * Control and data plane functions.
 
 
 Removed Items
lib/dmadev/rte_dmadev.h +461 -0

481 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h +41 -0

53 lines not on this page · raw message

lib/dmadev/version.map +6 -0
index 6b7939b10f..c780463bb2 100644
@@ -2,10 +2,15 @@ EXPERIMENTAL {
 	global:
 
 	rte_dma_close;
+	rte_dma_completed;
+	rte_dma_completed_status;
 	rte_dma_configure;
+	rte_dma_copy;
+	rte_dma_copy_sg;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
 	rte_dma_dump;
+	rte_dma_fill;
 	rte_dma_get_dev_id;
 	rte_dma_info_get;
 	rte_dma_is_valid;
@@ -13,6 +18,7 @@ EXPERIMENTAL {
 	rte_dma_stats_get;
 	rte_dma_stats_reset;
 	rte_dma_stop;
+	rte_dma_submit;
 	rte_dma_vchan_setup;
 
 	local: *;
Signature · 1 line
2.33.0
#284 lore raw

PATCH v22 4/5 dma/skeleton: introduce skeleton dmadev driver

In reply to Chengwen Feng

Skeleton dmadevice driver, on the lines of rawdev skeleton, is for showcasing of the dmadev library.

Design of skeleton involves a virtual device which is plugged into VDEV bus on initialization.

Also, enable compilation of dmadev skeleton drivers.

---
 MAINTAINERS                            |   1 +
 drivers/dma/meson.build                |   4 +-
 drivers/dma/skeleton/meson.build       |   7 +
 drivers/dma/skeleton/skeleton_dmadev.c | 571 +++++++++++++++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |  61 +++
 drivers/dma/skeleton/version.map       |   3 +
 6 files changed, 646 insertions(+), 1 deletion(-)
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
MAINTAINERS +1 -0
index 8af9522a5a..418f775382 100644
@@ -455,6 +455,7 @@ F: doc/guides/regexdevs/features/default.ini
 DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
+F: drivers/dma/skeleton/
 F: doc/guides/prog_guide/dmadev.rst
 
 Eventdev API
drivers/dma/meson.build +3 -1
index a24c56d8ff..d9c7ede32f 100644
@@ -1,4 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2021 HiSilicon Limited
 
-drivers = []
+drivers = [
+        'skeleton',
+]
drivers/dma/skeleton/meson.build new file +7 -0
new file mode 100644
index 0000000000..8871b80956
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited
+
+deps += ['dmadev', 'kvargs', 'ring', 'bus_vdev']
+sources = files(
+        'skeleton_dmadev.c',
+)
drivers/dma/skeleton/skeleton_dmadev.c new file +571 -0

571 lines not on this page · raw message

drivers/dma/skeleton/skeleton_dmadev.h new file +61 -0

61 lines not on this page · raw message

drivers/dma/skeleton/version.map new file +3 -0
new file mode 100644
index 0000000000..c2e0723b4c
@@ -0,0 +1,3 @@
+DPDK_22 {
+	local: *;
+};
Signature · 1 line
2.33.0
#285 lore raw

PATCH v22 5/5 app/test: add dmadev API test

In reply to Chengwen Feng

This patch add dmadev API test which based on 'dma_skeleton' vdev. The test cases could be executed using 'dmadev_autotest' command in test framework.

---
 MAINTAINERS                |   1 +
 app/test/meson.build       |   4 +
 app/test/test_dmadev.c     |  43 +++
 app/test/test_dmadev_api.c | 539 +++++++++++++++++++++++++++++++++++++
 4 files changed, 587 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
MAINTAINERS +1 -0
index 418f775382..9966797d12 100644
@@ -456,6 +456,7 @@ DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
 F: drivers/dma/skeleton/
+F: app/test/test_dmadev*
 F: doc/guides/prog_guide/dmadev.rst
 
 Eventdev API
app/test/meson.build +4 -0
index a7611686ad..9027eba3a4 100644
@@ -43,6 +43,8 @@ test_sources = files(
         'test_debug.c',
         'test_distributor.c',
         'test_distributor_perf.c',
+        'test_dmadev.c',
+        'test_dmadev_api.c',
         'test_eal_flags.c',
         'test_eal_fs.c',
         'test_efd.c',
@@ -162,6 +164,7 @@ test_deps = [
         'cmdline',
         'cryptodev',
         'distributor',
+        'dmadev',
         'efd',
         'ethdev',
         'eventdev',
@@ -333,6 +336,7 @@ driver_test_names = [
         'cryptodev_sw_mvsam_autotest',
         'cryptodev_sw_snow3g_autotest',
         'cryptodev_sw_zuc_autotest',
+        'dmadev_autotest',
         'eventdev_selftest_octeontx',
         'eventdev_selftest_sw',
         'rawdev_autotest',
app/test/test_dmadev.c new file +43 -0

43 lines not on this page · raw message

app/test/test_dmadev_api.c new file +539 -0

539 lines not on this page · raw message

Signature · 1 line
2.33.0
#286 lore raw

v23 cover support dmadev

In reply to Chengwen Feng

This patch set contains six patch for new add dmadev.

Chengwen Feng (6):
dmadev: introduce DMA device library
dmadev: add control plane function support
dmadev: add data plane function support
dmadev: add multi-process support
dma/skeleton: introduce skeleton dmadev driver
app/test: add dmadev API test

---
v23:
* split multi-process support from 1st patch.
* fix some static check warning.
* fix skeleton cpu thread zero_req_count flip bug.
* add test_dmadev_api.h.
* add the description of modifying the dmadev state when init OK. v22:
* function prefix change from rte_dmadev_* to rte_dma_*.
* change to prefix comment in most scenarios.
* dmadev dev_id use int16_t type.
* fix typo.
* organize patchsets in incremental mode.
v21:
* add comment for reserved fields of struct rte_dmadev. v20:
* delete unnecessary and duplicate include header files.
* the conf_sz parameter is added to the configure and vchan-setup callbacks of the PMD, this is mainly used to enhance ABI compatibility.
* the rte_dmadev structure field is rearranged to reserve more space for I/O functions.
* fix some ambiguous and unnecessary comments.
* fix the potential memory leak of ut.
* redefine skeldma_init_once to skeldma_count.
* suppress rte_dmadev error output when execute ut.

 MAINTAINERS                            |    7 +
 app/test/meson.build                   |    4 +
 app/test/test_dmadev.c                 |   41 +
 app/test/test_dmadev_api.c             |  574 +++++++++++++
 app/test/test_dmadev_api.h             |    5 +
 config/rte_config.h                    |    3 +
 doc/api/doxy-api-index.md              |    1 +
 doc/api/doxy-api.conf.in               |    1 +
 doc/guides/dmadevs/index.rst           |   12 +
 doc/guides/index.rst                   |    1 +
 doc/guides/prog_guide/dmadev.rst       |  127 +++
 doc/guides/prog_guide/img/dmadev.svg   |  283 +++++++
 doc/guides/prog_guide/index.rst        |    1 +
 doc/guides/rel_notes/release_21_11.rst |    7 +
 drivers/dma/meson.build                |    6 +
 drivers/dma/skeleton/meson.build       |    7 +
 drivers/dma/skeleton/skeleton_dmadev.c |  570 +++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |   61 ++
 drivers/dma/skeleton/version.map       |    3 +
 drivers/meson.build                    |    1 +
 lib/dmadev/meson.build                 |    7 +
 lib/dmadev/rte_dmadev.c                |  728 ++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 1074 ++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  181 ++++
 lib/dmadev/rte_dmadev_pmd.h            |   60 ++
 lib/dmadev/version.map                 |   35 +
 lib/meson.build                        |    1 +
 27 files changed, 3801 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
 create mode 100644 app/test/test_dmadev_api.h
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
Signature · 1 line
2.33.0
#287 lore raw

PATCH v23 1/6 dmadev: introduce DMA device library

The 'dmadevice' is a generic type of DMA device.

This patch introduce the 'dmadevice' device allocation APIs.

The infrastructure is prepared to welcome drivers in drivers/dma/

---
 MAINTAINERS                            |   5 +
 config/rte_config.h                    |   3 +
 doc/api/doxy-api-index.md              |   1 +
 doc/api/doxy-api.conf.in               |   1 +
 doc/guides/dmadevs/index.rst           |  12 ++
 doc/guides/index.rst                   |   1 +
 doc/guides/prog_guide/dmadev.rst       |  64 ++++++
 doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
 doc/guides/prog_guide/index.rst        |   1 +
 doc/guides/rel_notes/release_21_11.rst |   4 +
 drivers/dma/meson.build                |   4 +
 drivers/meson.build                    |   1 +
 lib/dmadev/meson.build                 |   7 +
 lib/dmadev/rte_dmadev.c                | 263 +++++++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 134 ++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  51 +++++
 lib/dmadev/rte_dmadev_pmd.h            |  60 ++++++
 lib/dmadev/version.map                 |  20 ++
 lib/meson.build                        |   1 +
 19 files changed, 916 insertions(+)
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
MAINTAINERS +5 -0
index 77a549a5e8..a5b11ac70b 100644
@@ -454,6 +454,11 @@ F: app/test-regex/
 F: doc/guides/prog_guide/regexdev.rst
 F: doc/guides/regexdevs/features/default.ini
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <[email protected]>
+F: lib/dmadev/
+F: doc/guides/prog_guide/dmadev.rst
+
 Eventdev API
 M: Jerin Jacob <[email protected]>
 T: git://dpdk.org/next/dpdk-next-eventdev
config/rte_config.h +3 -0
index 590903c07d..6e397a62ab 100644
@@ -70,6 +70,9 @@
 /* regexdev defines */
 #define RTE_MAX_REGEXDEV_DEVS 32
 
+/* dmadev defines */
+#define RTE_DMADEV_DEFAULT_MAX_DEVS 64
+
 /* eventdev defines */
 #define RTE_EVENT_MAX_DEVS 16
 #define RTE_EVENT_MAX_QUEUES_PER_DEV 255
doc/api/doxy-api-index.md +1 -0
index 1992107a03..2939050431 100644
@@ -21,6 +21,7 @@ The public API headers are grouped by topics:
   [compressdev]        (@ref rte_compressdev.h),
   [compress]           (@ref rte_comp.h),
   [regexdev]           (@ref rte_regexdev.h),
+  [dmadev]             (@ref rte_dmadev.h),
   [eventdev]           (@ref rte_eventdev.h),
   [event_eth_rx_adapter]   (@ref rte_event_eth_rx_adapter.h),
   [event_eth_tx_adapter]   (@ref rte_event_eth_tx_adapter.h),
doc/api/doxy-api.conf.in +1 -0
index 325a0195c6..109ec1f682 100644
@@ -35,6 +35,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/compressdev \
                           @TOPDIR@/lib/cryptodev \
                           @TOPDIR@/lib/distributor \
+                          @TOPDIR@/lib/dmadev \
                           @TOPDIR@/lib/efd \
                           @TOPDIR@/lib/ethdev \
                           @TOPDIR@/lib/eventdev \
doc/guides/dmadevs/index.rst new file +12 -0
new file mode 100644
index 0000000000..0bce29d766
@@ -0,0 +1,12 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright 2021 HiSilicon Limited
+
+DMA Device Drivers
+==================
+
+The following are a list of DMA device drivers, which can be used from
+an application through DMA API.
+
+.. toctree::
+   :maxdepth: 2
+   :numbered:
doc/guides/index.rst +1 -0
index 857f0363d3..919825992e 100644
@@ -21,6 +21,7 @@ DPDK documentation
    compressdevs/index
    vdpadevs/index
    regexdevs/index
+   dmadevs/index
    eventdevs/index
    rawdevs/index
    mempool/index
doc/guides/prog_guide/dmadev.rst new file +64 -0

64 lines not on this page · raw message

doc/guides/prog_guide/img/dmadev.svg new file +283 -0

283 lines not on this page · raw message

doc/guides/prog_guide/index.rst +1 -0
index 2dce507f46..89af28dacb 100644
@@ -27,6 +27,7 @@ Programmer's Guide
     cryptodev_lib
     compressdev
     regexdev
+    dmadev
     rte_security
     rawdev
     link_bonding_poll_mode_drv_lib
doc/guides/rel_notes/release_21_11.rst +4 -0
index 19356ac53c..74639f1e81 100644
@@ -106,6 +106,10 @@ New Features
   Added command-line options to specify total number of processes and
   current process ID. Each process owns subset of Rx and Tx queues.
 
+* **Introduced dmadev library with:**
+
+  * Device allocation APIs.
+
 
 Removed Items
 -------------
drivers/dma/meson.build new file +4 -0
new file mode 100644
index 0000000000..a24c56d8ff
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2021 HiSilicon Limited
+
+drivers = []
drivers/meson.build +1 -0
index 3d08540581..b7d680868a 100644
@@ -18,6 +18,7 @@ subdirs = [
         'vdpa',           # depends on common, bus and mempool.
         'event',          # depends on common, bus, mempool and net.
         'baseband',       # depends on common and bus.
+        'dma',            # depends on common and bus.
 ]
 
 if meson.is_cross_build()
lib/dmadev/meson.build new file +7 -0
new file mode 100644
index 0000000000..d2fc85e8c7
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited.
+
+sources = files('rte_dmadev.c')
+headers = files('rte_dmadev.h')
+indirect_headers += files('rte_dmadev_core.h')
+driver_sdk_headers += files('rte_dmadev_pmd.h')
lib/dmadev/rte_dmadev.c new file +263 -0

263 lines not on this page · raw message

lib/dmadev/rte_dmadev.h new file +134 -0

134 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h new file +51 -0

51 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h new file +60 -0

60 lines not on this page · raw message

lib/dmadev/version.map new file +20 -0
new file mode 100644
index 0000000000..56ea0332cb
@@ -0,0 +1,20 @@
+EXPERIMENTAL {
+	global:
+
+	rte_dma_count_avail;
+	rte_dma_dev_max;
+	rte_dma_get_dev_id;
+	rte_dma_is_valid;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	rte_dma_devices;
+	rte_dma_pmd_allocate;
+	rte_dma_pmd_release;
+
+	local: *;
+};
lib/meson.build +1 -0
index 1673ca4323..3dd920f5c5 100644
@@ -45,6 +45,7 @@ libraries = [
         'pdump',
         'rawdev',
         'regexdev',
+        'dmadev',
         'rib',
         'reorder',
         'sched',
Signature · 1 line
2.33.0
#288 lore raw
Quoted 36 lines · Chengwen Feng

The 'dmadevice' is a generic type of DMA device.

This patch introduce the 'dmadevice' device allocation APIs.

The infrastructure is prepared to welcome drivers in drivers/dma/

---
 MAINTAINERS                            |   5 +
 config/rte_config.h                    |   3 +
 doc/api/doxy-api-index.md              |   1 +
 doc/api/doxy-api.conf.in               |   1 +
 doc/guides/dmadevs/index.rst           |  12 ++
 doc/guides/index.rst                   |   1 +
 doc/guides/prog_guide/dmadev.rst       |  64 ++++++
 doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
 doc/guides/prog_guide/index.rst        |   1 +
 doc/guides/rel_notes/release_21_11.rst |   4 +
 drivers/dma/meson.build                |   4 +
 drivers/meson.build                    |   1 +
 lib/dmadev/meson.build                 |   7 +
 lib/dmadev/rte_dmadev.c                | 263 +++++++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 134 ++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  51 +++++
 lib/dmadev/rte_dmadev_pmd.h            |  60 ++++++
 lib/dmadev/version.map                 |  20 ++
 lib/meson.build                        |   1 +
 19 files changed, 916 insertions(+)
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
Quoted 9 lines
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map

<snip>
Hi Chengwen,
I see that the new version removed the "rte_dmadev_get_device_by_name()". What is the way to get the dmadev from inside the PMD .remove ? I am looking to get the dev_private as we need to do some cleanup operations from the remove function.

regards,
Radha Mohan

#289 lore raw
Quoted 52 lines · Radha Mohan
Quoted 33 lines · Chengwen Feng

The 'dmadevice' is a generic type of DMA device.

This patch introduce the 'dmadevice' device allocation APIs.

The infrastructure is prepared to welcome drivers in drivers/dma/

---
  MAINTAINERS                            |   5 +
  config/rte_config.h                    |   3 +
  doc/api/doxy-api-index.md              |   1 +
  doc/api/doxy-api.conf.in               |   1 +
  doc/guides/dmadevs/index.rst           |  12 ++
  doc/guides/index.rst                   |   1 +
  doc/guides/prog_guide/dmadev.rst       |  64 ++++++
  doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
  doc/guides/prog_guide/index.rst        |   1 +
  doc/guides/rel_notes/release_21_11.rst |   4 +
  drivers/dma/meson.build                |   4 +
  drivers/meson.build                    |   1 +
  lib/dmadev/meson.build                 |   7 +
  lib/dmadev/rte_dmadev.c                | 263 +++++++++++++++++++++++
  lib/dmadev/rte_dmadev.h                | 134 ++++++++++++
  lib/dmadev/rte_dmadev_core.h           |  51 +++++
  lib/dmadev/rte_dmadev_pmd.h            |  60 ++++++
  lib/dmadev/version.map                 |  20 ++
  lib/meson.build                        |   1 +
  19 files changed, 916 insertions(+)
Quoted 11 lines

create mode 100644 doc/guides/dmadevs/index.rst create mode 100644 doc/guides/prog_guide/dmadev.rst create mode 100644 doc/guides/prog_guide/img/dmadev.svg create mode 100644 drivers/dma/meson.build create mode 100644 lib/dmadev/meson.build create mode 100644 lib/dmadev/rte_dmadev.c create mode 100644 lib/dmadev/rte_dmadev.h create mode 100644 lib/dmadev/rte_dmadev_core.h create mode 100644 lib/dmadev/rte_dmadev_pmd.h create mode 100644 lib/dmadev/version.map

<snip>
Hi Chengwen,
I see that the new version removed the "rte_dmadev_get_device_by_name()". What is the way to get the dmadev from inside the PMD .remove ? I am looking to get the dev_private as we need to do some cleanup operations from the remove function.

Quoted 2 lines

regards,
Radha Mohan

Hi Radha,

You can use rte_dma_get_dev_id(name) to get the device ID, which can then be used to get the rte_dma_dev struct (which contains dev_private) for that device from rte_dma_devices[].

See "idxd_dmadev_destroy()" in http://patches.dpdk.org/project/dpdk/patch/[email protected]/ for an example.

Hope that helps,

Kevin

#290 lore raw
Quoted 63 lines · Kevin Laatz
Quoted 52 lines · Radha Mohan
Quoted 33 lines · Chengwen Feng

The 'dmadevice' is a generic type of DMA device.

This patch introduce the 'dmadevice' device allocation APIs.

The infrastructure is prepared to welcome drivers in drivers/dma/

---
  MAINTAINERS                            |   5 +
  config/rte_config.h                    |   3 +
  doc/api/doxy-api-index.md              |   1 +
  doc/api/doxy-api.conf.in               |   1 +
  doc/guides/dmadevs/index.rst           |  12 ++
  doc/guides/index.rst                   |   1 +
  doc/guides/prog_guide/dmadev.rst       |  64 ++++++
  doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
  doc/guides/prog_guide/index.rst        |   1 +
  doc/guides/rel_notes/release_21_11.rst |   4 +
  drivers/dma/meson.build                |   4 +
  drivers/meson.build                    |   1 +
  lib/dmadev/meson.build                 |   7 +
  lib/dmadev/rte_dmadev.c                | 263 +++++++++++++++++++++++
  lib/dmadev/rte_dmadev.h                | 134 ++++++++++++
  lib/dmadev/rte_dmadev_core.h           |  51 +++++
  lib/dmadev/rte_dmadev_pmd.h            |  60 ++++++
  lib/dmadev/version.map                 |  20 ++
  lib/meson.build                        |   1 +
  19 files changed, 916 insertions(+)
Quoted 11 lines

create mode 100644 doc/guides/dmadevs/index.rst create mode 100644 doc/guides/prog_guide/dmadev.rst create mode 100644 doc/guides/prog_guide/img/dmadev.svg create mode 100644 drivers/dma/meson.build create mode 100644 lib/dmadev/meson.build create mode 100644 lib/dmadev/rte_dmadev.c create mode 100644 lib/dmadev/rte_dmadev.h create mode 100644 lib/dmadev/rte_dmadev_core.h create mode 100644 lib/dmadev/rte_dmadev_pmd.h create mode 100644 lib/dmadev/version.map

<snip>
Hi Chengwen,
I see that the new version removed the "rte_dmadev_get_device_by_name()". What is the way to get the dmadev from inside the PMD .remove ? I am looking to get the dev_private as we need to do some cleanup operations from the remove function.

Quoted 2 lines

regards,
Radha Mohan

Hi Radha,

You can use rte_dma_get_dev_id(name) to get the device ID, which can then be used to get the rte_dma_dev struct (which contains dev_private) for that device from rte_dma_devices[].

Quoted 5 lines

See "idxd_dmadev_destroy()" in
http://patches.dpdk.org/project/dpdk/patch/[email protected]/
for an example.

Hope that helps,

Thanks Kevin. It helped. I wasn't looking at accessing the rte_dma_devices[] array directly. A library API would've been good.

regards,
Radha

Quoted 4 lines

Kevin

#291 lore raw
In reply to Radha Mohan
Quoted 46 lines · Radha Mohan
Quoted 36 lines · Chengwen Feng

The 'dmadevice' is a generic type of DMA device.

This patch introduce the 'dmadevice' device allocation APIs.

The infrastructure is prepared to welcome drivers in drivers/dma/

---
 MAINTAINERS                            |   5 +
 config/rte_config.h                    |   3 +
 doc/api/doxy-api-index.md              |   1 +
 doc/api/doxy-api.conf.in               |   1 +
 doc/guides/dmadevs/index.rst           |  12 ++
 doc/guides/index.rst                   |   1 +
 doc/guides/prog_guide/dmadev.rst       |  64 ++++++
 doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
 doc/guides/prog_guide/index.rst        |   1 +
 doc/guides/rel_notes/release_21_11.rst |   4 +
 drivers/dma/meson.build                |   4 +
 drivers/meson.build                    |   1 +
 lib/dmadev/meson.build                 |   7 +
 lib/dmadev/rte_dmadev.c                | 263 +++++++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 134 ++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  51 +++++
 lib/dmadev/rte_dmadev_pmd.h            |  60 ++++++
 lib/dmadev/version.map                 |  20 ++
 lib/meson.build                        |   1 +
 19 files changed, 916 insertions(+)
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
Quoted 9 lines
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
Quoted 6 lines

<snip>
Hi Chengwen,
I see that the new version removed the "rte_dmadev_get_device_by_name()". What is the way to get the dmadev from inside the PMD .remove ? I am looking to get the dev_private as we need to do some cleanup operations from the remove function.

Hi Radha,

PMD should invoke rte_dma_pmd_release when .remove, and the rte_dma_pmd_release will call dev_close ops, so that PMD could do some cleanup operations in dev_close ops.

Thanks

Quoted 6 lines

regards,
Radha Mohan

.

#292 lore raw
In reply to Chengwen Feng

24/09/2021 12:53, Chengwen Feng:

Quoted 1 line

The 'dmadevice' is a generic type of DMA device.

Do you mean 'dmadev' ?

Quoted 3 lines

This patch introduce the 'dmadevice' device allocation APIs.

The infrastructure is prepared to welcome drivers in drivers/dma/

Good

[...]

doc/guides/prog_guide/dmadev.rst line 8

Quoted 2 lines
doc/guides/prog_guide/dmadev.rst new file +2 -0
+The DMA library provides a DMA device framework for management and provisioning
+of hardware and software DMA poll mode drivers, defining generic APIs which

We could consider the whole as *one* API.

doc/guides/prog_guide/dmadev.rst

Quoted 8 lines
doc/guides/prog_guide/dmadev.rst new file +8 -0
+support a number of different DMA operations.
+
+
+Design Principles
+-----------------
+
+The DMA library follows the same basic principles as those used in DPDK's
+Ethernet Device framework and the RegEx framework.

Not sure what this sentence means. Better to remove.

doc/guides/prog_guide/dmadev.rst line 20

Quoted 5 lines

The DMA framework provides

doc/guides/prog_guide/dmadev.rst new file +4 -0
+a generic DMA device framework which supports both physical (hardware)
+and virtual (software) DMA devices as well as a generic DMA API which allows
+DMA devices to be managed and configured and supports DMA operations to be
+provisioned on DMA poll mode driver.

You could split this long sentence.

[...]

doc/guides/prog_guide/dmadev.rst line 48

Quoted 6 lines
doc/guides/prog_guide/dmadev.rst new file +6 -0
+Physical DMA controllers are discovered during the PCI probe/enumeration of the
+EAL function which is executed at DPDK initialization, this is based on their
+PCI BDF (bus/bridge, device, function). Specific physical DMA controllers, like
+other physical devices in DPDK can be listed using the EAL command line options.
+
+The dmadevs are dynamically allocated by using the API

not API, but function

Quoted 3 lines

+``rte_dma_pmd_allocate`` based on the number of hardware DMA channels. After the +dmadev initialized successfully, the driver needs to switch the dmadev state to +``RTE_DMA_DEV_READY``.

Are we sure we need these details?

doc/guides/prog_guide/dmadev.rst line 64

Quoted 11 lines
doc/guides/prog_guide/dmadev.rst new file +11 -0
+Device Identification
+~~~~~~~~~~~~~~~~~~~~~
+
+Each DMA device, whether physical or virtual is uniquely designated by two
+identifiers:
+
+- A unique device index used to designate the DMA device in all functions
+  exported by the DMA API.
+
+- A device name used to designate the DMA device in console messages, for
+  administration or debugging purposes.

Good

[...]

doc/guides/rel_notes/release_21_11.rst New Features · line 111

Quoted 9 lines
doc/guides/rel_notes/release_21_11.rst +3 -0
@@ -106,6 +106,10 @@ New Features
   Added command-line options to specify total number of processes and
   current process ID. Each process owns subset of Rx and Tx queues.
 
+* **Introduced dmadev library with:**
+
+  * Device allocation APIs.

There is no API for that, it is internal.
From an user perspective, you need only to list outstanding features in the release notes.

[...]

doc/guides/rel_notes/release_21_11.rst New Features

Quoted 5 lines
doc/guides/rel_notes/release_21_11.rst +5 -0
+++ b/lib/dmadev/rte_dmadev.c
+RTE_LOG_REGISTER_DEFAULT(rte_dma_logtype, INFO);
+#define RTE_DMA_LOG(level, fmt, args...) \
+	rte_log(RTE_LOG_ ## level, rte_dma_logtype, "%s(): " fmt "\n", \
+		__func__, ##args)

I don't like having function name in all logs.
I recommend this form of macro:

#define RTE_DMA_LOG(level, ...) \
    rte_log(RTE_LOG_ ## level, rte_dma_logtype, RTE_FMT("dma: " \
        RTE_FMT_HEAD(__VA_ARGS__,) "\n", RTE_FMT_TAIL(__VA_ARGS__,)))

doc/guides/rel_notes/release_21_11.rst

Quoted 7 lines
doc/guides/rel_notes/release_21_11.rst +7 -0
+/* Macros to check for valid device id */
+#define RTE_DMA_VALID_DEV_ID_OR_ERR_RET(dev_id, retval) do { \
+	if (!rte_dma_is_valid(dev_id)) { \
+		RTE_DMA_LOG(ERR, "Invalid dev_id=%d", dev_id); \
+		return retval; \
+	} \
+} while (0)

I dislike this macro doing a return. It is hiding stuff. I know we have it in other classes but I think it is a mistake, we should avoid macro blocks.

doc/guides/rel_notes/release_21_11.rst

Quoted 2 lines
doc/guides/rel_notes/release_21_11.rst +2 -0
+static int16_t
+dma_find_free_dev(void)

Actually it is looking for an ID,
so it should be dma_find_free_id.

doc/guides/rel_notes/release_21_11.rst

Quoted 8 lines
doc/guides/rel_notes/release_21_11.rst +8 -0
+{
+	int16_t i;
+
+	if (rte_dma_devices == NULL)
+		return -1;
+
+	for (i = 0; i < dma_devices_max; i++) {
+		if (rte_dma_devices[i].dev_name[0] == '\0')

Instead of checking its name, it looks more logical to check the state.

doc/guides/rel_notes/release_21_11.rst

Quoted 8 lines
doc/guides/rel_notes/release_21_11.rst +8 -0
+			return i;
+	}
+
+	return -1;
+}
+
+static struct rte_dma_dev*
+dma_find(const char *name)

dma_find_by_name?

[...]

Quoted 5 lines

+++ b/lib/dmadev/rte_dmadev.h
+ * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
+ * PCI/SoC device probing phase performed at EAL initialization time. And could
+ * be released by rte_dma_pmd_release() during the PCI/SoC device removing
+ * phase.

I don't think this text has value,
and we could imagine allocating a device ata later stage.

[...]

Quoted 3 lines

+ * Configure the maximum number of dmadevs.
+ * @note This function can be invoked before the primary process rte_eal_init()
+ * to change the maximum number of dmadevs.

You should mention what is the default.
Is the default exported to the app in this file?

doc/guides/rel_notes/release_21_11.rst

Quoted 9 lines

+ *
+ * @param dev_max
+ * maximum number of dmadevs.
+ *
+ * @return
+ * 0 on success. Otherwise negative value is returned.
+ */

doc/guides/rel_notes/release_21_11.rst +2 -0
+__rte_experimental
+int rte_dma_dev_max(size_t dev_max);

What about a function able to do more with the name rte_dma_init? It should allocate the inter-process shared memory,
and do the lookup in case of secondary process.

Quoted 5 lines
doc/guides/rel_notes/release_21_11.rst +5 -0
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get the device identifier for the named DMA device.

doc/guides/rel_notes/release_21_11.rst

Quoted 10 lines
doc/guides/rel_notes/release_21_11.rst +10 -0
+ *
+ * @param name
+ *   DMA device name.
+ *
+ * @return
+ *   Returns DMA device identifier on success.
+ *   - <0: Failure to find named DMA device.
+ */
+__rte_experimental
+int rte_dma_get_dev_id(const char *name);

Should we add _by_name?
We could have a function to retrieve the ID by devargs as well.

Quoted 10 lines

+++ b/lib/dmadev/rte_dmadev_core.h
+/**
+ * @file
+ *
+ * DMA Device internal header.
+ *
+ * This header contains internal data types, that are used by the DMA devices
+ * in order to expose their ops to the class.
+ *
+ * Applications should not use these API directly.

If it is not part of the API, it should not be exposed at all. Why not having all these stuff in a file dmadev_driver.h? Is it used by some inline functions?

[...]

Quoted 7 lines

+++ b/lib/dmadev/rte_dmadev_pmd.h
+/**
+ * @file
+ *
+ * DMA Device PMD APIs
+ *
+ * Driver facing APIs for a DMA device. These are not to be called directly by

You cannot say API for drivers, because API means application interface. What you mean is "driver interface".

Quoted 2 lines

+ * any application.
+ */

[...]

Quoted 2 lines

+ * Allocates a new dmadev slot for an DMA device and returns the pointer
+ * to that slot for the driver to use.

Please in all comments, use the infinitive form. Example:

	Allocates -> Allocate
Quoted 5 lines

+ *
+ * @param name
+ * DMA device name.
+ * @param numa_node
+ * Driver's private data's numa node.

s/numa/NUMA/

doc/guides/rel_notes/release_21_11.rst

Quoted 10 lines

+ * @param private_data_size
+ * Driver's private data size.
+ *
+ * @return
+ * A pointer to the DMA device slot case of success,
+ * NULL otherwise.
+ */

doc/guides/rel_notes/release_21_11.rst +3 -0
+__rte_internal
+struct rte_dma_dev *rte_dma_pmd_allocate(const char *name, int numa_node,
+					 size_t private_data_size);

OK, sorry there are a lot of comments.
Overrall, that's a good work.
I know you are in holidays, I hope we can finish during next week.

#293 lore raw
Quoted 14 lines · Thomas Monjalon

24/09/2021 12:53, Chengwen Feng:

Quoted 1 line

The 'dmadevice' is a generic type of DMA device.

Do you mean 'dmadev' ?

Quoted 3 lines

This patch introduce the 'dmadevice' device allocation APIs.

The infrastructure is prepared to welcome drivers in drivers/dma/

Good

[...]

Quoted 2 lines
doc/guides/prog_guide/dmadev.rst new file +2 -0
+The DMA library provides a DMA device framework for management and provisioning
+of hardware and software DMA poll mode drivers, defining generic APIs which

[snip]

Quoted 10 lines

[...]

Quoted 5 lines

+++ b/lib/dmadev/rte_dmadev.h
+ * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
+ * PCI/SoC device probing phase performed at EAL initialization time. And could
+ * be released by rte_dma_pmd_release() during the PCI/SoC device removing
+ * phase.

I don't think this text has value,
and we could imagine allocating a device ata later stage.

Yes, we could remove the stage descriptor because it's a well-known knowledge, but I recommend keeping the rte_dma_pmd_allocate and rte_dma_pmd_release functions, how about:

* The dmadev are dynamically allocated by rte_dma_pmd_allocate(). And could
* be released by rte_dma_pmd_release().

Quoted 8 lines

[...]

Quoted 3 lines

+ * Configure the maximum number of dmadevs.
+ * @note This function can be invoked before the primary process rte_eal_init()
+ * to change the maximum number of dmadevs.

You should mention what is the default.
Is the default exported to the app in this file?

The default macro is RTE_DMADEV_DEFAULT_MAX_DEVS, and I place it at rte_config.h.

I think it's better to focus on one place (rte_config.h) than to modify config in multiple places (e.g. rte_dmadev.h/rte_xxx.h).

Quoted 14 lines

lib/dmadev/rte_dmadev.h line 84

Quoted 10 lines

+ *
+ * @param dev_max
+ * maximum number of dmadevs.
+ *
+ * @return
+ * 0 on success. Otherwise negative value is returned.
+ */

lib/dmadev/rte_dmadev.h new file +2 -0
+__rte_experimental
+int rte_dma_dev_max(size_t dev_max);

What about a function able to do more with the name rte_dma_init? It should allocate the inter-process shared memory,
and do the lookup in case of secondary process.

Yes, we defined dma_data_prepare() which do above thing, it's in 4th patch.

Because we could not invoke some like allocate inter-process shared memory before rte_eal_init, so I think it's better keep rte_dma_dev_max as it is.

Quoted 20 lines
Quoted 6 lines
lib/dmadev/rte_dmadev.h new file +5 -0
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get the device identifier for the named DMA device.

lib/dmadev/rte_dmadev.h line 100

Quoted 10 lines
lib/dmadev/rte_dmadev.h new file +10 -0
+ *
+ * @param name
+ *   DMA device name.
+ *
+ * @return
+ *   Returns DMA device identifier on success.
+ *   - <0: Failure to find named DMA device.
+ */
+__rte_experimental
+int rte_dma_get_dev_id(const char *name);

Should we add _by_name?
We could have a function to retrieve the ID by devargs as well.

Quoted 14 lines
Quoted 10 lines

+++ b/lib/dmadev/rte_dmadev_core.h
+/**
+ * @file
+ *
+ * DMA Device internal header.
+ *
+ * This header contains internal data types, that are used by the DMA devices
+ * in order to expose their ops to the class.
+ *
+ * Applications should not use these API directly.

If it is not part of the API, it should not be exposed at all. Why not having all these stuff in a file dmadev_driver.h? Is it used by some inline functions?

Yes, it's used by dataplane inline functions.

[snip]

Quoted 2 lines

.

Thanks

#294 lore raw

08/10/2021 09:13, fengchengwen:

Quoted 11 lines · Thomas Monjalon
Quoted 9 lines

24/09/2021 12:53, Chengwen Feng:

Quoted 5 lines

+++ b/lib/dmadev/rte_dmadev.h
+ * The dmadev are dynamically allocated by rte_dma_pmd_allocate() during the
+ * PCI/SoC device probing phase performed at EAL initialization time. And could
+ * be released by rte_dma_pmd_release() during the PCI/SoC device removing
+ * phase.

I don't think this text has value,
and we could imagine allocating a device ata later stage.

Quoted 5 lines

Yes, we could remove the stage descriptor because it's a well-known knowledge, but I recommend keeping the rte_dma_pmd_allocate and rte_dma_pmd_release functions, how about:

* The dmadev are dynamically allocated by rte_dma_pmd_allocate(). And could
* be released by rte_dma_pmd_release().

These functions are for PMD.
This file is for applications, so it is not appropriate.

Quoted 9 lines
Quoted 7 lines

[...]

Quoted 3 lines

+ * Configure the maximum number of dmadevs.
+ * @note This function can be invoked before the primary process rte_eal_init()
+ * to change the maximum number of dmadevs.

You should mention what is the default.
Is the default exported to the app in this file?

The default macro is RTE_DMADEV_DEFAULT_MAX_DEVS, and I place it at rte_config.h.

No we avoid adding thinds in rte_config.h.
There should a static default which can be changed at runtime only.

Quoted 1 line

I think it's better to focus on one place (rte_config.h) than to modify config in multiple places (e.g. rte_dmadev.h/rte_xxx.h).

Config is modified only in one place: the function.

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines

+ *
+ * @param dev_max
+ * maximum number of dmadevs.
+ *
+ * @return

Quoted 8 lines

lib/dmadev/rte_dmadev.h line 84

Quoted 4 lines

+ * 0 on success. Otherwise negative value is returned.
+ */

lib/dmadev/rte_dmadev.h new file +2 -0
+__rte_experimental
+int rte_dma_dev_max(size_t dev_max);

What about a function able to do more with the name rte_dma_init? It should allocate the inter-process shared memory,
and do the lookup in case of secondary process.

Quoted 4 lines

Yes, we defined dma_data_prepare() which do above thing, it's in 4th patch.

Because we could not invoke some like allocate inter-process shared memory before rte_eal_init, so I think it's better keep rte_dma_dev_max as it is.

Good point.

Quoted 15 lines
Quoted 6 lines
Quoted 6 lines

+++ b/lib/dmadev/rte_dmadev_core.h
+/**
+ * @file
+ *
+ * DMA Device internal header.
+ *

Quoted 8 lines
Quoted 4 lines

+ * This header contains internal data types, that are used by the DMA devices
+ * in order to expose their ops to the class.
+ *
+ * Applications should not use these API directly.

If it is not part of the API, it should not be exposed at all. Why not having all these stuff in a file dmadev_driver.h? Is it used by some inline functions?

Quoted 1 line

Yes, it's used by dataplane inline functions.

OK, please give this reason in the description.

#295 lore raw

PATCH v23 2/6 dmadev: add control plane function support

In reply to Chengwen Feng

This patch add control plane functions for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  41 +++
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.c                | 359 ++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 480 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  62 +++-
 lib/dmadev/version.map                 |   9 +
 6 files changed, 951 insertions(+), 1 deletion(-)
doc/guides/prog_guide/dmadev.rst +41 -0
index 822282213c..c2b0b0420b 100644
@@ -62,3 +62,44 @@ identifiers:
 
 - A device name used to designate the DMA device in console messages, for
   administration or debugging purposes.
+
+
+Device Configuration
+~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_configure API is used to configure a DMA device.
+
+.. code-block:: c
+
+   int rte_dma_configure(int16_t dev_id,
+                         const struct rte_dma_conf *dev_conf);
+
+The ``rte_dma_conf`` structure is used to pass the configuration parameters
+for the DMA device for example the number of virtual DMA channels to set up,
+indication of whether to enable silent mode.
+
+
+Configuration of Virtual DMA Channels
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_vchan_setup API is used to configure a virtual DMA channel.
+
+.. code-block:: c
+
+   int rte_dma_vchan_setup(int16_t dev_id, uint16_t vchan,
+                           const struct rte_dma_vchan_conf *conf);
+
+The ``rte_dma_vchan_conf`` structure is used to pass the configuration
+parameters for the virtual DMA channel for example transfer direction, number of
+descriptor for the virtual DMA channel, source device access port parameter,
+destination device access port parameter.
+
+
+Device Features and Capabilities
+--------------------------------
+
+DMA devices may support different feature sets. The ``rte_dma_info_get`` API
+can be used to get the device info and supported features.
+
+Silent mode is a special device capability which does not require the
+application to invoke dequeue APIs.
doc/guides/rel_notes/release_21_11.rst +1 -0
index 74639f1e81..0aceaa8837 100644
@@ -109,6 +109,7 @@ New Features
 * **Introduced dmadev library with:**
 
   * Device allocation APIs.
+  * Control plane APIs.
 
 
 Removed Items
lib/dmadev/rte_dmadev.c +359 -0

368 lines not on this page · raw message

lib/dmadev/rte_dmadev.h +480 -0

492 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h +61 -1

83 lines not on this page · raw message

lib/dmadev/version.map +9 -0
index 56ea0332cb..6b7939b10f 100644
@@ -1,10 +1,19 @@
 EXPERIMENTAL {
 	global:
 
+	rte_dma_close;
+	rte_dma_configure;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
+	rte_dma_dump;
 	rte_dma_get_dev_id;
+	rte_dma_info_get;
 	rte_dma_is_valid;
+	rte_dma_start;
+	rte_dma_stats_get;
+	rte_dma_stats_reset;
+	rte_dma_stop;
+	rte_dma_vchan_setup;
 
 	local: *;
 };
Signature · 1 line
2.33.0
#296 lore raw

Re: [dpdk-dev] [PATCH v23 2/6] dmadev: add control plane function support

Hi Chengwen

API looks good to me, thanks!

I have some questions below.

Quoted 2 lines

This patch add control plane functions for dmadev.

<snip>

Quoted 15 lines
lib/dmadev/rte_dmadev.h +14 -0
+/**
+ * DMA transfer direction defines.
+ *
+ * @see struct rte_dma_vchan_conf::direction  */ enum rte_dma_direction
+{
+       /** DMA transfer direction - from memory to memory.
+        *
+        * @see struct rte_dma_vchan_conf::direction
+        */
+       RTE_DMA_DIR_MEM_TO_MEM,
+       /** DMA transfer direction - from memory to device.
+        * In a typical scenario, the SoCs are installed on host servers as
+        * iNICs through the PCIe interface. In this case, the SoCs works in
+        * EP(endpoint) mode, it could initiate a DMA move request from

memory

lib/dmadev/rte_dmadev.h bool rte_dma_is_valid(int16_t dev_id); · line 364

Quoted 5 lines
lib/dmadev/rte_dmadev.h +5 -0
+        * (which is SoCs memory) to device (which is host memory).
+        *
+        * @see struct rte_dma_vchan_conf::direction
+        */
+       RTE_DMA_DIR_MEM_TO_DEV,

I don't understand precisely the meaning of mem and dev.

What does it mean SoCs memory?

What does it mean host memory?

What is the memory HW in these two types?

How does the user get the addresses of SoCs memory?

How does the user get the addresses of host memory?

Can dpdk app here access physical memory not mapped\allocated to the app?

Matan

Quoted 137 lines
lib/dmadev/rte_dmadev.h +14 -0
+       /** DMA transfer direction - from device to memory.
+        * In a typical scenario, the SoCs are installed on host servers as
+        * iNICs through the PCIe interface. In this case, the SoCs works in
+        * EP(endpoint) mode, it could initiate a DMA move request from device
+        * (which is host memory) to memory (which is SoCs memory).
+        *
+        * @see struct rte_dma_vchan_conf::direction
+        */
+       RTE_DMA_DIR_DEV_TO_MEM,
+       /** DMA transfer direction - from device to device.
+        * In a typical scenario, the SoCs are installed on host servers as
+        * iNICs through the PCIe interface. In this case, the SoCs works in
+        * EP(endpoint) mode, it could initiate a DMA move request from device
+        * (which is host memory) to the device (which is another host

memory).

lib/dmadev/rte_dmadev.h +10 -0
+        *
+        * @see struct rte_dma_vchan_conf::direction
+        */
+       RTE_DMA_DIR_DEV_TO_DEV,
+};
+
+/**
+ * DMA access port type defines.
+ *
+ * @see struct rte_dma_port_param::port_type  */ enum

rte_dma_port_type

lib/dmadev/rte_dmadev.h +111 -0
+{
+       RTE_DMA_PORT_NONE,
+       RTE_DMA_PORT_PCIE, /**< The DMA access port is PCIe. */ };
+
+/**
+ * A structure used to descript DMA access port parameters.
+ *
+ * @see struct rte_dma_vchan_conf::src_port
+ * @see struct rte_dma_vchan_conf::dst_port  */ struct
+rte_dma_port_param {
+       /** The device access port type.
+        *
+        * @see enum rte_dma_port_type
+        */
+       enum rte_dma_port_type port_type;
+       union {
+               /** PCIe access port parameters.
+                *
+                * The following model shows SoC's PCIe module connects to
+                * multiple PCIe hosts and multiple endpoints. The PCIe module
+                * has an integrated DMA controller.
+                *
+                * If the DMA wants to access the memory of host A, it can be
+                * initiated by PF1 in core0, or by VF0 of PF0 in core0.
+                *
+                * \code{.unparsed}
+                * System Bus
+                *    |     ----------PCIe module----------
+                *    |     Bus
+                *    |     Interface
+                *    |     -----        ------------------
+                *    |     |   |        | PCIe Core0     |
+                *    |     |   |        |                |        -----------
+                *    |     |   |        |   PF-0 -- VF-0 |        | Host A  |
+                *    |     |   |--------|        |- VF-1 |--------| Root    |
+                *    |     |   |        |   PF-1         |        | Complex |
+                *    |     |   |        |   PF-2         |        -----------
+                *    |     |   |        ------------------
+                *    |     |   |
+                *    |     |   |        ------------------
+                *    |     |   |        | PCIe Core1     |
+                *    |     |   |        |                |        -----------
+                *    |     |   |        |   PF-0 -- VF-0 |        | Host B  |
+                *    |-----|   |--------|   PF-1 -- VF-0 |--------| Root    |
+                *    |     |   |        |        |- VF-1 |        | Complex |
+                *    |     |   |        |   PF-2         |        -----------
+                *    |     |   |        ------------------
+                *    |     |   |
+                *    |     |   |        ------------------
+                *    |     |DMA|        |                |        ------
+                *    |     |   |        |                |--------| EP |
+                *    |     |   |--------| PCIe Core2     |        ------
+                *    |     |   |        |                |        ------
+                *    |     |   |        |                |--------| EP |
+                *    |     |   |        |                |        ------
+                *    |     -----        ------------------
+                *
+                * \endcode
+                *
+                * @note If some fields can not be supported by the
+                * hardware/driver, then the driver ignores those fields.
+                * Please check driver-specific documentation for limitations
+                * and capablites.
+                */
+               struct {
+                       uint64_t coreid : 4; /**< PCIe core id used. */
+                       uint64_t pfid : 8; /**< PF id used. */
+                       uint64_t vfen : 1; /**< VF enable bit. */
+                       uint64_t vfid : 16; /**< VF id used. */
+                       /** The pasid filed in TLP packet. */
+                       uint64_t pasid : 20;
+                       /** The attributes filed in TLP packet. */
+                       uint64_t attr : 3;
+                       /** The processing hint filed in TLP packet. */
+                       uint64_t ph : 2;
+                       /** The steering tag filed in TLP packet. */
+                       uint64_t st : 16;
+               } pcie;
+       };
+       uint64_t reserved[2]; /**< Reserved for future fields. */ };
+
+/**
+ * A structure used to configure a virtual DMA channel.
+ *
+ * @see rte_dma_vchan_setup
+ */
+struct rte_dma_vchan_conf {
+       /** Transfer direction
+        *
+        * @see enum rte_dma_direction
+        */
+       enum rte_dma_direction direction;
+       /** Number of descriptor for the virtual DMA channel */
+       uint16_t nb_desc;
+       /** 1) Used to describes the device access port parameter in the
+        * device-to-memory transfer scenario.
+        * 2) Used to describes the source device access port parameter in the
+        * device-to-device transfer scenario.
+        *
+        * @see struct rte_dma_port_param
+        */
+       struct rte_dma_port_param src_port;
+       /** 1) Used to describes the device access port parameter in the
+        * memory-to-device transfer scenario.
+        * 2) Used to describes the destination device access port parameter in
+        * the device-to-device transfer scenario.
+        *
+        * @see struct rte_dma_port_param
+        */
+       struct rte_dma_port_param dst_port; };
+

<snip>

#297 lore raw

Re: [dpdk-dev] [PATCH v23 2/6] dmadev: add control plane function support

Quoted 35 lines · Matan Azrad

Hi Chengwen

API looks good to me, thanks!

I have some questions below.

Quoted 2 lines

This patch add control plane functions for dmadev.

<snip>

Quoted 15 lines
lib/dmadev/rte_dmadev.h +14 -0
+/**
+ * DMA transfer direction defines.
+ *
+ * @see struct rte_dma_vchan_conf::direction  */ enum rte_dma_direction
+{
+       /** DMA transfer direction - from memory to memory.
+        *
+        * @see struct rte_dma_vchan_conf::direction
+        */
+       RTE_DMA_DIR_MEM_TO_MEM,
+       /** DMA transfer direction - from memory to device.
+        * In a typical scenario, the SoCs are installed on host servers as
+        * iNICs through the PCIe interface. In this case, the SoCs works in
+        * EP(endpoint) mode, it could initiate a DMA move request from

memory

lib/dmadev/rte_dmadev.h bool rte_dma_is_valid(int16_t dev_id); · line 364

Quoted 5 lines
lib/dmadev/rte_dmadev.h +5 -0
+        * (which is SoCs memory) to device (which is host memory).
+        *
+        * @see struct rte_dma_vchan_conf::direction
+        */
+       RTE_DMA_DIR_MEM_TO_DEV,

I don't understand precisely the meaning of mem and dev.

What does it mean SoCs memory?

Quoted 8 lines

What does it mean host memory?

What is the memory HW in these two types?

How does the user get the addresses of SoCs memory?

How does the user get the addresses of host memory?

Hi Matan,

	System Bus
	    |     ----------PCIe module----------
	    |     Bus
	    |     Interface
	    |     -----        ------------------
	    |     |   |        | PCIe Core0     |
 DDR3 ------|     |   |        |                |        -----------
	    |     |   |        |   PF-0 -- VF-0 |        | Host A  |
	    |     |   |--------|        |- VF-1 |--------| Root    | ---DDR1
	    |     |   |        |   PF-1         |        | Complex |
	    |     |   |        |   PF-2         |        -----------
	    |     |   |        ------------------
	    |     |DMA|
	    |     |   |        ------------------
	    |     |   |        | PCIe Core1     |
	    |     |   |        |                |        -----------
	    |     |   |        |   PF-0 -- VF-0 |        | Host B  |
	    |-----|   |--------|   PF-1 -- VF-0 |--------| Root    | ---DDR2
	    |     |   |        |        |- VF-1 |        | Complex |
	    |     |   |        |   PF-2         |        -----------
	    |     |   |        ------------------
	    |     -----

---------------------SOC------------------------- -------HOST--------

As shown in the above figure, the SOC is connected to two hosts.
The DDR3 is the SOC's memory, the DDR1 is HOST-A's memory, and DDR2 is HOST-B's memory.

To access DDR3 memory, APP could mmap and pass its IOVA address to DMA.
To access DDR1/2 memory, some devices use parameterized descriptor which contain the function from which the request is sent and the destination address. In this case, it is up to the user to confirm the function and destination address which could passed in as parameters.

Thanks.

Quoted 146 lines

Can dpdk app here access physical memory not mapped\allocated to the app?

Matan

Quoted 137 lines
lib/dmadev/rte_dmadev.h +14 -0
+       /** DMA transfer direction - from device to memory.
+        * In a typical scenario, the SoCs are installed on host servers as
+        * iNICs through the PCIe interface. In this case, the SoCs works in
+        * EP(endpoint) mode, it could initiate a DMA move request from device
+        * (which is host memory) to memory (which is SoCs memory).
+        *
+        * @see struct rte_dma_vchan_conf::direction
+        */
+       RTE_DMA_DIR_DEV_TO_MEM,
+       /** DMA transfer direction - from device to device.
+        * In a typical scenario, the SoCs are installed on host servers as
+        * iNICs through the PCIe interface. In this case, the SoCs works in
+        * EP(endpoint) mode, it could initiate a DMA move request from device
+        * (which is host memory) to the device (which is another host

memory).

lib/dmadev/rte_dmadev.h +10 -0
+        *
+        * @see struct rte_dma_vchan_conf::direction
+        */
+       RTE_DMA_DIR_DEV_TO_DEV,
+};
+
+/**
+ * DMA access port type defines.
+ *
+ * @see struct rte_dma_port_param::port_type  */ enum

rte_dma_port_type

lib/dmadev/rte_dmadev.h +111 -0
+{
+       RTE_DMA_PORT_NONE,
+       RTE_DMA_PORT_PCIE, /**< The DMA access port is PCIe. */ };
+
+/**
+ * A structure used to descript DMA access port parameters.
+ *
+ * @see struct rte_dma_vchan_conf::src_port
+ * @see struct rte_dma_vchan_conf::dst_port  */ struct
+rte_dma_port_param {
+       /** The device access port type.
+        *
+        * @see enum rte_dma_port_type
+        */
+       enum rte_dma_port_type port_type;
+       union {
+               /** PCIe access port parameters.
+                *
+                * The following model shows SoC's PCIe module connects to
+                * multiple PCIe hosts and multiple endpoints. The PCIe module
+                * has an integrated DMA controller.
+                *
+                * If the DMA wants to access the memory of host A, it can be
+                * initiated by PF1 in core0, or by VF0 of PF0 in core0.
+                *
+                * \code{.unparsed}
+                * System Bus
+                *    |     ----------PCIe module----------
+                *    |     Bus
+                *    |     Interface
+                *    |     -----        ------------------
+                *    |     |   |        | PCIe Core0     |
+                *    |     |   |        |                |        -----------
+                *    |     |   |        |   PF-0 -- VF-0 |        | Host A  |
+                *    |     |   |--------|        |- VF-1 |--------| Root    |
+                *    |     |   |        |   PF-1         |        | Complex |
+                *    |     |   |        |   PF-2         |        -----------
+                *    |     |   |        ------------------
+                *    |     |   |
+                *    |     |   |        ------------------
+                *    |     |   |        | PCIe Core1     |
+                *    |     |   |        |                |        -----------
+                *    |     |   |        |   PF-0 -- VF-0 |        | Host B  |
+                *    |-----|   |--------|   PF-1 -- VF-0 |--------| Root    |
+                *    |     |   |        |        |- VF-1 |        | Complex |
+                *    |     |   |        |   PF-2         |        -----------
+                *    |     |   |        ------------------
+                *    |     |   |
+                *    |     |   |        ------------------
+                *    |     |DMA|        |                |        ------
+                *    |     |   |        |                |--------| EP |
+                *    |     |   |--------| PCIe Core2     |        ------
+                *    |     |   |        |                |        ------
+                *    |     |   |        |                |--------| EP |
+                *    |     |   |        |                |        ------
+                *    |     -----        ------------------
+                *
+                * \endcode
+                *
+                * @note If some fields can not be supported by the
+                * hardware/driver, then the driver ignores those fields.
+                * Please check driver-specific documentation for limitations
+                * and capablites.
+                */
+               struct {
+                       uint64_t coreid : 4; /**< PCIe core id used. */
+                       uint64_t pfid : 8; /**< PF id used. */
+                       uint64_t vfen : 1; /**< VF enable bit. */
+                       uint64_t vfid : 16; /**< VF id used. */
+                       /** The pasid filed in TLP packet. */
+                       uint64_t pasid : 20;
+                       /** The attributes filed in TLP packet. */
+                       uint64_t attr : 3;
+                       /** The processing hint filed in TLP packet. */
+                       uint64_t ph : 2;
+                       /** The steering tag filed in TLP packet. */
+                       uint64_t st : 16;
+               } pcie;
+       };
+       uint64_t reserved[2]; /**< Reserved for future fields. */ };
+
+/**
+ * A structure used to configure a virtual DMA channel.
+ *
+ * @see rte_dma_vchan_setup
+ */
+struct rte_dma_vchan_conf {
+       /** Transfer direction
+        *
+        * @see enum rte_dma_direction
+        */
+       enum rte_dma_direction direction;
+       /** Number of descriptor for the virtual DMA channel */
+       uint16_t nb_desc;
+       /** 1) Used to describes the device access port parameter in the
+        * device-to-memory transfer scenario.
+        * 2) Used to describes the source device access port parameter in the
+        * device-to-device transfer scenario.
+        *
+        * @see struct rte_dma_port_param
+        */
+       struct rte_dma_port_param src_port;
+       /** 1) Used to describes the device access port parameter in the
+        * memory-to-device transfer scenario.
+        * 2) Used to describes the destination device access port parameter in
+        * the device-to-device transfer scenario.
+        *
+        * @see struct rte_dma_port_param
+        */
+       struct rte_dma_port_param dst_port; };
+

<snip>

#298 lore raw

Re: [dpdk-dev] [PATCH v23 2/6] dmadev: add control plane function support

In reply to Chengwen Feng

24/09/2021 12:53, Chengwen Feng:

Quoted 27 lines
doc/guides/prog_guide/dmadev.rst +21 -0
@@ -62,3 +62,44 @@ identifiers:
 
 - A device name used to designate the DMA device in console messages, for
   administration or debugging purposes.
+
+
+Device Configuration
+~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_configure API is used to configure a DMA device.
+
+.. code-block:: c
+
+   int rte_dma_configure(int16_t dev_id,
+                         const struct rte_dma_conf *dev_conf);
+
+The ``rte_dma_conf`` structure is used to pass the configuration parameters
+for the DMA device for example the number of virtual DMA channels to set up,
+indication of whether to enable silent mode.
+
+
+Configuration of Virtual DMA Channels
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_vchan_setup API is used to configure a virtual DMA channel.

doc/guides/prog_guide/dmadev.rst identifiers: · line 95

Quoted 10 lines
doc/guides/prog_guide/dmadev.rst +10 -0
+
+.. code-block:: c
+
+   int rte_dma_vchan_setup(int16_t dev_id, uint16_t vchan,
+                           const struct rte_dma_vchan_conf *conf);
+
+The ``rte_dma_vchan_conf`` structure is used to pass the configuration
+parameters for the virtual DMA channel for example transfer direction, number of
+descriptor for the virtual DMA channel, source device access port parameter,
+destination device access port parameter.

You should avoid being redundant with the Doxygen documentation. In the guide, it should be only explaining the concepts, not the details. For the details of each function, we refer to Doxygen.

lib/dmadev/rte_dmadev.c rte_dma_pmd_release(const char *name) · line 222

Quoted 8 lines
lib/dmadev/rte_dmadev.c +2 -0
@@ -218,6 +218,9 @@ rte_dma_pmd_release(const char *name)
 	if (dev == NULL)
 		return -EINVAL;
 
+	if (dev->state == RTE_DMA_DEV_READY)
+		return rte_dma_close(dev->dev_id);

What is the logic here?
The only exposed function should be rte_dma_close()
and it should call the freeing function.
The API should use the dev_id. As you said somewhere else, the name is only for debugging.
Please remove the function rte_dma_pmd_release(const char *name).

[...]

Quoted 6 lines
lib/dmadev/rte_dmadev.h +4 -0
+ * The functions exported by the dmadev API to setup a device designated by its
+ * device identifier must be invoked in the following order:
+ *     - rte_dma_configure()
+ *     - rte_dma_vchan_setup()

lib/dmadev/rte_dmadev.h line 69

Quoted 10 lines
lib/dmadev/rte_dmadev.h +10 -0
+ *     - rte_dma_start()
+ *
+ * If the application wants to change the configuration (i.e. invoke
+ * rte_dma_configure() or rte_dma_vchan_setup()), it must invoke
+ * rte_dma_stop() first to stop the device and then do the reconfiguration
+ * before invoking rte_dma_start() again. The dataplane functions should not
+ * be invoked when the device is stopped.
+ *
+ * Finally, an application can close a dmadev by invoking the rte_dma_close()
+ * function.

Yes rte_dma_close, not rte_dma_pmd_release.

Quoted 2 lines

+ *
+ * About MT-safe, all the functions of the dmadev API exported by a PMD are

API is not exported by a PMD, but implemented.

Quoted 7 lines

+ * lock-free functions which assume to not be invoked in parallel on different
+ * logical cores to work on the same target dmadev object.
+ * @note Different virtual DMA channels on the same dmadev *DO NOT* support
+ * parallel invocation because these virtual DMA channels share the same
+ * HW-DMA-channel.
+ *
*/

No need of final blank line in a comment.

lib/dmadev/rte_dmadev.h bool rte_dma_is_valid(int16_t dev_id); · line 159

Quoted 10 lines
lib/dmadev/rte_dmadev.h +10 -0
+/** DMA device support memory-to-memory transfer.
+ *
+ * @see struct rte_dma_info::dev_capa
+ */
+#define RTE_DMA_CAPA_MEM_TO_MEM		RTE_BIT64(0)
+/** DMA device support memory-to-device transfer.
+ *
+ * @see struct rte_dma_info::dev_capa
+ */
+#define RTE_DMA_CAPA_MEM_TO_DEV		RTE_BIT64(1)

Same comment as in earlier version: please group the flags in a doxygen group. Example of doxygen group:
https://patches.dpdk.org/project/dpdk/patch/[email protected]/

[...]
You are using uint64_t bitfields and anonymous union in below struct, it may not compile if not using __extension__ from RTE_STD_C11.

Quoted 7 lines
lib/dmadev/rte_dmadev.h +7 -0
+struct rte_dma_port_param {
+	/** The device access port type.
+	 *
+	 * @see enum rte_dma_port_type
+	 */
+	enum rte_dma_port_type port_type;
+	union {

[...]

Quoted 20 lines
lib/dmadev/rte_dmadev.h +17 -0
+		struct {
+			uint64_t coreid : 4; /**< PCIe core id used. */
+			uint64_t pfid : 8; /**< PF id used. */
+			uint64_t vfen : 1; /**< VF enable bit. */
+			uint64_t vfid : 16; /**< VF id used. */
+			/** The pasid filed in TLP packet. */
+			uint64_t pasid : 20;
+			/** The attributes filed in TLP packet. */
+			uint64_t attr : 3;
+			/** The processing hint filed in TLP packet. */
+			uint64_t ph : 2;
+			/** The steering tag filed in TLP packet. */
+			uint64_t st : 16;
+		} pcie;
+	};
+	uint64_t reserved[2]; /**< Reserved for future fields. */
+};

lib/dmadev/rte_dmadev_core.h line 26

Quoted 4 lines
lib/dmadev/rte_dmadev_core.h +4 -0
+/** @internal Used to get device information of a device. */
+typedef int (*rte_dma_info_get_t)(const struct rte_dma_dev *dev,
+				  struct rte_dma_info *dev_info,
+				  uint32_t info_sz);

Please move all driver interfaces in a file dedicated to drivers.

[...]

lib/dmadev/rte_dmadev_core.h struct rte_dma_dev { · line 99

Quoted 5 lines
lib/dmadev/rte_dmadev_core.h +1 -0
@@ -40,9 +96,13 @@ struct rte_dma_dev {
 	int16_t dev_id; /**< Device [external] identifier. */
 	int16_t numa_node; /**< Local NUMA memory ID. -1 if unknown. */
 	void *dev_private; /**< PMD-specific private data. */
+	/** Functions exported by PMD. */

s/exported/implemented/

Quoted 8 lines
lib/dmadev/rte_dmadev_core.h +3 -0
+	const struct rte_dma_dev_ops *dev_ops;
+	struct rte_dma_conf dev_conf; /**< DMA device configuration. */
 	/** Device info which supplied during device initialization. */
 	struct rte_device *device;
 	enum rte_dma_dev_state state; /**< Flag indicating the device state. */
+	uint8_t dev_started : 1; /**< Device state: STARTED(1)/STOPPED(0). */
 	uint64_t reserved[2]; /**< Reserved for future fields. */
 } __rte_cache_aligned;
#299 lore raw

Re: [dpdk-dev] [PATCH v23 2/6] dmadev: add control plane function support

Quoted 7 lines · Thomas Monjalon

24/09/2021 12:53, Chengwen Feng:

Quoted 6 lines
doc/guides/prog_guide/dmadev.rst +0 -0
@@ -62,3 +62,44 @@ identifiers:
 
 - A device name used to designate the DMA device in console messages, for
   administration or debugging purposes.

[snip]

Quoted 7 lines
Quoted 7 lines
lib/dmadev/rte_dmadev.c +0 -0
@@ -218,6 +218,9 @@ rte_dma_pmd_release(const char *name)
 	if (dev == NULL)
 		return -EINVAL;
 
Quoted 9 lines

lib/dmadev/rte_dmadev.c rte_dma_pmd_release(const char *name) · line 222

Quoted 2 lines
lib/dmadev/rte_dmadev.c +2 -0
+	if (dev->state == RTE_DMA_DEV_READY)
+		return rte_dma_close(dev->dev_id);

What is the logic here?
The only exposed function should be rte_dma_close()
and it should call the freeing function.
The API should use the dev_id. As you said somewhere else, the name is only for debugging.
Please remove the function rte_dma_pmd_release(const char *name).

The rte_dma_pmd_release corresponding to pmd_allocate, so both use the 'const char *name' parameter.

The rte_dma_pmd_release is also used for error handling when PMD init. Therefore, a status variable is used here. If the device is not ready, only resources need to be released. Otherwise, the close interface of the driver is invoked.

For PMD, the rte_dma_pmd_release is only wrap for dev_close when remove device, it does not need to implement two callbacks.

If we replace rte_dma_pmd_release with rte_dma_close, then we should invoke rte_dma_close in error handling when PMD init, this can lead to conceptual inconsistencies because the initialization has not been successful.

So I think it's better keep rte_dma_pmd_release.

Quoted 36 lines

[...]

Quoted 6 lines
lib/dmadev/rte_dmadev.h +4 -0
+ * The functions exported by the dmadev API to setup a device designated by its
+ * device identifier must be invoked in the following order:
+ *     - rte_dma_configure()
+ *     - rte_dma_vchan_setup()

lib/dmadev/rte_dmadev.h line 69

Quoted 10 lines
lib/dmadev/rte_dmadev.h +10 -0
+ *     - rte_dma_start()
+ *
+ * If the application wants to change the configuration (i.e. invoke
+ * rte_dma_configure() or rte_dma_vchan_setup()), it must invoke
+ * rte_dma_stop() first to stop the device and then do the reconfiguration
+ * before invoking rte_dma_start() again. The dataplane functions should not
+ * be invoked when the device is stopped.
+ *
+ * Finally, an application can close a dmadev by invoking the rte_dma_close()
+ * function.

Yes rte_dma_close, not rte_dma_pmd_release.

Quoted 2 lines

+ *
+ * About MT-safe, all the functions of the dmadev API exported by a PMD are

API is not exported by a PMD, but implemented.

Quoted 7 lines

+ * lock-free functions which assume to not be invoked in parallel on different
+ * logical cores to work on the same target dmadev object.
+ * @note Different virtual DMA channels on the same dmadev *DO NOT* support
+ * parallel invocation because these virtual DMA channels share the same
+ * HW-DMA-channel.
+ *
*/

No need of final blank line in a comment.

Quoted 14 lines

lib/dmadev/rte_dmadev.h bool rte_dma_is_valid(int16_t dev_id); · line 159

Quoted 10 lines
lib/dmadev/rte_dmadev.h +10 -0
+/** DMA device support memory-to-memory transfer.
+ *
+ * @see struct rte_dma_info::dev_capa
+ */
+#define RTE_DMA_CAPA_MEM_TO_MEM		RTE_BIT64(0)
+/** DMA device support memory-to-device transfer.
+ *
+ * @see struct rte_dma_info::dev_capa
+ */
+#define RTE_DMA_CAPA_MEM_TO_DEV		RTE_BIT64(1)

Same comment as in earlier version: please group the flags in a doxygen group. Example of doxygen group:
https://patches.dpdk.org/project/dpdk/patch/[email protected]/

Tried, but found it didn't coexist well with multi-line comments.

Quoted 31 lines

[...]
You are using uint64_t bitfields and anonymous union in below struct, it may not compile if not using __extension__ from RTE_STD_C11.

Quoted 7 lines
lib/dmadev/rte_dmadev.h +7 -0
+struct rte_dma_port_param {
+	/** The device access port type.
+	 *
+	 * @see enum rte_dma_port_type
+	 */
+	enum rte_dma_port_type port_type;
+	union {

[...]

Quoted 17 lines
lib/dmadev/rte_dmadev.h +17 -0
+		struct {
+			uint64_t coreid : 4; /**< PCIe core id used. */
+			uint64_t pfid : 8; /**< PF id used. */
+			uint64_t vfen : 1; /**< VF enable bit. */
+			uint64_t vfid : 16; /**< VF id used. */
+			/** The pasid filed in TLP packet. */
+			uint64_t pasid : 20;
+			/** The attributes filed in TLP packet. */
+			uint64_t attr : 3;
+			/** The processing hint filed in TLP packet. */
+			uint64_t ph : 2;
+			/** The steering tag filed in TLP packet. */
+			uint64_t st : 16;
+		} pcie;
+	};
+	uint64_t reserved[2]; /**< Reserved for future fields. */
+};
Quoted 8 lines

lib/dmadev/rte_dmadev_core.h line 26

Quoted 6 lines
lib/dmadev/rte_dmadev_core.h +4 -0
+/** @internal Used to get device information of a device. */
+typedef int (*rte_dma_info_get_t)(const struct rte_dma_dev *dev,
+				  struct rte_dma_info *dev_info,
+				  uint32_t info_sz);

Please move all driver interfaces in a file dedicated to drivers.

There are three head file: rte_dmadev.h, rte_dmadev_core.h, rte_dmadev_pmd.h And we build the following dependency:

                rte_dmadev.h   ---> rte_dmadev_core.h          // mainly because dataplane inline API.
                    ^
                    |
           ---------------------
           |                   |
       Application       rte_dmadev_pmd.h
                               ^
                               |
                             DMA PMD

If move all driver interfaces to rte_dmadev_pmd.h from rte_dmadev_core.h, bidirectional dependency may exist, e.g.

                rte_dmadev.h   ---> rte_dmadev_core.h  ---> rte_dmadev_pmd.h
                    ^
                    |
           ---------------------
           |                   |
       Application       rte_dmadev_pmd.h
                               ^
                               |
                             DMA PMD

So I think it's better keep it that way.

Quoted 10 lines

[...]

lib/dmadev/rte_dmadev_core.h struct rte_dma_dev { · line 99

Quoted 5 lines
lib/dmadev/rte_dmadev_core.h +1 -0
@@ -40,9 +96,13 @@ struct rte_dma_dev {
 	int16_t dev_id; /**< Device [external] identifier. */
 	int16_t numa_node; /**< Local NUMA memory ID. -1 if unknown. */
 	void *dev_private; /**< PMD-specific private data. */
+	/** Functions exported by PMD. */

s/exported/implemented/

Quoted 14 lines

lib/dmadev/rte_dmadev_core.h struct rte_dma_dev { · line 107

Quoted 8 lines
lib/dmadev/rte_dmadev_core.h +3 -0
+	const struct rte_dma_dev_ops *dev_ops;
+	struct rte_dma_conf dev_conf; /**< DMA device configuration. */
 	/** Device info which supplied during device initialization. */
 	struct rte_device *device;
 	enum rte_dma_dev_state state; /**< Flag indicating the device state. */
+	uint8_t dev_started : 1; /**< Device state: STARTED(1)/STOPPED(0). */
 	uint64_t reserved[2]; /**< Reserved for future fields. */
 } __rte_cache_aligned;

.

Thanks

#300 lore raw

Re: [dpdk-dev] [PATCH v23 2/6] dmadev: add control plane function support

08/10/2021 09:55, fengchengwen:

Quoted 24 lines · Thomas Monjalon
Quoted 7 lines

24/09/2021 12:53, Chengwen Feng:

Quoted 6 lines
lib/dmadev/rte_dmadev.c +0 -0
@@ -218,6 +218,9 @@ rte_dma_pmd_release(const char *name)
 	if (dev == NULL)
 		return -EINVAL;
 
Quoted 9 lines

lib/dmadev/rte_dmadev.c rte_dma_pmd_release(const char *name) · line 222

Quoted 2 lines
lib/dmadev/rte_dmadev.c +2 -0
+	if (dev->state == RTE_DMA_DEV_READY)
+		return rte_dma_close(dev->dev_id);

What is the logic here?
The only exposed function should be rte_dma_close()
and it should call the freeing function.
The API should use the dev_id. As you said somewhere else, the name is only for debugging.
Please remove the function rte_dma_pmd_release(const char *name).

The rte_dma_pmd_release corresponding to pmd_allocate, so both use the 'const char *name' parameter.

The rte_dma_pmd_release is also used for error handling when PMD init. Therefore, a status variable is used here. If the device is not ready, only resources need to be released. Otherwise, the close interface of the driver is invoked.

Quoted 8 lines

For PMD, the rte_dma_pmd_release is only wrap for dev_close when remove device, it does not need to implement two callbacks.

If we replace rte_dma_pmd_release with rte_dma_close, then we should invoke rte_dma_close in error handling when PMD init, this can lead to conceptual inconsistencies because the initialization has not been successful.

So I think it's better keep rte_dma_pmd_release.

I will review again this logic in the next version.

Quoted 15 lines
Quoted 6 lines
Quoted 6 lines
lib/dmadev/rte_dmadev.c +6 -0
+/** DMA device support memory-to-memory transfer.
+ *
+ * @see struct rte_dma_info::dev_capa
+ */
+#define RTE_DMA_CAPA_MEM_TO_MEM		RTE_BIT64(0)
+/** DMA device support memory-to-device transfer.
Quoted 8 lines
Quoted 4 lines

+ *
+ * @see struct rte_dma_info::dev_capa
+ */
+#define RTE_DMA_CAPA_MEM_TO_DEV RTE_BIT64(1)

Same comment as in earlier version: please group the flags in a doxygen group. Example of doxygen group:
https://patches.dpdk.org/project/dpdk/patch/[email protected]/

Quoted 1 line

Tried, but found it didn't coexist well with multi-line comments.

What is not working? Example?
I think you didn't get what to do.
You must add a comment to give a title and group all these flags.

Quoted 36 lines
Quoted 8 lines

lib/dmadev/rte_dmadev_core.h line 26

Quoted 6 lines
lib/dmadev/rte_dmadev_core.h +4 -0
+/** @internal Used to get device information of a device. */
+typedef int (*rte_dma_info_get_t)(const struct rte_dma_dev *dev,
+				  struct rte_dma_info *dev_info,
+				  uint32_t info_sz);

Please move all driver interfaces in a file dedicated to drivers.

There are three head file: rte_dmadev.h, rte_dmadev_core.h, rte_dmadev_pmd.h And we build the following dependency:

                rte_dmadev.h   ---> rte_dmadev_core.h          // mainly because dataplane inline API.
                    ^
                    |
           ---------------------
           |                   |
       Application       rte_dmadev_pmd.h
                               ^
                               |
                             DMA PMD

If move all driver interfaces to rte_dmadev_pmd.h from rte_dmadev_core.h, bidirectional dependency may exist, e.g.

                rte_dmadev.h   ---> rte_dmadev_core.h  ---> rte_dmadev_pmd.h
                    ^
                    |
           ---------------------
           |                   |
       Application       rte_dmadev_pmd.h
                               ^
                               |
                             DMA PMD
Quoted 1 line

So I think it's better keep it that way.

Please make sure only what is needed for inline is kept in the "core.h" You should look at the current effort done by Konstanti in ethdev to hide everything.

#301 lore raw

PATCH v23 3/6 dmadev: add data plane function support

In reply to Chengwen Feng

This patch add data plane functions for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  22 ++
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.h                | 460 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  51 ++-
 lib/dmadev/version.map                 |   6 +
 5 files changed, 537 insertions(+), 3 deletions(-)
doc/guides/prog_guide/dmadev.rst +22 -0
index c2b0b0420b..de8b599d96 100644
@@ -103,3 +103,25 @@ can be used to get the device info and supported features.
 
 Silent mode is a special device capability which does not require the
 application to invoke dequeue APIs.
+
+
+Enqueue / Dequeue APIs
+~~~~~~~~~~~~~~~~~~~~~~
+
+Enqueue APIs such as ``rte_dma_copy`` and ``rte_dma_fill`` can be used to
+enqueue operations to hardware. If an enqueue is successful, a ``ring_idx`` is
+returned. This ``ring_idx`` can be used by applications to track per operation
+metadata in an application-defined circular ring.
+
+The ``rte_dma_submit`` API is used to issue doorbell to hardware.
+Alternatively the ``RTE_DMA_OP_FLAG_SUBMIT`` flag can be passed to the enqueue
+APIs to also issue the doorbell to hardware.
+
+There are two dequeue APIs ``rte_dma_completed`` and
+``rte_dma_completed_status``, these are used to obtain the results of the
+enqueue requests. ``rte_dma_completed`` will return the number of successfully
+completed operations. ``rte_dma_completed_status`` will return the number of
+completed operations along with the status of each operation (filled into the
+``status`` array passed by user). These two APIs can also return the last
+completed operation's ``ring_idx`` which could help user track operations within
+their own application-defined rings.
doc/guides/rel_notes/release_21_11.rst +1 -0
index 0aceaa8837..21b3c48257 100644
@@ -110,6 +110,7 @@ New Features
 
   * Device allocation APIs.
   * Control plane APIs.
+  * Data plane APIs.
 
 
 Removed Items
lib/dmadev/rte_dmadev.h +460 -0

480 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h +48 -3

68 lines not on this page · raw message

lib/dmadev/version.map +6 -0
index 6b7939b10f..c780463bb2 100644
@@ -2,10 +2,15 @@ EXPERIMENTAL {
 	global:
 
 	rte_dma_close;
+	rte_dma_completed;
+	rte_dma_completed_status;
 	rte_dma_configure;
+	rte_dma_copy;
+	rte_dma_copy_sg;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
 	rte_dma_dump;
+	rte_dma_fill;
 	rte_dma_get_dev_id;
 	rte_dma_info_get;
 	rte_dma_is_valid;
@@ -13,6 +18,7 @@ EXPERIMENTAL {
 	rte_dma_stats_get;
 	rte_dma_stats_reset;
 	rte_dma_stop;
+	rte_dma_submit;
 	rte_dma_vchan_setup;
 
 	local: *;
Signature · 1 line
2.33.0
#302 lore raw

PATCH v23 4/6 dmadev: add multi-process support

In reply to Chengwen Feng

This patch add multi-process support for dmadev.

---
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.c                | 168 ++++++++++++++++++++-----
 lib/dmadev/rte_dmadev.h                |  24 ++--
 lib/dmadev/rte_dmadev_core.h           |  45 +++++--
 4 files changed, 185 insertions(+), 53 deletions(-)
doc/guides/rel_notes/release_21_11.rst +1 -0
index 21b3c48257..67d2bf5101 100644
@@ -111,6 +111,7 @@ New Features
   * Device allocation APIs.
   * Control plane APIs.
   * Data plane APIs.
+  * Multi-process support.
 
 
 Removed Items
lib/dmadev/rte_dmadev.c +137 -31

311 lines not on this page · raw message

lib/dmadev/rte_dmadev.h +12 -12
index 84e30f7e61..561a1b1154 100644
@@ -821,8 +821,8 @@ rte_dma_copy(int16_t dev_id, uint16_t vchan, rte_iova_t src, rte_iova_t dst,
 	struct rte_dma_dev *dev = &rte_dma_devices[dev_id];
 
 #ifdef RTE_DMADEV_DEBUG
-	if (!rte_dma_is_valid(dev_id) || !dev->dev_started ||
-	    vchan >= dev->dev_conf.nb_vchans || length == 0)
+	if (!rte_dma_is_valid(dev_id) || !dev->data->dev_started ||
+	    vchan >= dev->data->dev_conf.nb_vchans || length == 0)
 		return -EINVAL;
 	RTE_FUNC_PTR_OR_ERR_RET(*dev->copy, -ENOTSUP);
 #endif
@@ -872,8 +872,8 @@ rte_dma_copy_sg(int16_t dev_id, uint16_t vchan, struct rte_dma_sge *src,
 	struct rte_dma_dev *dev = &rte_dma_devices[dev_id];
 
 #ifdef RTE_DMADEV_DEBUG
-	if (!rte_dma_is_valid(dev_id) || !dev->dev_started ||
-	    vchan >= dev->dev_conf.nb_vchans ||
+	if (!rte_dma_is_valid(dev_id) || !dev->data->dev_started ||
+	    vchan >= dev->data->dev_conf.nb_vchans ||
 	    src == NULL || dst == NULL || nb_src == 0 || nb_dst == 0)
 		return -EINVAL;
 	RTE_FUNC_PTR_OR_ERR_RET(*dev->copy_sg, -ENOTSUP);
@@ -919,8 +919,8 @@ rte_dma_fill(int16_t dev_id, uint16_t vchan, uint64_t pattern,
 	struct rte_dma_dev *dev = &rte_dma_devices[dev_id];
 
 #ifdef RTE_DMADEV_DEBUG
-	if (!rte_dma_is_valid(dev_id) || !dev->dev_started ||
-	    vchan >= dev->dev_conf.nb_vchans || length == 0)
+	if (!rte_dma_is_valid(dev_id) || !dev->data->dev_started ||
+	    vchan >= dev->data->dev_conf.nb_vchans || length == 0)
 		return -EINVAL;
 	RTE_FUNC_PTR_OR_ERR_RET(*dev->fill, -ENOTSUP);
 #endif
@@ -952,8 +952,8 @@ rte_dma_submit(int16_t dev_id, uint16_t vchan)
 	struct rte_dma_dev *dev = &rte_dma_devices[dev_id];
 
 #ifdef RTE_DMADEV_DEBUG
-	if (!rte_dma_is_valid(dev_id) || !dev->dev_started ||
-	    vchan >= dev->dev_conf.nb_vchans)
+	if (!rte_dma_is_valid(dev_id) || !dev->data->dev_started ||
+	    vchan >= dev->data->dev_conf.nb_vchans)
 		return -EINVAL;
 	RTE_FUNC_PTR_OR_ERR_RET(*dev->submit, -ENOTSUP);
 #endif
@@ -994,8 +994,8 @@ rte_dma_completed(int16_t dev_id, uint16_t vchan, const uint16_t nb_cpls,
 	bool err;
 
 #ifdef RTE_DMADEV_DEBUG
-	if (!rte_dma_is_valid(dev_id) || !dev->dev_started ||
-	    vchan >= dev->dev_conf.nb_vchans || nb_cpls == 0)
+	if (!rte_dma_is_valid(dev_id) || !dev->data->dev_started ||
+	    vchan >= dev->data->dev_conf.nb_vchans || nb_cpls == 0)
 		return 0;
 	RTE_FUNC_PTR_OR_ERR_RET(*dev->completed, 0);
 #endif
@@ -1054,8 +1054,8 @@ rte_dma_completed_status(int16_t dev_id, uint16_t vchan,
 	uint16_t idx;
 
 #ifdef RTE_DMADEV_DEBUG
-	if (!rte_dma_is_valid(dev_id) || !dev->dev_started ||
-	    vchan >= dev->dev_conf.nb_vchans ||
+	if (!rte_dma_is_valid(dev_id) || !dev->data->dev_started ||
+	    vchan >= dev->data->dev_conf.nb_vchans ||
 	    nb_cpls == 0 || status == NULL)
 		return 0;
 	RTE_FUNC_PTR_OR_ERR_RET(*dev->completed_status, 0);
lib/dmadev/rte_dmadev_core.h +35 -10

65 lines not on this page · raw message

Signature · 1 line
2.33.0
#303 lore raw

PATCH v23 5/6 dma/skeleton: introduce skeleton dmadev driver

In reply to Chengwen Feng

Skeleton dmadevice driver, on the lines of rawdev skeleton, is for showcasing of the dmadev library.

Design of skeleton involves a virtual device which is plugged into VDEV bus on initialization.

Also, enable compilation of dmadev skeleton drivers.

---
 MAINTAINERS                            |   1 +
 drivers/dma/meson.build                |   4 +-
 drivers/dma/skeleton/meson.build       |   7 +
 drivers/dma/skeleton/skeleton_dmadev.c | 570 +++++++++++++++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |  61 +++
 drivers/dma/skeleton/version.map       |   3 +
 6 files changed, 645 insertions(+), 1 deletion(-)
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
MAINTAINERS +1 -0
index a5b11ac70b..85d4f83395 100644
@@ -457,6 +457,7 @@ F: doc/guides/regexdevs/features/default.ini
 DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
+F: drivers/dma/skeleton/
 F: doc/guides/prog_guide/dmadev.rst
 
 Eventdev API
drivers/dma/meson.build +3 -1
index a24c56d8ff..d9c7ede32f 100644
@@ -1,4 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2021 HiSilicon Limited
 
-drivers = []
+drivers = [
+        'skeleton',
+]
drivers/dma/skeleton/meson.build new file +7 -0
new file mode 100644
index 0000000000..8871b80956
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited
+
+deps += ['dmadev', 'kvargs', 'ring', 'bus_vdev']
+sources = files(
+        'skeleton_dmadev.c',
+)
drivers/dma/skeleton/skeleton_dmadev.c new file +570 -0

570 lines not on this page · raw message

drivers/dma/skeleton/skeleton_dmadev.h new file +61 -0

61 lines not on this page · raw message

drivers/dma/skeleton/version.map new file +3 -0
new file mode 100644
index 0000000000..c2e0723b4c
@@ -0,0 +1,3 @@
+DPDK_22 {
+	local: *;
+};
Signature · 1 line
2.33.0
#304 lore raw

PATCH v23 6/6 app/test: add dmadev API test

In reply to Chengwen Feng

This patch add dmadev API test which based on 'dma_skeleton' vdev. The test cases could be executed using 'dmadev_autotest' command in test framework.

---
 MAINTAINERS                |   1 +
 app/test/meson.build       |   4 +
 app/test/test_dmadev.c     |  41 +++
 app/test/test_dmadev_api.c | 574 +++++++++++++++++++++++++++++++++++++
 app/test/test_dmadev_api.h |   5 +
 5 files changed, 625 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
 create mode 100644 app/test/test_dmadev_api.h
MAINTAINERS +1 -0
index 85d4f83395..3258da194d 100644
@@ -458,6 +458,7 @@ DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
 F: drivers/dma/skeleton/
+F: app/test/test_dmadev*
 F: doc/guides/prog_guide/dmadev.rst
 
 Eventdev API
app/test/meson.build +4 -0
index a7611686ad..9027eba3a4 100644
@@ -43,6 +43,8 @@ test_sources = files(
         'test_debug.c',
         'test_distributor.c',
         'test_distributor_perf.c',
+        'test_dmadev.c',
+        'test_dmadev_api.c',
         'test_eal_flags.c',
         'test_eal_fs.c',
         'test_efd.c',
@@ -162,6 +164,7 @@ test_deps = [
         'cmdline',
         'cryptodev',
         'distributor',
+        'dmadev',
         'efd',
         'ethdev',
         'eventdev',
@@ -333,6 +336,7 @@ driver_test_names = [
         'cryptodev_sw_mvsam_autotest',
         'cryptodev_sw_snow3g_autotest',
         'cryptodev_sw_zuc_autotest',
+        'dmadev_autotest',
         'eventdev_selftest_octeontx',
         'eventdev_selftest_sw',
         'rawdev_autotest',
app/test/test_dmadev.c new file +41 -0

41 lines not on this page · raw message

app/test/test_dmadev_api.c new file +574 -0

574 lines not on this page · raw message

app/test/test_dmadev_api.h new file +5 -0
new file mode 100644
index 0000000000..33fbc5bd41
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ */
+
+int test_dma_api(uint16_t dev_id);
Signature · 1 line
2.33.0
#305 lore raw

v24 cover support dmadev

In reply to Chengwen Feng

This patch set contains six patch for new add dmadev.

Chengwen Feng (6):
dmadev: introduce DMA device library
dmadev: add control plane API support
dmadev: add data plane API support
dmadev: add multi-process support
dma/skeleton: introduce skeleton dmadev driver
app/test: add dmadev API test

---
v24:
* use rte_dma_fp_object to hide implementation details.
* support group doxygen for RTE_DMA_CAPA_* and RTE_DMA_OP_*.
* adjusted the naming of some functions.
* fix typo.
v23:
* split multi-process support from 1st patch.
* fix some static check warning.
* fix skeleton cpu thread zero_req_count flip bug.
* add test_dmadev_api.h.
* add the description of modifying the dmadev state when init OK. v22:
* function prefix change from rte_dmadev_* to rte_dma_*.
* change to prefix comment in most scenarios.
* dmadev dev_id use int16_t type.
* fix typo.
* organize patchsets in incremental mode.
v21:
* add comment for reserved fields of struct rte_dmadev. v20:
* delete unnecessary and duplicate include header files.
* the conf_sz parameter is added to the configure and vchan-setup callbacks of the PMD, this is mainly used to enhance ABI compatibility.
* the rte_dmadev structure field is rearranged to reserve more space for I/O functions.
* fix some ambiguous and unnecessary comments.
* fix the potential memory leak of ut.
* redefine skeldma_init_once to skeldma_count.
* suppress rte_dmadev error output when execute ut.

 MAINTAINERS                            |    7 +
 app/test/meson.build                   |    4 +
 app/test/test_dmadev.c                 |   41 +
 app/test/test_dmadev_api.c             |  574 +++++++++++++
 app/test/test_dmadev_api.h             |    5 +
 doc/api/doxy-api-index.md              |    1 +
 doc/api/doxy-api.conf.in               |    1 +
 doc/guides/dmadevs/index.rst           |   12 +
 doc/guides/index.rst                   |    1 +
 doc/guides/prog_guide/dmadev.rst       |  120 +++
 doc/guides/prog_guide/img/dmadev.svg   |  283 +++++++
 doc/guides/prog_guide/index.rst        |    1 +
 doc/guides/rel_notes/release_21_11.rst |    6 +
 drivers/dma/meson.build                |    6 +
 drivers/dma/skeleton/meson.build       |    7 +
 drivers/dma/skeleton/skeleton_dmadev.c |  571 +++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |   61 ++
 drivers/dma/skeleton/version.map       |    3 +
 drivers/meson.build                    |    1 +
 lib/dmadev/meson.build                 |    7 +
 lib/dmadev/rte_dmadev.c                |  844 +++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 1048 ++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |   78 ++
 lib/dmadev/rte_dmadev_pmd.h            |  173 ++++
 lib/dmadev/version.map                 |   35 +
 lib/meson.build                        |    1 +
 26 files changed, 3891 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
 create mode 100644 app/test/test_dmadev_api.h
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
Signature · 1 line
2.33.0
#306 lore raw

PATCH v24 1/6 dmadev: introduce DMA device library

The 'dmadev' is a generic type of DMA device.

This patch introduce the 'dmadev' device allocation functions.

The infrastructure is prepared to welcome drivers in drivers/dma/

---
 MAINTAINERS                            |   5 +
 doc/api/doxy-api-index.md              |   1 +
 doc/api/doxy-api.conf.in               |   1 +
 doc/guides/dmadevs/index.rst           |  12 ++
 doc/guides/index.rst                   |   1 +
 doc/guides/prog_guide/dmadev.rst       |  60 ++++++
 doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
 doc/guides/prog_guide/index.rst        |   1 +
 doc/guides/rel_notes/release_21_11.rst |   4 +
 drivers/dma/meson.build                |   4 +
 drivers/meson.build                    |   1 +
 lib/dmadev/meson.build                 |   6 +
 lib/dmadev/rte_dmadev.c                | 246 +++++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 133 ++++++++++++
 lib/dmadev/rte_dmadev_pmd.h            |  90 ++++++++
 lib/dmadev/version.map                 |  20 ++
 lib/meson.build                        |   1 +
 17 files changed, 869 insertions(+)
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
MAINTAINERS +5 -0
index 278e5b3226..119cfaa04e 100644
@@ -454,6 +454,11 @@ F: app/test-regex/
 F: doc/guides/prog_guide/regexdev.rst
 F: doc/guides/regexdevs/features/default.ini
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <[email protected]>
+F: lib/dmadev/
+F: doc/guides/prog_guide/dmadev.rst
+
 Eventdev API
 M: Jerin Jacob <[email protected]>
 T: git://dpdk.org/next/dpdk-next-eventdev
doc/api/doxy-api-index.md +1 -0
index 1992107a03..2939050431 100644
@@ -21,6 +21,7 @@ The public API headers are grouped by topics:
   [compressdev]        (@ref rte_compressdev.h),
   [compress]           (@ref rte_comp.h),
   [regexdev]           (@ref rte_regexdev.h),
+  [dmadev]             (@ref rte_dmadev.h),
   [eventdev]           (@ref rte_eventdev.h),
   [event_eth_rx_adapter]   (@ref rte_event_eth_rx_adapter.h),
   [event_eth_tx_adapter]   (@ref rte_event_eth_tx_adapter.h),
doc/api/doxy-api.conf.in +1 -0
index 325a0195c6..109ec1f682 100644
@@ -35,6 +35,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/compressdev \
                           @TOPDIR@/lib/cryptodev \
                           @TOPDIR@/lib/distributor \
+                          @TOPDIR@/lib/dmadev \
                           @TOPDIR@/lib/efd \
                           @TOPDIR@/lib/ethdev \
                           @TOPDIR@/lib/eventdev \
doc/guides/dmadevs/index.rst new file +12 -0
new file mode 100644
index 0000000000..0bce29d766
@@ -0,0 +1,12 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright 2021 HiSilicon Limited
+
+DMA Device Drivers
+==================
+
+The following are a list of DMA device drivers, which can be used from
+an application through DMA API.
+
+.. toctree::
+   :maxdepth: 2
+   :numbered:
doc/guides/index.rst +1 -0
index 857f0363d3..919825992e 100644
@@ -21,6 +21,7 @@ DPDK documentation
    compressdevs/index
    vdpadevs/index
    regexdevs/index
+   dmadevs/index
    eventdevs/index
    rawdevs/index
    mempool/index
doc/guides/prog_guide/dmadev.rst new file +60 -0

60 lines not on this page · raw message

doc/guides/prog_guide/img/dmadev.svg new file +283 -0

283 lines not on this page · raw message

doc/guides/prog_guide/index.rst +1 -0
index 2dce507f46..89af28dacb 100644
@@ -27,6 +27,7 @@ Programmer's Guide
     cryptodev_lib
     compressdev
     regexdev
+    dmadev
     rte_security
     rawdev
     link_bonding_poll_mode_drv_lib
doc/guides/rel_notes/release_21_11.rst +4 -0
index 89d4b33ef1..929c0d6113 100644
@@ -141,6 +141,10 @@ New Features
   * Added tests to validate packets hard expiry.
   * Added tests to verify tunnel header verification in IPsec inbound.
 
+* **Introduced dmadev library with:**
+
+  * Device allocation functions.
+
 
 Removed Items
 -------------
drivers/dma/meson.build new file +4 -0
new file mode 100644
index 0000000000..a24c56d8ff
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2021 HiSilicon Limited
+
+drivers = []
drivers/meson.build +1 -0
index 3d08540581..b7d680868a 100644
@@ -18,6 +18,7 @@ subdirs = [
         'vdpa',           # depends on common, bus and mempool.
         'event',          # depends on common, bus, mempool and net.
         'baseband',       # depends on common and bus.
+        'dma',            # depends on common and bus.
 ]
 
 if meson.is_cross_build()
lib/dmadev/meson.build new file +6 -0
new file mode 100644
index 0000000000..f8d54c6e74
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited.
+
+sources = files('rte_dmadev.c')
+headers = files('rte_dmadev.h')
+driver_sdk_headers += files('rte_dmadev_pmd.h')
lib/dmadev/rte_dmadev.c new file +246 -0

246 lines not on this page · raw message

lib/dmadev/rte_dmadev.h new file +133 -0

133 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h new file +90 -0

90 lines not on this page · raw message

lib/dmadev/version.map new file +20 -0
new file mode 100644
index 0000000000..f8a0076468
@@ -0,0 +1,20 @@
+EXPERIMENTAL {
+	global:
+
+	rte_dma_count_avail;
+	rte_dma_dev_max;
+	rte_dma_get_dev_id_by_name;
+	rte_dma_is_valid;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	rte_dma_devices;
+	rte_dma_pmd_allocate;
+	rte_dma_pmd_release;
+
+	local: *;
+};
lib/meson.build +1 -0
index 9c4841fe40..0bf1f51357 100644
@@ -45,6 +45,7 @@ libraries = [
         'pdump',
         'rawdev',
         'regexdev',
+        'dmadev',
         'rib',
         'reorder',
         'sched',
Signature · 1 line
2.33.0
#307 lore raw

PATCH v24 2/6 dmadev: add control plane API support

In reply to Chengwen Feng

This patch add control plane API for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  38 ++
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.c                | 360 +++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 464 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_pmd.h            |  61 ++++
 lib/dmadev/version.map                 |   9 +
 6 files changed, 933 insertions(+)
doc/guides/prog_guide/dmadev.rst +38 -0
index 90bda28f33..5c70ad3d6a 100644
@@ -58,3 +58,41 @@ identifiers:
 
 - A device name used to designate the DMA device in console messages, for
   administration or debugging purposes.
+
+
+Device Configuration
+~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_configure API is used to configure a DMA device.
+
+.. code-block:: c
+
+   int rte_dma_configure(int16_t dev_id,
+                         const struct rte_dma_conf *dev_conf);
+
+The ``rte_dma_conf`` structure is used to pass the configuration parameters
+for the DMA device.
+
+
+Configuration of Virtual DMA Channels
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_vchan_setup API is used to configure a virtual DMA channel.
+
+.. code-block:: c
+
+   int rte_dma_vchan_setup(int16_t dev_id, uint16_t vchan,
+                           const struct rte_dma_vchan_conf *conf);
+
+The ``rte_dma_vchan_conf`` structure is used to pass the configuration
+parameters for the virtual DMA channel.
+
+
+Device Features and Capabilities
+--------------------------------
+
+DMA devices may support different feature sets. The ``rte_dma_info_get`` API
+can be used to get the device info and supported features.
+
+Silent mode is a special device capability which does not require the
+application to invoke dequeue APIs.
doc/guides/rel_notes/release_21_11.rst +1 -0
index 929c0d6113..f935a3f395 100644
@@ -144,6 +144,7 @@ New Features
 * **Introduced dmadev library with:**
 
   * Device allocation functions.
+  * Control plane API.
 
 
 Removed Items
lib/dmadev/rte_dmadev.c +360 -0

369 lines not on this page · raw message

lib/dmadev/rte_dmadev.h +464 -0

476 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h +61 -0

74 lines not on this page · raw message

lib/dmadev/version.map +9 -0
index f8a0076468..e925dfcd6d 100644
@@ -1,10 +1,19 @@
 EXPERIMENTAL {
 	global:
 
+	rte_dma_close;
+	rte_dma_configure;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
+	rte_dma_dump;
 	rte_dma_get_dev_id_by_name;
+	rte_dma_info_get;
 	rte_dma_is_valid;
+	rte_dma_start;
+	rte_dma_stats_get;
+	rte_dma_stats_reset;
+	rte_dma_stop;
+	rte_dma_vchan_setup;
 
 	local: *;
 };
Signature · 1 line
2.33.0
#308 lore raw

PATCH v24 3/6 dmadev: add data plane API support

In reply to Chengwen Feng

This patch add data plane API for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  22 ++
 doc/guides/rel_notes/release_21_11.rst |   2 +-
 lib/dmadev/meson.build                 |   1 +
 lib/dmadev/rte_dmadev.c                | 134 ++++++++
 lib/dmadev/rte_dmadev.h                | 451 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  78 +++++
 lib/dmadev/rte_dmadev_pmd.h            |   7 +
 lib/dmadev/version.map                 |   6 +
 8 files changed, 700 insertions(+), 1 deletion(-)
 create mode 100644 lib/dmadev/rte_dmadev_core.h
doc/guides/prog_guide/dmadev.rst +22 -0
index 5c70ad3d6a..2e2a4bb62a 100644
@@ -96,3 +96,25 @@ can be used to get the device info and supported features.
 
 Silent mode is a special device capability which does not require the
 application to invoke dequeue APIs.
+
+
+Enqueue / Dequeue APIs
+~~~~~~~~~~~~~~~~~~~~~~
+
+Enqueue APIs such as ``rte_dma_copy`` and ``rte_dma_fill`` can be used to
+enqueue operations to hardware. If an enqueue is successful, a ``ring_idx`` is
+returned. This ``ring_idx`` can be used by applications to track per operation
+metadata in an application-defined circular ring.
+
+The ``rte_dma_submit`` API is used to issue doorbell to hardware.
+Alternatively the ``RTE_DMA_OP_FLAG_SUBMIT`` flag can be passed to the enqueue
+APIs to also issue the doorbell to hardware.
+
+There are two dequeue APIs ``rte_dma_completed`` and
+``rte_dma_completed_status``, these are used to obtain the results of the
+enqueue requests. ``rte_dma_completed`` will return the number of successfully
+completed operations. ``rte_dma_completed_status`` will return the number of
+completed operations along with the status of each operation (filled into the
+``status`` array passed by user). These two APIs can also return the last
+completed operation's ``ring_idx`` which could help user track operations within
+their own application-defined rings.
doc/guides/rel_notes/release_21_11.rst +1 -1
index f935a3f395..d1d7abf694 100644
@@ -144,7 +144,7 @@ New Features
 * **Introduced dmadev library with:**
 
   * Device allocation functions.
-  * Control plane API.
+  * Control and data plane API.
 
 
 Removed Items
lib/dmadev/meson.build +1 -0
index f8d54c6e74..d2fc85e8c7 100644
@@ -3,4 +3,5 @@
 
 sources = files('rte_dmadev.c')
 headers = files('rte_dmadev.h')
+indirect_headers += files('rte_dmadev_core.h')
 driver_sdk_headers += files('rte_dmadev_pmd.h')
lib/dmadev/rte_dmadev.c +134 -0

175 lines not on this page · raw message

lib/dmadev/rte_dmadev.h +451 -0

463 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h new file +78 -0

78 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h +7 -0
index 5fcf0f60b8..07056b45e7 100644
@@ -75,6 +75,13 @@ struct rte_dma_dev_ops {
 	rte_dma_stats_reset_t      stats_reset;
 
 	rte_dma_dump_t             dev_dump;
+
+	rte_dma_copy_t             copy;
+	rte_dma_copy_sg_t          copy_sg;
+	rte_dma_fill_t             fill;
+	rte_dma_submit_t           submit;
+	rte_dma_completed_t        completed;
+	rte_dma_completed_status_t completed_status;
 };
 /**
  * Possible states of a DMA device.
lib/dmadev/version.map +6 -0
index e925dfcd6d..4d40104689 100644
@@ -2,10 +2,15 @@ EXPERIMENTAL {
 	global:
 
 	rte_dma_close;
+	rte_dma_completed;
+	rte_dma_completed_status;
 	rte_dma_configure;
+	rte_dma_copy;
+	rte_dma_copy_sg;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
 	rte_dma_dump;
+	rte_dma_fill;
 	rte_dma_get_dev_id_by_name;
 	rte_dma_info_get;
 	rte_dma_is_valid;
@@ -13,6 +18,7 @@ EXPERIMENTAL {
 	rte_dma_stats_get;
 	rte_dma_stats_reset;
 	rte_dma_stop;
+	rte_dma_submit;
 	rte_dma_vchan_setup;
 
 	local: *;
Signature · 1 line
2.33.0
#309 lore raw

Re: [dpdk-dev] [PATCH v24 3/6] dmadev: add data plane API support

This patch follows the idea of Konstantin, and introduces rte_dma_fp_object to hide implementation detail.
This change modify the first parameter of drivers's dataplane interface: from 'struct rte_dma_dev *dev' to 'void *dev_private'.

Quoted 19 lines · Chengwen Feng

This patch add data plane API for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  22 ++
 doc/guides/rel_notes/release_21_11.rst |   2 +-
 lib/dmadev/meson.build                 |   1 +
 lib/dmadev/rte_dmadev.c                | 134 ++++++++
 lib/dmadev/rte_dmadev.h                | 451 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  78 +++++
 lib/dmadev/rte_dmadev_pmd.h            |   7 +
 lib/dmadev/version.map                 |   6 +
 8 files changed, 700 insertions(+), 1 deletion(-)
 create mode 100644 lib/dmadev/rte_dmadev_core.h

[snip]

#310 lore raw

Re: [dpdk-dev] [PATCH v24 3/6] dmadev: add data plane API support

In reply to Chengwen Feng
Quoted 2 lines · Chengwen Feng

This patch add data plane API for dmadev.

A few initial comments inline. I'll work on rebasing my follow-up patchset to this, and let you know if I have any more feedback based on that.

/Bruce

lib/dmadev/rte_dmadev.c line 20

Quoted 9 lines
lib/dmadev/rte_dmadev.c +1 -0
index a6a5680d2b..891ceeb988 100644
@@ -17,6 +17,7 @@
 
 static int16_t dma_devices_max;
 
+struct rte_dma_fp_object *rte_dma_fp_objs;

While I think I like this approach of making more of the dmadev hidden, I think we need a better name for this. While there is the dev_private pointer in it, the struct is pretty much the datapath functions, so how about "rte_dma_funcs" as a name?

Quoted 2 lines
lib/dmadev/rte_dmadev.c +0 -0
 struct rte_dma_dev *rte_dma_devices;
 

<snip>

Quoted 13 lines
lib/dmadev/rte_dmadev.c +13 -0
+/**
+ * @internal
+ * Fast-path dmadev functions and related data are hold in a flat array.
+ * One entry per dmadev.
+ *
+ * On 64-bit systems contents of this structure occupy exactly two 64B lines.
+ * On 32-bit systems contents of this structure fits into one 64B line.
+ *
+ * The 'dev_private' field was placed in the first cache line to optimize
+ * performance because the PMD driver mainly depends on this field.
+ */
+struct rte_dma_fp_object {
+	void *dev_private; /**< PMD-specific private data. */

lib/dmadev/rte_dmadev.c

Quoted 10 lines
lib/dmadev/rte_dmadev.c +10 -0
+	rte_dma_copy_t             copy;
+	rte_dma_copy_sg_t          copy_sg;
+	rte_dma_fill_t             fill;
+	rte_dma_submit_t           submit;
+	rte_dma_completed_t        completed;
+	rte_dma_completed_status_t completed_status;
+	void *reserved_cl0;
+	/** Reserve space for future IO functions, while keeping data and
+	 * dev_ops pointers on the second cacheline.
+	 */

This comment is out of date.

lib/dmadev/rte_dmadev.c

Quoted 2 lines
lib/dmadev/rte_dmadev.c +2 -0
+	void *reserved_cl1[6];
+} __rte_cache_aligned;

Small suggestion: since there is no data at the end of the structure, rather than adding in padding arrays which need to be adjusted as we add fields into the struct, let's just change the "__rte_cache_aligned" macro to "__rte_aligned(128)". This will explicitly set the size to 128-bytes and allow us to remove the reserved fields - making it easier to add new pointers.

Quoted 4 lines
lib/dmadev/rte_dmadev.c +4 -0
+
+extern struct rte_dma_fp_object *rte_dma_fp_objs;
+
+#endif /* RTE_DMADEV_CORE_H */
#311 lore raw

Re: [dpdk-dev] [PATCH v24 3/6] dmadev: add data plane API support

Quoted 15 lines · Bruce Richardson
Quoted 2 lines · Chengwen Feng

This patch add data plane API for dmadev.

A few initial comments inline. I'll work on rebasing my follow-up patchset to this, and let you know if I have any more feedback based on that.

/Bruce

Quoted 6 lines
lib/dmadev/rte_dmadev.c +0 -0
index a6a5680d2b..891ceeb988 100644
@@ -17,6 +17,7 @@
 
Quoted 8 lines

lib/dmadev/rte_dmadev.c line 20

Quoted 3 lines
lib/dmadev/rte_dmadev.c +1 -0
 static int16_t dma_devices_max;
 
+struct rte_dma_fp_object *rte_dma_fp_objs;

While I think I like this approach of making more of the dmadev hidden, I think we need a better name for this. While there is the dev_private pointer in it, the struct is pretty much the datapath functions, so how about "rte_dma_funcs" as a name?

en, I notice ethdev and eventdev both use rte_xxx_fp_ops, but this structure has other fileds(e.g. data pointers) in addition to ops, it's inappropriate to use ops suffix. So I use the 'object' which is widely used in object-oriented.

It's better to use uniform naming in ethdev/eventdev/dmadev and so on, would be happy to hear more.

Quoted 31 lines
Quoted 3 lines
lib/dmadev/rte_dmadev.c +0 -0
 struct rte_dma_dev *rte_dma_devices;
 

<snip>

Quoted 13 lines
lib/dmadev/rte_dmadev.c +13 -0
+/**
+ * @internal
+ * Fast-path dmadev functions and related data are hold in a flat array.
+ * One entry per dmadev.
+ *
+ * On 64-bit systems contents of this structure occupy exactly two 64B lines.
+ * On 32-bit systems contents of this structure fits into one 64B line.
+ *
+ * The 'dev_private' field was placed in the first cache line to optimize
+ * performance because the PMD driver mainly depends on this field.
+ */
+struct rte_dma_fp_object {
+	void *dev_private; /**< PMD-specific private data. */

lib/dmadev/rte_dmadev.c

Quoted 10 lines
lib/dmadev/rte_dmadev.c +10 -0
+	rte_dma_copy_t             copy;
+	rte_dma_copy_sg_t          copy_sg;
+	rte_dma_fill_t             fill;
+	rte_dma_submit_t           submit;
+	rte_dma_completed_t        completed;
+	rte_dma_completed_status_t completed_status;
+	void *reserved_cl0;
+	/** Reserve space for future IO functions, while keeping data and
+	 * dev_ops pointers on the second cacheline.
+	 */

This comment is out of date.

Quoted 9 lines

lib/dmadev/rte_dmadev.c

Quoted 2 lines
lib/dmadev/rte_dmadev.c +2 -0
+	void *reserved_cl1[6];
+} __rte_cache_aligned;

Small suggestion: since there is no data at the end of the structure, rather than adding in padding arrays which need to be adjusted as we add fields into the struct, let's just change the "__rte_cache_aligned" macro to "__rte_aligned(128)". This will explicitly set the size to 128-bytes and allow us to remove the reserved fields - making it easier to add new pointers.

Agree

Quoted 8 lines

lib/dmadev/rte_dmadev.c

Quoted 5 lines
lib/dmadev/rte_dmadev.c +4 -0
+
+extern struct rte_dma_fp_object *rte_dma_fp_objs;
+
+#endif /* RTE_DMADEV_CORE_H */

.

Thanks

#312 lore raw

PATCH v24 4/6 dmadev: add multi-process support

In reply to Chengwen Feng

This patch add multi-process support for dmadev.

---
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.c                | 176 ++++++++++++++++++++-----
 lib/dmadev/rte_dmadev_pmd.h            |  29 +++-
 3 files changed, 163 insertions(+), 43 deletions(-)
doc/guides/rel_notes/release_21_11.rst +1 -0
index d1d7abf694..af32fce1ed 100644
@@ -145,6 +145,7 @@ New Features
 
   * Device allocation functions.
   * Control and data plane API.
+  * Multi-process support.
 
 
 Removed Items
lib/dmadev/rte_dmadev.c +140 -36

328 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h +22 -7
index 07056b45e7..c2902eddd9 100644
@@ -83,6 +83,27 @@ struct rte_dma_dev_ops {
 	rte_dma_completed_t        completed;
 	rte_dma_completed_status_t completed_status;
 };
+
+/**
+ * @internal
+ * The data part, with no function pointers, associated with each DMA device.
+ *
+ * This structure is safe to place in shared memory to be common among different
+ * processes in a multi-process configuration.
+ *
+ * @see struct rte_dma_dev::data
+ */
+struct rte_dma_dev_data {
+	char dev_name[RTE_DEV_NAME_MAX_LEN]; /**< Unique identifier name */
+	int16_t dev_id; /**< Device [external] identifier. */
+	int16_t numa_node; /**< Local NUMA memory ID. -1 if unknown. */
+	void *dev_private; /**< PMD-specific private data. */
+	struct rte_dma_conf dev_conf; /**< DMA device configuration. */
+	__extension__
+	uint8_t dev_started : 1; /**< Device state: STARTED(1)/STOPPED(0). */
+	uint64_t reserved[2]; /**< Reserved for future fields */
+} __rte_cache_aligned;
+
 /**
  * Possible states of a DMA device.
  *
@@ -101,18 +122,12 @@ enum rte_dma_dev_state {
  * The generic data structure associated with each DMA device.
  */
 struct rte_dma_dev {
-	char dev_name[RTE_DEV_NAME_MAX_LEN]; /**< Unique identifier name */
-	int16_t dev_id; /**< Device [external] identifier. */
-	int16_t numa_node; /**< Local NUMA memory ID. -1 if unknown. */
-	void *dev_private; /**< PMD-specific private data. */
 	/** Device info which supplied during device initialization. */
 	struct rte_device *device;
+	struct rte_dma_dev_data *data; /**< Pointer to shared device data. */
 	/** Functions implemented by PMD. */
 	const struct rte_dma_dev_ops *dev_ops;
-	struct rte_dma_conf dev_conf; /**< DMA device configuration. */
 	enum rte_dma_dev_state state; /**< Flag indicating the device state. */
-	__extension__
-	uint8_t dev_started : 1; /**< Device state: STARTED(1)/STOPPED(0). */
 	uint64_t reserved[2]; /**< Reserved for future fields. */
 } __rte_cache_aligned;
 
Signature · 1 line
2.33.0
#313 lore raw

PATCH v24 5/6 dma/skeleton: introduce skeleton dmadev driver

In reply to Chengwen Feng

Skeleton dmadevice driver, on the lines of rawdev skeleton, is for showcasing of the dmadev library.

Design of skeleton involves a virtual device which is plugged into VDEV bus on initialization.

Also, enable compilation of dmadev skeleton drivers.

---
 MAINTAINERS                            |   1 +
 drivers/dma/meson.build                |   4 +-
 drivers/dma/skeleton/meson.build       |   7 +
 drivers/dma/skeleton/skeleton_dmadev.c | 571 +++++++++++++++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |  61 +++
 drivers/dma/skeleton/version.map       |   3 +
 6 files changed, 646 insertions(+), 1 deletion(-)
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
MAINTAINERS +1 -0
index 119cfaa04e..ec887ac49f 100644
@@ -457,6 +457,7 @@ F: doc/guides/regexdevs/features/default.ini
 DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
+F: drivers/dma/skeleton/
 F: doc/guides/prog_guide/dmadev.rst
 
 Eventdev API
drivers/dma/meson.build +3 -1
index a24c56d8ff..d9c7ede32f 100644
@@ -1,4 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2021 HiSilicon Limited
 
-drivers = []
+drivers = [
+        'skeleton',
+]
drivers/dma/skeleton/meson.build new file +7 -0
new file mode 100644
index 0000000000..8871b80956
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited
+
+deps += ['dmadev', 'kvargs', 'ring', 'bus_vdev']
+sources = files(
+        'skeleton_dmadev.c',
+)
drivers/dma/skeleton/skeleton_dmadev.c new file +571 -0

571 lines not on this page · raw message

drivers/dma/skeleton/skeleton_dmadev.h new file +61 -0

61 lines not on this page · raw message

drivers/dma/skeleton/version.map new file +3 -0
new file mode 100644
index 0000000000..c2e0723b4c
@@ -0,0 +1,3 @@
+DPDK_22 {
+	local: *;
+};
Signature · 1 line
2.33.0
#314 lore raw

PATCH v24 6/6 app/test: add dmadev API test

In reply to Chengwen Feng

This patch add dmadev API test which based on 'dma_skeleton' vdev. The test cases could be executed using 'dmadev_autotest' command in test framework.

---
 MAINTAINERS                |   1 +
 app/test/meson.build       |   4 +
 app/test/test_dmadev.c     |  41 +++
 app/test/test_dmadev_api.c | 574 +++++++++++++++++++++++++++++++++++++
 app/test/test_dmadev_api.h |   5 +
 5 files changed, 625 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
 create mode 100644 app/test/test_dmadev_api.h
MAINTAINERS +1 -0
index ec887ac49f..d329873465 100644
@@ -458,6 +458,7 @@ DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
 F: drivers/dma/skeleton/
+F: app/test/test_dmadev*
 F: doc/guides/prog_guide/dmadev.rst
 
 Eventdev API
app/test/meson.build +4 -0
index f144d8b8ed..a16374b7a1 100644
@@ -44,6 +44,8 @@ test_sources = files(
         'test_debug.c',
         'test_distributor.c',
         'test_distributor_perf.c',
+        'test_dmadev.c',
+        'test_dmadev_api.c',
         'test_eal_flags.c',
         'test_eal_fs.c',
         'test_efd.c',
@@ -163,6 +165,7 @@ test_deps = [
         'cmdline',
         'cryptodev',
         'distributor',
+        'dmadev',
         'efd',
         'ethdev',
         'eventdev',
@@ -334,6 +337,7 @@ driver_test_names = [
         'cryptodev_sw_mvsam_autotest',
         'cryptodev_sw_snow3g_autotest',
         'cryptodev_sw_zuc_autotest',
+        'dmadev_autotest',
         'eventdev_selftest_octeontx',
         'eventdev_selftest_sw',
         'rawdev_autotest',
app/test/test_dmadev.c new file +41 -0

41 lines not on this page · raw message

app/test/test_dmadev_api.c new file +574 -0

574 lines not on this page · raw message

app/test/test_dmadev_api.h new file +5 -0
new file mode 100644
index 0000000000..33fbc5bd41
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ */
+
+int test_dma_api(uint16_t dev_id);
Signature · 1 line
2.33.0
#315 lore raw

v25 cover support dmadev

In reply to Chengwen Feng

This patch set contains six patch for new add dmadev.

Chengwen Feng (6):
dmadev: introduce DMA device library
dmadev: add control plane API support
dmadev: add data plane API support
dmadev: add multi-process support
dma/skeleton: introduce skeleton dmadev driver
app/test: add dmadev API test

---
v25:
* fix undefined reference to rte_dma_fp_objs with some compiler suite.
* use rte_dma_dev hold pointer to rte_dma_fp_objs to avoid primary and secondary disuniformity.
v24:
* use rte_dma_fp_object to hide implementation details.
* support group doxygen for RTE_DMA_CAPA_* and RTE_DMA_OP_*.
* adjusted the naming of some functions.
* fix typo.
v23:
* split multi-process support from 1st patch.
* fix some static check warning.
* fix skeleton cpu thread zero_req_count flip bug.
* add test_dmadev_api.h.
* add the description of modifying the dmadev state when init OK. v22:
* function prefix change from rte_dmadev_* to rte_dma_*.
* change to prefix comment in most scenarios.
* dmadev dev_id use int16_t type.
* fix typo.
* organize patchsets in incremental mode.
v21:
* add comment for reserved fields of struct rte_dmadev.

 MAINTAINERS                            |    7 +
 app/test/meson.build                   |    4 +
 app/test/test_dmadev.c                 |   41 +
 app/test/test_dmadev_api.c             |  574 +++++++++++++
 app/test/test_dmadev_api.h             |    5 +
 doc/api/doxy-api-index.md              |    1 +
 doc/api/doxy-api.conf.in               |    1 +
 doc/guides/dmadevs/index.rst           |   12 +
 doc/guides/index.rst                   |    1 +
 doc/guides/prog_guide/dmadev.rst       |  120 +++
 doc/guides/prog_guide/img/dmadev.svg   |  283 +++++++
 doc/guides/prog_guide/index.rst        |    1 +
 doc/guides/rel_notes/release_21_11.rst |    6 +
 drivers/dma/meson.build                |    6 +
 drivers/dma/skeleton/meson.build       |    7 +
 drivers/dma/skeleton/skeleton_dmadev.c |  571 +++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |   61 ++
 drivers/dma/skeleton/version.map       |    3 +
 drivers/meson.build                    |    1 +
 lib/dmadev/meson.build                 |    7 +
 lib/dmadev/rte_dmadev.c                |  825 +++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 1048 ++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |   81 ++
 lib/dmadev/rte_dmadev_pmd.h            |  168 ++++
 lib/dmadev/version.map                 |   36 +
 lib/meson.build                        |    1 +
 26 files changed, 3871 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
 create mode 100644 app/test/test_dmadev_api.h
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
Signature · 1 line
2.33.0
#316 lore raw

PATCH v25 1/6 dmadev: introduce DMA device library

The 'dmadev' is a generic type of DMA device.

This patch introduce the 'dmadev' device allocation functions.

The infrastructure is prepared to welcome drivers in drivers/dma/

---
 MAINTAINERS                            |   5 +
 doc/api/doxy-api-index.md              |   1 +
 doc/api/doxy-api.conf.in               |   1 +
 doc/guides/dmadevs/index.rst           |  12 ++
 doc/guides/index.rst                   |   1 +
 doc/guides/prog_guide/dmadev.rst       |  60 ++++++
 doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
 doc/guides/prog_guide/index.rst        |   1 +
 doc/guides/rel_notes/release_21_11.rst |   4 +
 drivers/dma/meson.build                |   4 +
 drivers/meson.build                    |   1 +
 lib/dmadev/meson.build                 |   6 +
 lib/dmadev/rte_dmadev.c                | 246 +++++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 133 ++++++++++++
 lib/dmadev/rte_dmadev_pmd.h            |  90 ++++++++
 lib/dmadev/version.map                 |  20 ++
 lib/meson.build                        |   1 +
 17 files changed, 869 insertions(+)
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
MAINTAINERS +5 -0
index 278e5b3226..119cfaa04e 100644
@@ -454,6 +454,11 @@ F: app/test-regex/
 F: doc/guides/prog_guide/regexdev.rst
 F: doc/guides/regexdevs/features/default.ini
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <[email protected]>
+F: lib/dmadev/
+F: doc/guides/prog_guide/dmadev.rst
+
 Eventdev API
 M: Jerin Jacob <[email protected]>
 T: git://dpdk.org/next/dpdk-next-eventdev
doc/api/doxy-api-index.md +1 -0
index 1992107a03..2939050431 100644
@@ -21,6 +21,7 @@ The public API headers are grouped by topics:
   [compressdev]        (@ref rte_compressdev.h),
   [compress]           (@ref rte_comp.h),
   [regexdev]           (@ref rte_regexdev.h),
+  [dmadev]             (@ref rte_dmadev.h),
   [eventdev]           (@ref rte_eventdev.h),
   [event_eth_rx_adapter]   (@ref rte_event_eth_rx_adapter.h),
   [event_eth_tx_adapter]   (@ref rte_event_eth_tx_adapter.h),
doc/api/doxy-api.conf.in +1 -0
index 325a0195c6..109ec1f682 100644
@@ -35,6 +35,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/compressdev \
                           @TOPDIR@/lib/cryptodev \
                           @TOPDIR@/lib/distributor \
+                          @TOPDIR@/lib/dmadev \
                           @TOPDIR@/lib/efd \
                           @TOPDIR@/lib/ethdev \
                           @TOPDIR@/lib/eventdev \
doc/guides/dmadevs/index.rst new file +12 -0
new file mode 100644
index 0000000000..0bce29d766
@@ -0,0 +1,12 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright 2021 HiSilicon Limited
+
+DMA Device Drivers
+==================
+
+The following are a list of DMA device drivers, which can be used from
+an application through DMA API.
+
+.. toctree::
+   :maxdepth: 2
+   :numbered:
doc/guides/index.rst +1 -0
index 857f0363d3..919825992e 100644
@@ -21,6 +21,7 @@ DPDK documentation
    compressdevs/index
    vdpadevs/index
    regexdevs/index
+   dmadevs/index
    eventdevs/index
    rawdevs/index
    mempool/index
doc/guides/prog_guide/dmadev.rst new file +60 -0

60 lines not on this page · raw message

doc/guides/prog_guide/img/dmadev.svg new file +283 -0

283 lines not on this page · raw message

doc/guides/prog_guide/index.rst +1 -0
index 2dce507f46..89af28dacb 100644
@@ -27,6 +27,7 @@ Programmer's Guide
     cryptodev_lib
     compressdev
     regexdev
+    dmadev
     rte_security
     rawdev
     link_bonding_poll_mode_drv_lib
doc/guides/rel_notes/release_21_11.rst +4 -0
index 89d4b33ef1..929c0d6113 100644
@@ -141,6 +141,10 @@ New Features
   * Added tests to validate packets hard expiry.
   * Added tests to verify tunnel header verification in IPsec inbound.
 
+* **Introduced dmadev library with:**
+
+  * Device allocation functions.
+
 
 Removed Items
 -------------
drivers/dma/meson.build new file +4 -0
new file mode 100644
index 0000000000..a24c56d8ff
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2021 HiSilicon Limited
+
+drivers = []
drivers/meson.build +1 -0
index 3d08540581..b7d680868a 100644
@@ -18,6 +18,7 @@ subdirs = [
         'vdpa',           # depends on common, bus and mempool.
         'event',          # depends on common, bus, mempool and net.
         'baseband',       # depends on common and bus.
+        'dma',            # depends on common and bus.
 ]
 
 if meson.is_cross_build()
lib/dmadev/meson.build new file +6 -0
new file mode 100644
index 0000000000..f8d54c6e74
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited.
+
+sources = files('rte_dmadev.c')
+headers = files('rte_dmadev.h')
+driver_sdk_headers += files('rte_dmadev_pmd.h')
lib/dmadev/rte_dmadev.c new file +246 -0

246 lines not on this page · raw message

lib/dmadev/rte_dmadev.h new file +133 -0

133 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h new file +90 -0

90 lines not on this page · raw message

lib/dmadev/version.map new file +20 -0
new file mode 100644
index 0000000000..f8a0076468
@@ -0,0 +1,20 @@
+EXPERIMENTAL {
+	global:
+
+	rte_dma_count_avail;
+	rte_dma_dev_max;
+	rte_dma_get_dev_id_by_name;
+	rte_dma_is_valid;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	rte_dma_devices;
+	rte_dma_pmd_allocate;
+	rte_dma_pmd_release;
+
+	local: *;
+};
lib/meson.build +1 -0
index 9c4841fe40..0bf1f51357 100644
@@ -45,6 +45,7 @@ libraries = [
         'pdump',
         'rawdev',
         'regexdev',
+        'dmadev',
         'rib',
         'reorder',
         'sched',
Signature · 1 line
2.33.0
#317 lore raw

11/10/2021 09:33, Chengwen Feng:

Quoted 25 lines
doc/guides/prog_guide/dmadev.rst new file +22 -0
@@ -0,0 +1,60 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright 2021 HiSilicon Limited
+
+DMA Device Library
+==================
+
+The DMA library provides a DMA device framework for management and provisioning
+of hardware and software DMA poll mode drivers, defining generic API which
+support a number of different DMA operations.
+
+
+Design Principles
+-----------------
+
+The DMA framework provides a generic DMA device framework which supports both
+physical (hardware) and virtual (software) DMA devices, as well as a generic DMA
+API which allows DMA devices to be managed and configured, and supports DMA
+operations to be provisioned on DMA poll mode driver.
+
+.. _figure_dmadev:
+
+.. figure:: img/dmadev.*

doc/guides/prog_guide/dmadev.rst line 32

Quoted 10 lines
doc/guides/prog_guide/dmadev.rst new file +10 -0
+
+The above figure shows the model on which the DMA framework is built on:
+
+ * The DMA controller could have multiple hardware DMA channels (aka. hardware
+   DMA queues), each hardware DMA channel should be represented by a dmadev.
+ * The dmadev could create multiple virtual DMA channels, each virtual DMA
+   channel represents a different transfer context. The DMA operation request
+   must be submitted to the virtual DMA channel. e.g. Application could create
+   virtual DMA channel 0 for memory-to-memory transfer scenario, and create
+   virtual DMA channel 1 for memory-to-device transfer scenario.

When updating the doc, we would like to change a minimum of lines, so it's better to split lines logically: after a comma, a point, or before the next part of the sentence.
Do not hesitate to make short lines if needed.
Such change is quite fast to do, thanks.

[...]

doc/guides/prog_guide/dmadev.rst line 35

Quoted 3 lines
doc/guides/prog_guide/dmadev.rst new file +3 -0
+* **Introduced dmadev library with:**
+
+  * Device allocation functions.

You can drop this line, it is not a feature.

[...]

Quoted 97 lines
doc/guides/prog_guide/dmadev.rst new file +97 -0
+static int
+dma_dev_data_prepare(void)
+{
+	size_t size;
+
+	if (rte_dma_devices != NULL)
+		return 0;
+
+	size = dma_devices_max * sizeof(struct rte_dma_dev);
+	rte_dma_devices = malloc(size);
+	if (rte_dma_devices == NULL)
+		return -ENOMEM;
+	memset(rte_dma_devices, 0, size);
+
+	return 0;
+}
+
+static int
+dma_data_prepare(void)
+{
+	if (dma_devices_max == 0)
+		dma_devices_max = RTE_DMADEV_DEFAULT_MAX;
+	return dma_dev_data_prepare();
+}
+
+static struct rte_dma_dev *
+dma_allocate(const char *name, int numa_node, size_t private_data_size)
+{
+	struct rte_dma_dev *dev;
+	void *dev_private;
+	int16_t dev_id;
+	int ret;
+
+	ret = dma_data_prepare();
+	if (ret < 0) {
+		RTE_DMA_LOG(ERR, "Cannot initialize dmadevs data");
+		return NULL;
+	}
+
+	dev = dma_find_by_name(name);
+	if (dev != NULL) {
+		RTE_DMA_LOG(ERR, "DMA device already allocated");
+		return NULL;
+	}
+
+	dev_private = rte_zmalloc_socket(name, private_data_size,
+					 RTE_CACHE_LINE_SIZE, numa_node);
+	if (dev_private == NULL) {
+		RTE_DMA_LOG(ERR, "Cannot allocate private data");
+		return NULL;
+	}
+
+	dev_id = dma_find_free_id();
+	if (dev_id < 0) {
+		RTE_DMA_LOG(ERR, "Reached maximum number of DMA devices");
+		rte_free(dev_private);
+		return NULL;
+	}
+
+	dev = &rte_dma_devices[dev_id];
+	rte_strscpy(dev->dev_name, name, sizeof(dev->dev_name));
+	dev->dev_id = dev_id;
+	dev->numa_node = numa_node;
+	dev->dev_private = dev_private;
+
+	return dev;
+}
+
+static void
+dma_release(struct rte_dma_dev *dev)
+{
+	rte_free(dev->dev_private);
+	memset(dev, 0, sizeof(struct rte_dma_dev));
+}
+
+struct rte_dma_dev *
+rte_dma_pmd_allocate(const char *name, int numa_node, size_t private_data_size)
+{
+	struct rte_dma_dev *dev;
+
+	if (dma_check_name(name) != 0 || private_data_size == 0)
+		return NULL;
+
+	dev = dma_allocate(name, numa_node, private_data_size);
+	if (dev == NULL)
+		return NULL;
+
+	dev->state = RTE_DMA_DEV_REGISTERED;
+
+	return dev;
+}
+
+int
+rte_dma_pmd_release(const char *name)
+{
+	struct rte_dma_dev *dev;
+

doc/guides/prog_guide/dmadev.rst

Quoted 10 lines
doc/guides/prog_guide/dmadev.rst new file +10 -0
+	if (dma_check_name(name) != 0)
+		return -EINVAL;
+
+	dev = dma_find_by_name(name);
+	if (dev == NULL)
+		return -EINVAL;
+
+	dma_release(dev);
+	return 0;
+}

Trying to understand the logic of creation/destroy. skeldma_probe
\-> skeldma_create

    \-> rte_dma_pmd_allocate
        \-> dma_allocate
            \-> dma_data_prepare
                \-> dma_dev_data_prepare
skeldma_remove
\-> skeldma_destroy
    \-> rte_dma_pmd_release
        \-> dma_release
app
\-> rte_dma_close
    \-> skeldma_close
    \-> dma_release

My only concern is that the PMD remove does not call rte_dma_close. The PMD should check which dmadev is open for the rte_device to remove, and close the dmadev first.
This way, no need for the function rte_dma_pmd_release,
and no need to duplicate the release process in two paths.
By the way, the function vchan_release is called only in the close function, not in the "remove path".

#318 lore raw
Quoted 43 lines · Thomas Monjalon

11/10/2021 09:33, Chengwen Feng:

Quoted 25 lines
doc/guides/prog_guide/dmadev.rst new file +22 -0
@@ -0,0 +1,60 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright 2021 HiSilicon Limited
+
+DMA Device Library
+==================
+
+The DMA library provides a DMA device framework for management and provisioning
+of hardware and software DMA poll mode drivers, defining generic API which
+support a number of different DMA operations.
+
+
+Design Principles
+-----------------
+
+The DMA framework provides a generic DMA device framework which supports both
+physical (hardware) and virtual (software) DMA devices, as well as a generic DMA
+API which allows DMA devices to be managed and configured, and supports DMA
+operations to be provisioned on DMA poll mode driver.
+
+.. _figure_dmadev:
+
+.. figure:: img/dmadev.*

doc/guides/prog_guide/dmadev.rst line 32

Quoted 10 lines
doc/guides/prog_guide/dmadev.rst new file +10 -0
+
+The above figure shows the model on which the DMA framework is built on:
+
+ * The DMA controller could have multiple hardware DMA channels (aka. hardware
+   DMA queues), each hardware DMA channel should be represented by a dmadev.
+ * The dmadev could create multiple virtual DMA channels, each virtual DMA
+   channel represents a different transfer context. The DMA operation request
+   must be submitted to the virtual DMA channel. e.g. Application could create
+   virtual DMA channel 0 for memory-to-memory transfer scenario, and create
+   virtual DMA channel 1 for memory-to-device transfer scenario.

When updating the doc, we would like to change a minimum of lines, so it's better to split lines logically: after a comma, a point, or before the next part of the sentence.
Do not hesitate to make short lines if needed.
Such change is quite fast to do, thanks.

Quoted 6 lines

[...]

doc/guides/prog_guide/dmadev.rst

Quoted 3 lines
doc/guides/prog_guide/dmadev.rst new file +3 -0
+* **Introduced dmadev library with:**
+
+  * Device allocation functions.

You can drop this line, it is not a feature.

I'm going to try another description.

Quoted 113 lines

[...]

Quoted 97 lines
doc/guides/prog_guide/dmadev.rst new file +97 -0
+static int
+dma_dev_data_prepare(void)
+{
+	size_t size;
+
+	if (rte_dma_devices != NULL)
+		return 0;
+
+	size = dma_devices_max * sizeof(struct rte_dma_dev);
+	rte_dma_devices = malloc(size);
+	if (rte_dma_devices == NULL)
+		return -ENOMEM;
+	memset(rte_dma_devices, 0, size);
+
+	return 0;
+}
+
+static int
+dma_data_prepare(void)
+{
+	if (dma_devices_max == 0)
+		dma_devices_max = RTE_DMADEV_DEFAULT_MAX;
+	return dma_dev_data_prepare();
+}
+
+static struct rte_dma_dev *
+dma_allocate(const char *name, int numa_node, size_t private_data_size)
+{
+	struct rte_dma_dev *dev;
+	void *dev_private;
+	int16_t dev_id;
+	int ret;
+
+	ret = dma_data_prepare();
+	if (ret < 0) {
+		RTE_DMA_LOG(ERR, "Cannot initialize dmadevs data");
+		return NULL;
+	}
+
+	dev = dma_find_by_name(name);
+	if (dev != NULL) {
+		RTE_DMA_LOG(ERR, "DMA device already allocated");
+		return NULL;
+	}
+
+	dev_private = rte_zmalloc_socket(name, private_data_size,
+					 RTE_CACHE_LINE_SIZE, numa_node);
+	if (dev_private == NULL) {
+		RTE_DMA_LOG(ERR, "Cannot allocate private data");
+		return NULL;
+	}
+
+	dev_id = dma_find_free_id();
+	if (dev_id < 0) {
+		RTE_DMA_LOG(ERR, "Reached maximum number of DMA devices");
+		rte_free(dev_private);
+		return NULL;
+	}
+
+	dev = &rte_dma_devices[dev_id];
+	rte_strscpy(dev->dev_name, name, sizeof(dev->dev_name));
+	dev->dev_id = dev_id;
+	dev->numa_node = numa_node;
+	dev->dev_private = dev_private;
+
+	return dev;
+}
+
+static void
+dma_release(struct rte_dma_dev *dev)
+{
+	rte_free(dev->dev_private);
+	memset(dev, 0, sizeof(struct rte_dma_dev));
+}
+
+struct rte_dma_dev *
+rte_dma_pmd_allocate(const char *name, int numa_node, size_t private_data_size)
+{
+	struct rte_dma_dev *dev;
+
+	if (dma_check_name(name) != 0 || private_data_size == 0)
+		return NULL;
+
+	dev = dma_allocate(name, numa_node, private_data_size);
+	if (dev == NULL)
+		return NULL;
+
+	dev->state = RTE_DMA_DEV_REGISTERED;
+
+	return dev;
+}
+
+int
+rte_dma_pmd_release(const char *name)
+{
+	struct rte_dma_dev *dev;
+

doc/guides/prog_guide/dmadev.rst

Quoted 10 lines
doc/guides/prog_guide/dmadev.rst new file +10 -0
+	if (dma_check_name(name) != 0)
+		return -EINVAL;
+
+	dev = dma_find_by_name(name);
+	if (dev == NULL)
+		return -EINVAL;
+
+	dma_release(dev);
+	return 0;
+}

Trying to understand the logic of creation/destroy. skeldma_probe
\-> skeldma_create

Quoted 8 lines
    \-> rte_dma_pmd_allocate
        \-> dma_allocate
            \-> dma_data_prepare
                \-> dma_dev_data_prepare
skeldma_remove
\-> skeldma_destroy
    \-> rte_dma_pmd_release
        \-> dma_release

This patch only provide device allocate function, the 2st patch provide extra logic:

	diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
	index 42a4693bd9..a6a5680d2b 100644
	--- a/lib/dmadev/rte_dmadev.c
	+++ b/lib/dmadev/rte_dmadev.c
	@@ -201,6 +201,9 @@ rte_dma_pmd_release(const char *name)
	        if (dev == NULL)
        	        return -EINVAL;

	+       if (dev->state == RTE_DMA_DEV_READY)
	+               return rte_dma_close(dev->dev_id);
	+
        	dma_release(dev);
	        return 0;
	 }

So the skeldma remove will be:

skeldma_remove
\-> skeldma_destroy

     \-> rte_dma_pmd_release
         \-> rte_dma_close
             \-> dma_release
Quoted 6 lines

app
\-> rte_dma_close

    \-> skeldma_close
    \-> dma_release

My only concern is that the PMD remove does not call rte_dma_close.

If the device create success, it will call rte_dma_close in device remove phase.

Quoted 10 lines

The PMD should check which dmadev is open for the rte_device to remove, and close the dmadev first.
This way, no need for the function rte_dma_pmd_release,
and no need to duplicate the release process in two paths.
By the way, the function vchan_release is called only in the close function, not in the "remove path".

Quoted 2 lines

.

Thanks

#319 lore raw

13/10/2021 02:21, fengchengwen:

Quoted 8 lines · Thomas Monjalon
Quoted 7 lines

11/10/2021 09:33, Chengwen Feng:

Quoted 6 lines
lib/dmadev/rte_dmadev.c new file +6 -0
+static void
+dma_release(struct rte_dma_dev *dev)
+{
+	rte_free(dev->dev_private);
+	memset(dev, 0, sizeof(struct rte_dma_dev));
+}

[...]

Quoted 45 lines
Quoted 19 lines
Quoted 5 lines
lib/dmadev/rte_dmadev.c new file +5 -0
+int
+rte_dma_pmd_release(const char *name)
+{
+	struct rte_dma_dev *dev;
+

lib/dmadev/rte_dmadev.c line 206

Quoted 10 lines
lib/dmadev/rte_dmadev.c new file +10 -0
+	if (dma_check_name(name) != 0)
+		return -EINVAL;
+
+	dev = dma_find_by_name(name);
+	if (dev == NULL)
+		return -EINVAL;
+
+	dma_release(dev);
+	return 0;
+}

Trying to understand the logic of creation/destroy. skeldma_probe
\-> skeldma_create

Quoted 8 lines
    \-> rte_dma_pmd_allocate
        \-> dma_allocate
            \-> dma_data_prepare
                \-> dma_dev_data_prepare
skeldma_remove
\-> skeldma_destroy
    \-> rte_dma_pmd_release
        \-> dma_release

This patch only provide device allocate function, the 2st patch provide extra logic:

	diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
	index 42a4693bd9..a6a5680d2b 100644
	--- a/lib/dmadev/rte_dmadev.c
	+++ b/lib/dmadev/rte_dmadev.c
	@@ -201,6 +201,9 @@ rte_dma_pmd_release(const char *name)
	        if (dev == NULL)
        	        return -EINVAL;

	+       if (dev->state == RTE_DMA_DEV_READY)
	+               return rte_dma_close(dev->dev_id);
	+
        	dma_release(dev);
	        return 0;
	 }
Quoted 7 lines

So the skeldma remove will be:

skeldma_remove
\-> skeldma_destroy

     \-> rte_dma_pmd_release
         \-> rte_dma_close
             \-> dma_release

OK, in this case, no need to dma_release from rte_dma_pmd_release, because it is already called from rte_dma_close.

#320 lore raw

13/10/2021 09:41, Thomas Monjalon:

Quoted 57 lines

13/10/2021 02:21, fengchengwen:

Quoted 8 lines · Thomas Monjalon
Quoted 7 lines

11/10/2021 09:33, Chengwen Feng:

Quoted 6 lines
lib/dmadev/rte_dmadev.c new file +6 -0
+static void
+dma_release(struct rte_dma_dev *dev)
+{
+	rte_free(dev->dev_private);
+	memset(dev, 0, sizeof(struct rte_dma_dev));
+}

[...]

Quoted 47 lines
Quoted 19 lines
Quoted 5 lines
lib/dmadev/rte_dmadev.c new file +5 -0
+int
+rte_dma_pmd_release(const char *name)
+{
+	struct rte_dma_dev *dev;
+

lib/dmadev/rte_dmadev.c line 206

Quoted 10 lines
lib/dmadev/rte_dmadev.c new file +10 -0
+	if (dma_check_name(name) != 0)
+		return -EINVAL;
+
+	dev = dma_find_by_name(name);
+	if (dev == NULL)
+		return -EINVAL;
+
+	dma_release(dev);
+	return 0;
+}

Trying to understand the logic of creation/destroy. skeldma_probe
\-> skeldma_create

Quoted 8 lines
    \-> rte_dma_pmd_allocate
        \-> dma_allocate
            \-> dma_data_prepare
                \-> dma_dev_data_prepare
skeldma_remove
\-> skeldma_destroy
    \-> rte_dma_pmd_release
        \-> dma_release

This patch only provide device allocate function, the 2st patch provide extra logic:

	diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
	index 42a4693bd9..a6a5680d2b 100644
	--- a/lib/dmadev/rte_dmadev.c
	+++ b/lib/dmadev/rte_dmadev.c
	@@ -201,6 +201,9 @@ rte_dma_pmd_release(const char *name)
	        if (dev == NULL)
        	        return -EINVAL;

	+       if (dev->state == RTE_DMA_DEV_READY)
	+               return rte_dma_close(dev->dev_id);
	+
        	dma_release(dev);
	        return 0;
	 }

So the skeldma remove will be:

Quoted 8 lines
Quoted 5 lines

skeldma_remove
\-> skeldma_destroy

     \-> rte_dma_pmd_release
         \-> rte_dma_close
             \-> dma_release

OK, in this case, no need to dma_release from rte_dma_pmd_release, because it is already called from rte_dma_close.

Ping for reply please.

#321 lore raw
Quoted 64 lines · Thomas Monjalon

13/10/2021 09:41, Thomas Monjalon:

Quoted 63 lines

13/10/2021 02:21, fengchengwen:

Quoted 8 lines · Thomas Monjalon
Quoted 7 lines

11/10/2021 09:33, Chengwen Feng:

Quoted 6 lines
lib/dmadev/rte_dmadev.c new file +6 -0
+static void
+dma_release(struct rte_dma_dev *dev)
+{
+	rte_free(dev->dev_private);
+	memset(dev, 0, sizeof(struct rte_dma_dev));
+}

[...]

Quoted 52 lines
Quoted 19 lines
Quoted 5 lines
lib/dmadev/rte_dmadev.c new file +5 -0
+int
+rte_dma_pmd_release(const char *name)
+{
+	struct rte_dma_dev *dev;
+

lib/dmadev/rte_dmadev.c line 206

Quoted 10 lines
lib/dmadev/rte_dmadev.c new file +10 -0
+	if (dma_check_name(name) != 0)
+		return -EINVAL;
+
+	dev = dma_find_by_name(name);
+	if (dev == NULL)
+		return -EINVAL;
+
+	dma_release(dev);
+	return 0;
+}

Trying to understand the logic of creation/destroy. skeldma_probe
\-> skeldma_create

Quoted 8 lines
    \-> rte_dma_pmd_allocate
        \-> dma_allocate
            \-> dma_data_prepare
                \-> dma_dev_data_prepare
skeldma_remove
\-> skeldma_destroy
    \-> rte_dma_pmd_release
        \-> dma_release

This patch only provide device allocate function, the 2st patch provide extra logic:

	diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
	index 42a4693bd9..a6a5680d2b 100644
	--- a/lib/dmadev/rte_dmadev.c
	+++ b/lib/dmadev/rte_dmadev.c
	@@ -201,6 +201,9 @@ rte_dma_pmd_release(const char *name)
	        if (dev == NULL)
        	        return -EINVAL;

	+       if (dev->state == RTE_DMA_DEV_READY)
	+               return rte_dma_close(dev->dev_id);
	+
        	dma_release(dev);
	        return 0;
	 }

So the skeldma remove will be:

skeldma_remove
\-> skeldma_destroy

     \-> rte_dma_pmd_release
         \-> rte_dma_close
             \-> dma_release
Quoted 4 lines
Quoted 2 lines

OK, in this case, no need to dma_release from rte_dma_pmd_release, because it is already called from rte_dma_close.

Ping for reply please.

Sorry, I think the previous reply was enough, Let me explain:

The PMD use following logic create dmadev:
skeldma_probe

    \-> skeldma_create
      \-> rte_dma_pmd_allocate
        \-> dma_allocate
      \-> mark dmadev state to READY.

The PMD remove will be:
skeldma_remove
\-> skeldma_destroy

      \-> rte_dma_pmd_release
          \-> rte_dma_close
              \-> dma_release

The application close dmadev:
rte_dma_close
\-> dma_release

in the above case, the PMD remove and application close both call rte_dma_close, I think that's what you expect.

skeldma is simple, please let me give you a complicated example:
hisi_dma_probe

    \-> hisi_dma_create
      \-> rte_dma_pmd_allocate
        \-> dma_allocate
      \-> hisi_hw_init
        \-> if init fail, call rte_dma_pmd_release.
            \-> dma_release
        \-> if init OK, mark dmadev state to READY.

as you can see, if hisi_hw_init fail, it call rte_dma_pmd_release to release dmadev, it will direct call dma_release.
if hisi_hw_init success, it mean the hardware also OK, then mark dmadev state to READY. if PMD remove the dmadev it will call rte_dma_close because the dmadev's state is READY, and the application could also call rte_dma_close to destroy dmadev.

The rte_dma_pmd_release take two function:
1. if the dmadev's hardware part init fail, the PMD could use this function release the dmadev.
2. if the dmadev's hardware part init success, the PMD could use this function destroy the dmadev.

If we don't have the rte_dma_pmd_release function, we should export dma_release function which invoked when the hardware init fail.

And if we keep rte_dma_pmd_release, it correspond the rte_dma_pmd_allocate, the PMD just invoke rte_dma_pmd_release to handle above both cases (hardware part init fail when probe and remove phase).

Thanks.

Quoted 6 lines

.

#322 lore raw

15/10/2021 11:59, fengchengwen:

Quoted 65 lines · Thomas Monjalon
Quoted 64 lines

13/10/2021 09:41, Thomas Monjalon:

Quoted 63 lines

13/10/2021 02:21, fengchengwen:

Quoted 8 lines · Thomas Monjalon
Quoted 7 lines

11/10/2021 09:33, Chengwen Feng:

Quoted 6 lines
lib/dmadev/rte_dmadev.c new file +6 -0
+static void
+dma_release(struct rte_dma_dev *dev)
+{
+	rte_free(dev->dev_private);
+	memset(dev, 0, sizeof(struct rte_dma_dev));
+}

[...]

Quoted 52 lines
Quoted 19 lines
Quoted 5 lines
lib/dmadev/rte_dmadev.c new file +5 -0
+int
+rte_dma_pmd_release(const char *name)
+{
+	struct rte_dma_dev *dev;
+

lib/dmadev/rte_dmadev.c line 206

Quoted 10 lines
lib/dmadev/rte_dmadev.c new file +10 -0
+	if (dma_check_name(name) != 0)
+		return -EINVAL;
+
+	dev = dma_find_by_name(name);
+	if (dev == NULL)
+		return -EINVAL;
+
+	dma_release(dev);
+	return 0;
+}

Trying to understand the logic of creation/destroy. skeldma_probe
\-> skeldma_create

Quoted 8 lines
    \-> rte_dma_pmd_allocate
        \-> dma_allocate
            \-> dma_data_prepare
                \-> dma_dev_data_prepare
skeldma_remove
\-> skeldma_destroy
    \-> rte_dma_pmd_release
        \-> dma_release

This patch only provide device allocate function, the 2st patch provide extra logic:

	diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
	index 42a4693bd9..a6a5680d2b 100644
	--- a/lib/dmadev/rte_dmadev.c
	+++ b/lib/dmadev/rte_dmadev.c
	@@ -201,6 +201,9 @@ rte_dma_pmd_release(const char *name)
	        if (dev == NULL)
        	        return -EINVAL;

	+       if (dev->state == RTE_DMA_DEV_READY)
	+               return rte_dma_close(dev->dev_id);
	+
        	dma_release(dev);
	        return 0;
	 }

So the skeldma remove will be:

skeldma_remove
\-> skeldma_destroy

     \-> rte_dma_pmd_release
         \-> rte_dma_close
             \-> dma_release
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

OK, in this case, no need to dma_release from rte_dma_pmd_release, because it is already called from rte_dma_close.

Ping for reply please.

Sorry, I think the previous reply was enough, Let me explain:

No, if previous answer was enough, I would not add a new comment. Please read again:
"
no need to dma_release from rte_dma_pmd_release, because it is already called from rte_dma_close
"

Quoted 46 lines

The PMD use following logic create dmadev:
skeldma_probe

    \-> skeldma_create
      \-> rte_dma_pmd_allocate
        \-> dma_allocate
      \-> mark dmadev state to READY.

The PMD remove will be:
skeldma_remove
\-> skeldma_destroy

      \-> rte_dma_pmd_release
          \-> rte_dma_close
              \-> dma_release

The application close dmadev:
rte_dma_close
\-> dma_release

in the above case, the PMD remove and application close both call rte_dma_close, I think that's what you expect.

skeldma is simple, please let me give you a complicated example:
hisi_dma_probe

    \-> hisi_dma_create
      \-> rte_dma_pmd_allocate
        \-> dma_allocate
      \-> hisi_hw_init
        \-> if init fail, call rte_dma_pmd_release.
            \-> dma_release
        \-> if init OK, mark dmadev state to READY.

as you can see, if hisi_hw_init fail, it call rte_dma_pmd_release to release dmadev, it will direct call dma_release.
if hisi_hw_init success, it mean the hardware also OK, then mark dmadev state to READY. if PMD remove the dmadev it will call rte_dma_close because the dmadev's state is READY, and the application could also call rte_dma_close to destroy dmadev.

The rte_dma_pmd_release take two function:
1. if the dmadev's hardware part init fail, the PMD could use this function release the dmadev.
2. if the dmadev's hardware part init success, the PMD could use this function destroy the dmadev.

Quoted 6 lines

If we don't have the rte_dma_pmd_release function, we should export dma_release function which invoked when the hardware init fail.

And if we keep rte_dma_pmd_release, it correspond the rte_dma_pmd_allocate, the PMD just invoke rte_dma_pmd_release to handle above both cases (hardware part init fail when probe and remove phase).

You are justifying the existence of the functions, OK, but I am just discussing one call of the function which is useless.

Anyway, now I am in the process of merging v26, so I will send a fix.

#323 lore raw

PATCH v25 2/6 dmadev: add control plane API support

In reply to Chengwen Feng

This patch add control plane API for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  38 ++
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.c                | 360 +++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 464 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_pmd.h            |  61 ++++
 lib/dmadev/version.map                 |   9 +
 6 files changed, 933 insertions(+)
doc/guides/prog_guide/dmadev.rst +38 -0
index 90bda28f33..5c70ad3d6a 100644
@@ -58,3 +58,41 @@ identifiers:
 
 - A device name used to designate the DMA device in console messages, for
   administration or debugging purposes.
+
+
+Device Configuration
+~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_configure API is used to configure a DMA device.
+
+.. code-block:: c
+
+   int rte_dma_configure(int16_t dev_id,
+                         const struct rte_dma_conf *dev_conf);
+
+The ``rte_dma_conf`` structure is used to pass the configuration parameters
+for the DMA device.
+
+
+Configuration of Virtual DMA Channels
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_vchan_setup API is used to configure a virtual DMA channel.
+
+.. code-block:: c
+
+   int rte_dma_vchan_setup(int16_t dev_id, uint16_t vchan,
+                           const struct rte_dma_vchan_conf *conf);
+
+The ``rte_dma_vchan_conf`` structure is used to pass the configuration
+parameters for the virtual DMA channel.
+
+
+Device Features and Capabilities
+--------------------------------
+
+DMA devices may support different feature sets. The ``rte_dma_info_get`` API
+can be used to get the device info and supported features.
+
+Silent mode is a special device capability which does not require the
+application to invoke dequeue APIs.
doc/guides/rel_notes/release_21_11.rst +1 -0
index 929c0d6113..f935a3f395 100644
@@ -144,6 +144,7 @@ New Features
 * **Introduced dmadev library with:**
 
   * Device allocation functions.
+  * Control plane API.
 
 
 Removed Items
lib/dmadev/rte_dmadev.c +360 -0

369 lines not on this page · raw message

lib/dmadev/rte_dmadev.h +464 -0

476 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h +61 -0

74 lines not on this page · raw message

lib/dmadev/version.map +9 -0
index f8a0076468..e925dfcd6d 100644
@@ -1,10 +1,19 @@
 EXPERIMENTAL {
 	global:
 
+	rte_dma_close;
+	rte_dma_configure;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
+	rte_dma_dump;
 	rte_dma_get_dev_id_by_name;
+	rte_dma_info_get;
 	rte_dma_is_valid;
+	rte_dma_start;
+	rte_dma_stats_get;
+	rte_dma_stats_reset;
+	rte_dma_stop;
+	rte_dma_vchan_setup;
 
 	local: *;
 };
Signature · 1 line
2.33.0
#324 lore raw

Re: [dpdk-dev] [PATCH v25 2/6] dmadev: add control plane API support

Quoted 16 lines · Chengwen Feng

This patch add control plane API for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  38 ++
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.c                | 360 +++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 464 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_pmd.h            |  61 ++++
 lib/dmadev/version.map                 |   9 +
 6 files changed, 933 insertions(+)

<snip>

Quoted 18 lines
lib/dmadev/rte_dmadev.h +18 -0
+/**
+ * A structure used to retrieve the information of a DMA device.
+ *
+ * @see rte_dma_info_get
+ */
+struct rte_dma_info {
+	/** Device capabilities (RTE_DMA_CAPA_*). */
+	uint64_t dev_capa;
+	/** Maximum number of virtual DMA channels supported. */
+	uint16_t max_vchans;
+	/** Maximum allowed number of virtual DMA channel descriptors. */
+	uint16_t max_desc;
+	/** Minimum allowed number of virtual DMA channel descriptors. */
+	uint16_t min_desc;
+	/** Maximum number of source or destination scatter-gather entry
+	 * supported.
+	 * If the device does not support COPY_SG capability, this value can be
+	 * zero.

lib/dmadev/rte_dmadev.h bool rte_dma_is_valid(int16_t dev_id); · line 214

Quoted 10 lines
lib/dmadev/rte_dmadev.h +10 -0
+	 * If the device supports COPY_SG capability, then rte_dma_copy_sg()
+	 * parameter nb_src/nb_dst should not exceed this value.
+	 */
+	uint16_t max_sges;
+	/** NUMA node connection, -1 if unknown. */
+	int16_t numa_node;
+	/** Number of virtual DMA channel configured. */
+	uint16_t nb_vchans;
+};
+

Since we really don't want apps to have to access the rte_dma_devices array, I think we should add "const char *name" to the info struct. Apps may find it useful for debugging or logging, and certainly the unit tests will use it.

/Bruce

#325 lore raw

Re: [dpdk-dev] [PATCH v25 2/6] dmadev: add control plane API support

Quoted 44 lines · Bruce Richardson
Quoted 16 lines · Chengwen Feng

This patch add control plane API for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  38 ++
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.c                | 360 +++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 464 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_pmd.h            |  61 ++++
 lib/dmadev/version.map                 |   9 +
 6 files changed, 933 insertions(+)

<snip>

Quoted 24 lines
lib/dmadev/rte_dmadev.h +24 -0
+/**
+ * A structure used to retrieve the information of a DMA device.
+ *
+ * @see rte_dma_info_get
+ */
+struct rte_dma_info {
+	/** Device capabilities (RTE_DMA_CAPA_*). */
+	uint64_t dev_capa;
+	/** Maximum number of virtual DMA channels supported. */
+	uint16_t max_vchans;
+	/** Maximum allowed number of virtual DMA channel descriptors. */
+	uint16_t max_desc;
+	/** Minimum allowed number of virtual DMA channel descriptors. */
+	uint16_t min_desc;
+	/** Maximum number of source or destination scatter-gather entry
+	 * supported.
+	 * If the device does not support COPY_SG capability, this value can be
+	 * zero.
+	 * If the device supports COPY_SG capability, then rte_dma_copy_sg()
+	 * parameter nb_src/nb_dst should not exceed this value.
+	 */
+	uint16_t max_sges;
+	/** NUMA node connection, -1 if unknown. */
+	int16_t numa_node;
Quoted 8 lines

lib/dmadev/rte_dmadev.h bool rte_dma_is_valid(int16_t dev_id); · line 214

Quoted 4 lines
lib/dmadev/rte_dmadev.h +4 -0
+	/** Number of virtual DMA channel configured. */
+	uint16_t nb_vchans;
+};
+

Since we really don't want apps to have to access the rte_dma_devices array, I think we should add "const char *name" to the info struct. Apps may find it useful for debugging or logging, and certainly the unit tests will use it.

Agree

Quoted 5 lines

/Bruce

.

Thanks

#326 lore raw

Re: [dpdk-dev] [PATCH v25 2/6] dmadev: add control plane API support

In reply to Chengwen Feng

11/10/2021 09:33, Chengwen Feng:

Quoted 16 lines
doc/guides/prog_guide/dmadev.rst +16 -0
+Device Configuration
+~~~~~~~~~~~~~~~~~~~~
+
+The rte_dma_configure API is used to configure a DMA device.
+
+.. code-block:: c
+
+   int rte_dma_configure(int16_t dev_id,
+                         const struct rte_dma_conf *dev_conf);
+
+The ``rte_dma_conf`` structure is used to pass the configuration parameters
+for the DMA device.
+
+
+Configuration of Virtual DMA Channels
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

doc/guides/prog_guide/dmadev.rst identifiers: · line 88

Quoted 10 lines
doc/guides/prog_guide/dmadev.rst +10 -0
+
+The rte_dma_vchan_setup API is used to configure a virtual DMA channel.
+
+.. code-block:: c
+
+   int rte_dma_vchan_setup(int16_t dev_id, uint16_t vchan,
+                           const struct rte_dma_vchan_conf *conf);
+
+The ``rte_dma_vchan_conf`` structure is used to pass the configuration
+parameters for the virtual DMA channel.

I think those 2 above sections don't bring anything in the guide. Functions are described in Doxygen comments, it is enough.

doc/guides/rel_notes/release_21_11.rst New Features · line 147

Quoted 7 lines
doc/guides/rel_notes/release_21_11.rst +1 -0
@@ -144,6 +144,7 @@ New Features
 * **Introduced dmadev library with:**
 
   * Device allocation functions.
+  * Control plane API.

This is not a feature, you can drop from the release notes.

[...]

doc/guides/rel_notes/release_21_11.rst New Features · line 150

Quoted 3 lines
doc/guides/rel_notes/release_21_11.rst +3 -0
+/**@{@name DMA capability
+ * @see struct rte_dma_info::dev_capa
+ */

Thank you for using Doxygen grouping.

doc/guides/rel_notes/release_21_11.rst New Features

Quoted 2 lines
doc/guides/rel_notes/release_21_11.rst +2 -0
+#define RTE_DMA_CAPA_MEM_TO_MEM		RTE_BIT64(0)
+/**< Support memory-to-memory transfer */

Would it be possible to put the comment before the flag?

Quoted 29 lines
doc/guides/rel_notes/release_21_11.rst +29 -0
+#define RTE_DMA_CAPA_MEM_TO_DEV		RTE_BIT64(1)
+/**< Support memory-to-device transfer. */
+#define RTE_DMA_CAPA_DEV_TO_MEM		RTE_BIT64(2)
+/**< Support device-to-memory transfer. */
+#define RTE_DMA_CAPA_DEV_TO_DEV		RTE_BIT64(3)
+/**< Support device-to-device transfer. */
+#define RTE_DMA_CAPA_SVA		RTE_BIT64(4)
+/**< Support SVA which could use VA as DMA address.
+ * If device support SVA then application could pass any VA address like memory
+ * from rte_malloc(), rte_memzone(), malloc, stack memory.
+ * If device don't support SVA, then application should pass IOVA address which
+ * from rte_malloc(), rte_memzone().
+ */
+#define RTE_DMA_CAPA_SILENT		RTE_BIT64(5)
+/**< Support work in silent mode.
+ * In this mode, application don't required to invoke rte_dma_completed*()
+ * API.
+ * @see struct rte_dma_conf::silent_mode
+ */
+#define RTE_DMA_CAPA_OPS_COPY		RTE_BIT64(32)
+/**< Support copy operation.
+ * This capability start with index of 32, so that it could leave gap between
+ * normal capability and ops capability.
+ */
+#define RTE_DMA_CAPA_OPS_COPY_SG	RTE_BIT64(33)
+/**< Support scatter-gather list copy operation. */
+#define RTE_DMA_CAPA_OPS_FILL		RTE_BIT64(34)
+/**< Support fill operation. */
+/**@}*/
#327 lore raw

PATCH v25 3/6 dmadev: add data plane API support

In reply to Chengwen Feng

This patch add data plane API for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  22 ++
 doc/guides/rel_notes/release_21_11.rst |   2 +-
 lib/dmadev/meson.build                 |   1 +
 lib/dmadev/rte_dmadev.c                | 112 ++++++
 lib/dmadev/rte_dmadev.h                | 451 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |  81 +++++
 lib/dmadev/rte_dmadev_pmd.h            |   2 +
 lib/dmadev/version.map                 |   7 +
 8 files changed, 677 insertions(+), 1 deletion(-)
 create mode 100644 lib/dmadev/rte_dmadev_core.h
doc/guides/prog_guide/dmadev.rst +22 -0
index 5c70ad3d6a..2e2a4bb62a 100644
@@ -96,3 +96,25 @@ can be used to get the device info and supported features.
 
 Silent mode is a special device capability which does not require the
 application to invoke dequeue APIs.
+
+
+Enqueue / Dequeue APIs
+~~~~~~~~~~~~~~~~~~~~~~
+
+Enqueue APIs such as ``rte_dma_copy`` and ``rte_dma_fill`` can be used to
+enqueue operations to hardware. If an enqueue is successful, a ``ring_idx`` is
+returned. This ``ring_idx`` can be used by applications to track per operation
+metadata in an application-defined circular ring.
+
+The ``rte_dma_submit`` API is used to issue doorbell to hardware.
+Alternatively the ``RTE_DMA_OP_FLAG_SUBMIT`` flag can be passed to the enqueue
+APIs to also issue the doorbell to hardware.
+
+There are two dequeue APIs ``rte_dma_completed`` and
+``rte_dma_completed_status``, these are used to obtain the results of the
+enqueue requests. ``rte_dma_completed`` will return the number of successfully
+completed operations. ``rte_dma_completed_status`` will return the number of
+completed operations along with the status of each operation (filled into the
+``status`` array passed by user). These two APIs can also return the last
+completed operation's ``ring_idx`` which could help user track operations within
+their own application-defined rings.
doc/guides/rel_notes/release_21_11.rst +1 -1
index f935a3f395..d1d7abf694 100644
@@ -144,7 +144,7 @@ New Features
 * **Introduced dmadev library with:**
 
   * Device allocation functions.
-  * Control plane API.
+  * Control and data plane API.
 
 
 Removed Items
lib/dmadev/meson.build +1 -0
index f8d54c6e74..d2fc85e8c7 100644
@@ -3,4 +3,5 @@
 
 sources = files('rte_dmadev.c')
 headers = files('rte_dmadev.h')
+indirect_headers += files('rte_dmadev_core.h')
 driver_sdk_headers += files('rte_dmadev_pmd.h')
lib/dmadev/rte_dmadev.c +112 -0

147 lines not on this page · raw message

lib/dmadev/rte_dmadev.h +451 -0

463 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h new file +81 -0

81 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h +2 -0
index 5fcf0f60b8..d6d2161306 100644
@@ -100,6 +100,8 @@ struct rte_dma_dev {
 	void *dev_private; /**< PMD-specific private data. */
 	/** Device info which supplied during device initialization. */
 	struct rte_device *device;
+	/**< Fast-path functions and related data. */
+	struct rte_dma_fp_object *fp_obj;
 	/** Functions implemented by PMD. */
 	const struct rte_dma_dev_ops *dev_ops;
 	struct rte_dma_conf dev_conf; /**< DMA device configuration. */
lib/dmadev/version.map +7 -0
index e925dfcd6d..e17207b212 100644
@@ -2,10 +2,15 @@ EXPERIMENTAL {
 	global:
 
 	rte_dma_close;
+	rte_dma_completed;
+	rte_dma_completed_status;
 	rte_dma_configure;
+	rte_dma_copy;
+	rte_dma_copy_sg;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
 	rte_dma_dump;
+	rte_dma_fill;
 	rte_dma_get_dev_id_by_name;
 	rte_dma_info_get;
 	rte_dma_is_valid;
@@ -13,6 +18,7 @@ EXPERIMENTAL {
 	rte_dma_stats_get;
 	rte_dma_stats_reset;
 	rte_dma_stop;
+	rte_dma_submit;
 	rte_dma_vchan_setup;
 
 	local: *;
@@ -22,6 +28,7 @@ INTERNAL {
 	global:
 
 	rte_dma_devices;
+	rte_dma_fp_objs;
 	rte_dma_pmd_allocate;
 	rte_dma_pmd_release;
 
Signature · 1 line
2.33.0
#328 lore raw

PATCH v25 4/6 dmadev: add multi-process support

In reply to Chengwen Feng

This patch add multi-process support for dmadev.

---
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.c                | 181 ++++++++++++++++++++-----
 lib/dmadev/rte_dmadev_core.h           |   2 +-
 lib/dmadev/rte_dmadev_pmd.h            |  29 +++-
 4 files changed, 168 insertions(+), 45 deletions(-)
doc/guides/rel_notes/release_21_11.rst +1 -0
index d1d7abf694..af32fce1ed 100644
@@ -145,6 +145,7 @@ New Features
 
   * Device allocation functions.
   * Control and data plane API.
+  * Multi-process support.
 
 
 Removed Items
lib/dmadev/rte_dmadev.c +144 -37

327 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h +1 -1
index 6947091924..ad4035e6dc 100644
@@ -60,7 +60,7 @@ typedef uint16_t (*rte_dma_completed_status_t)(void *dev_private,
  */
 struct rte_dma_fp_object {
 	/** PMD-specific private data. The driver should copy
-	 * rte_dma_dev.dev_private to this field during initialization.
+	 * rte_dma_dev.data->dev_private to this field during initialization.
 	 */
 	void *dev_private;
 	rte_dma_copy_t             copy;
lib/dmadev/rte_dmadev_pmd.h +22 -7
index d6d2161306..23b07a4e1c 100644
@@ -76,6 +76,27 @@ struct rte_dma_dev_ops {
 
 	rte_dma_dump_t             dev_dump;
 };
+
+/**
+ * @internal
+ * The data part, with no function pointers, associated with each DMA device.
+ *
+ * This structure is safe to place in shared memory to be common among different
+ * processes in a multi-process configuration.
+ *
+ * @see struct rte_dma_dev::data
+ */
+struct rte_dma_dev_data {
+	char dev_name[RTE_DEV_NAME_MAX_LEN]; /**< Unique identifier name */
+	int16_t dev_id; /**< Device [external] identifier. */
+	int16_t numa_node; /**< Local NUMA memory ID. -1 if unknown. */
+	void *dev_private; /**< PMD-specific private data. */
+	struct rte_dma_conf dev_conf; /**< DMA device configuration. */
+	__extension__
+	uint8_t dev_started : 1; /**< Device state: STARTED(1)/STOPPED(0). */
+	uint64_t reserved[2]; /**< Reserved for future fields */
+} __rte_cache_aligned;
+
 /**
  * Possible states of a DMA device.
  *
@@ -94,20 +115,14 @@ enum rte_dma_dev_state {
  * The generic data structure associated with each DMA device.
  */
 struct rte_dma_dev {
-	char dev_name[RTE_DEV_NAME_MAX_LEN]; /**< Unique identifier name */
-	int16_t dev_id; /**< Device [external] identifier. */
-	int16_t numa_node; /**< Local NUMA memory ID. -1 if unknown. */
-	void *dev_private; /**< PMD-specific private data. */
 	/** Device info which supplied during device initialization. */
 	struct rte_device *device;
+	struct rte_dma_dev_data *data; /**< Pointer to shared device data. */
 	/**< Fast-path functions and related data. */
 	struct rte_dma_fp_object *fp_obj;
 	/** Functions implemented by PMD. */
 	const struct rte_dma_dev_ops *dev_ops;
-	struct rte_dma_conf dev_conf; /**< DMA device configuration. */
 	enum rte_dma_dev_state state; /**< Flag indicating the device state. */
-	__extension__
-	uint8_t dev_started : 1; /**< Device state: STARTED(1)/STOPPED(0). */
 	uint64_t reserved[2]; /**< Reserved for future fields. */
 } __rte_cache_aligned;
 
Signature · 1 line
2.33.0
#329 lore raw

PATCH v25 5/6 dma/skeleton: introduce skeleton dmadev driver

In reply to Chengwen Feng

Skeleton dmadevice driver, on the lines of rawdev skeleton, is for showcasing of the dmadev library.

Design of skeleton involves a virtual device which is plugged into VDEV bus on initialization.

Also, enable compilation of dmadev skeleton drivers.

---
 MAINTAINERS                            |   1 +
 drivers/dma/meson.build                |   4 +-
 drivers/dma/skeleton/meson.build       |   7 +
 drivers/dma/skeleton/skeleton_dmadev.c | 571 +++++++++++++++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |  61 +++
 drivers/dma/skeleton/version.map       |   3 +
 6 files changed, 646 insertions(+), 1 deletion(-)
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
MAINTAINERS +1 -0
index 119cfaa04e..ec887ac49f 100644
@@ -457,6 +457,7 @@ F: doc/guides/regexdevs/features/default.ini
 DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
+F: drivers/dma/skeleton/
 F: doc/guides/prog_guide/dmadev.rst
 
 Eventdev API
drivers/dma/meson.build +3 -1
index a24c56d8ff..d9c7ede32f 100644
@@ -1,4 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2021 HiSilicon Limited
 
-drivers = []
+drivers = [
+        'skeleton',
+]
drivers/dma/skeleton/meson.build new file +7 -0
new file mode 100644
index 0000000000..8871b80956
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited
+
+deps += ['dmadev', 'kvargs', 'ring', 'bus_vdev']
+sources = files(
+        'skeleton_dmadev.c',
+)
drivers/dma/skeleton/skeleton_dmadev.c new file +571 -0

571 lines not on this page · raw message

drivers/dma/skeleton/skeleton_dmadev.h new file +61 -0

61 lines not on this page · raw message

drivers/dma/skeleton/version.map new file +3 -0
new file mode 100644
index 0000000000..c2e0723b4c
@@ -0,0 +1,3 @@
+DPDK_22 {
+	local: *;
+};
Signature · 1 line
2.33.0
#330 lore raw

PATCH v25 6/6 app/test: add dmadev API test

In reply to Chengwen Feng

This patch add dmadev API test which based on 'dma_skeleton' vdev. The test cases could be executed using 'dmadev_autotest' command in test framework.

---
 MAINTAINERS                |   1 +
 app/test/meson.build       |   4 +
 app/test/test_dmadev.c     |  41 +++
 app/test/test_dmadev_api.c | 574 +++++++++++++++++++++++++++++++++++++
 app/test/test_dmadev_api.h |   5 +
 5 files changed, 625 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
 create mode 100644 app/test/test_dmadev_api.h
MAINTAINERS +1 -0
index ec887ac49f..d329873465 100644
@@ -458,6 +458,7 @@ DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
 F: drivers/dma/skeleton/
+F: app/test/test_dmadev*
 F: doc/guides/prog_guide/dmadev.rst
 
 Eventdev API
app/test/meson.build +4 -0
index f144d8b8ed..a16374b7a1 100644
@@ -44,6 +44,8 @@ test_sources = files(
         'test_debug.c',
         'test_distributor.c',
         'test_distributor_perf.c',
+        'test_dmadev.c',
+        'test_dmadev_api.c',
         'test_eal_flags.c',
         'test_eal_fs.c',
         'test_efd.c',
@@ -163,6 +165,7 @@ test_deps = [
         'cmdline',
         'cryptodev',
         'distributor',
+        'dmadev',
         'efd',
         'ethdev',
         'eventdev',
@@ -334,6 +337,7 @@ driver_test_names = [
         'cryptodev_sw_mvsam_autotest',
         'cryptodev_sw_snow3g_autotest',
         'cryptodev_sw_zuc_autotest',
+        'dmadev_autotest',
         'eventdev_selftest_octeontx',
         'eventdev_selftest_sw',
         'rawdev_autotest',
app/test/test_dmadev.c new file +41 -0

41 lines not on this page · raw message

app/test/test_dmadev_api.c new file +574 -0

574 lines not on this page · raw message

app/test/test_dmadev_api.h new file +5 -0
new file mode 100644
index 0000000000..33fbc5bd41
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ */
+
+int test_dma_api(uint16_t dev_id);
Signature · 1 line
2.33.0
#331 lore raw

v26 cover support dmadev

In reply to Chengwen Feng

This patch set contains six patch for new add dmadev.

Chengwen Feng (6):
dmadev: introduce DMA device library
dmadev: add control plane API support
dmadev: add data plane API support
dmadev: add multi-process support
dma/skeleton: introduce skeleton dmadev driver
app/test: add dmadev API test

---
v26:
* use prefix comment in rte_dmadev.h group macro.
* rte_dma_info add filed dev_name.
* rte_dma_fp_object fix align 128.
* delete some useless content of dma.rst.
v25:
* fix undefined reference to rte_dma_fp_objs with some compiler suite.
* use rte_dma_dev hold pointer to rte_dma_fp_objs to avoid primary and secondary disuniformity.
v24:
* use rte_dma_fp_object to hide implementation details.
* support group doxygen for RTE_DMA_CAPA_* and RTE_DMA_OP_*.
* adjusted the naming of some functions.
* fix typo.
v23:
* split multi-process support from 1st patch.
* fix some static check warning.
* fix skeleton cpu thread zero_req_count flip bug.
* add test_dmadev_api.h.
* add the description of modifying the dmadev state when init OK. v22:
* function prefix change from rte_dmadev_* to rte_dma_*.
* change to prefix comment in most scenarios.
* dmadev dev_id use int16_t type.
* fix typo.
* organize patchsets in incremental mode.

 MAINTAINERS                            |    7 +
 app/test/meson.build                   |    4 +
 app/test/test_dmadev.c                 |   41 +
 app/test/test_dmadev_api.c             |  574 +++++++++++++
 app/test/test_dmadev_api.h             |    5 +
 doc/api/doxy-api-index.md              |    1 +
 doc/api/doxy-api.conf.in               |    1 +
 doc/guides/dmadevs/index.rst           |   12 +
 doc/guides/index.rst                   |    1 +
 doc/guides/prog_guide/dmadev.rst       |   90 ++
 doc/guides/prog_guide/img/dmadev.svg   |  283 +++++++
 doc/guides/prog_guide/index.rst        |    1 +
 doc/guides/rel_notes/release_21_11.rst |    8 +
 drivers/dma/meson.build                |    6 +
 drivers/dma/skeleton/meson.build       |    7 +
 drivers/dma/skeleton/skeleton_dmadev.c |  571 +++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |   61 ++
 drivers/dma/skeleton/version.map       |    3 +
 drivers/meson.build                    |    1 +
 lib/dmadev/meson.build                 |    7 +
 lib/dmadev/rte_dmadev.c                |  826 +++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 1049 ++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h           |   76 ++
 lib/dmadev/rte_dmadev_pmd.h            |  168 ++++
 lib/dmadev/version.map                 |   36 +
 lib/meson.build                        |    1 +
 26 files changed, 3840 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
 create mode 100644 app/test/test_dmadev_api.h
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_core.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
Signature · 1 line
2.33.0
#332 lore raw

PATCH v26 1/6 dmadev: introduce DMA device library

The 'dmadev' is a generic type of DMA device.

This patch introduce the 'dmadev' device allocation functions.

The infrastructure is prepared to welcome drivers in drivers/dma/

---
 MAINTAINERS                            |   5 +
 doc/api/doxy-api-index.md              |   1 +
 doc/api/doxy-api.conf.in               |   1 +
 doc/guides/dmadevs/index.rst           |  12 ++
 doc/guides/index.rst                   |   1 +
 doc/guides/prog_guide/dmadev.rst       |  58 +++++
 doc/guides/prog_guide/img/dmadev.svg   | 283 +++++++++++++++++++++++++
 doc/guides/prog_guide/index.rst        |   1 +
 doc/guides/rel_notes/release_21_11.rst |   5 +
 drivers/dma/meson.build                |   4 +
 drivers/meson.build                    |   1 +
 lib/dmadev/meson.build                 |   6 +
 lib/dmadev/rte_dmadev.c                | 246 +++++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 133 ++++++++++++
 lib/dmadev/rte_dmadev_pmd.h            |  90 ++++++++
 lib/dmadev/version.map                 |  20 ++
 lib/meson.build                        |   1 +
 17 files changed, 868 insertions(+)
 create mode 100644 doc/guides/dmadevs/index.rst
 create mode 100644 doc/guides/prog_guide/dmadev.rst
 create mode 100644 doc/guides/prog_guide/img/dmadev.svg
 create mode 100644 drivers/dma/meson.build
 create mode 100644 lib/dmadev/meson.build
 create mode 100644 lib/dmadev/rte_dmadev.c
 create mode 100644 lib/dmadev/rte_dmadev.h
 create mode 100644 lib/dmadev/rte_dmadev_pmd.h
 create mode 100644 lib/dmadev/version.map
MAINTAINERS +5 -0
index 278e5b3226..119cfaa04e 100644
@@ -454,6 +454,11 @@ F: app/test-regex/
 F: doc/guides/prog_guide/regexdev.rst
 F: doc/guides/regexdevs/features/default.ini
 
+DMA device API - EXPERIMENTAL
+M: Chengwen Feng <[email protected]>
+F: lib/dmadev/
+F: doc/guides/prog_guide/dmadev.rst
+
 Eventdev API
 M: Jerin Jacob <[email protected]>
 T: git://dpdk.org/next/dpdk-next-eventdev
doc/api/doxy-api-index.md +1 -0
index 1992107a03..2939050431 100644
@@ -21,6 +21,7 @@ The public API headers are grouped by topics:
   [compressdev]        (@ref rte_compressdev.h),
   [compress]           (@ref rte_comp.h),
   [regexdev]           (@ref rte_regexdev.h),
+  [dmadev]             (@ref rte_dmadev.h),
   [eventdev]           (@ref rte_eventdev.h),
   [event_eth_rx_adapter]   (@ref rte_event_eth_rx_adapter.h),
   [event_eth_tx_adapter]   (@ref rte_event_eth_tx_adapter.h),
doc/api/doxy-api.conf.in +1 -0
index 325a0195c6..109ec1f682 100644
@@ -35,6 +35,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/compressdev \
                           @TOPDIR@/lib/cryptodev \
                           @TOPDIR@/lib/distributor \
+                          @TOPDIR@/lib/dmadev \
                           @TOPDIR@/lib/efd \
                           @TOPDIR@/lib/ethdev \
                           @TOPDIR@/lib/eventdev \
doc/guides/dmadevs/index.rst new file +12 -0
new file mode 100644
index 0000000000..0bce29d766
@@ -0,0 +1,12 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright 2021 HiSilicon Limited
+
+DMA Device Drivers
+==================
+
+The following are a list of DMA device drivers, which can be used from
+an application through DMA API.
+
+.. toctree::
+   :maxdepth: 2
+   :numbered:
doc/guides/index.rst +1 -0
index 857f0363d3..919825992e 100644
@@ -21,6 +21,7 @@ DPDK documentation
    compressdevs/index
    vdpadevs/index
    regexdevs/index
+   dmadevs/index
    eventdevs/index
    rawdevs/index
    mempool/index
doc/guides/prog_guide/dmadev.rst new file +58 -0

58 lines not on this page · raw message

doc/guides/prog_guide/img/dmadev.svg new file +283 -0

283 lines not on this page · raw message

doc/guides/prog_guide/index.rst +1 -0
index 2dce507f46..89af28dacb 100644
@@ -27,6 +27,7 @@ Programmer's Guide
     cryptodev_lib
     compressdev
     regexdev
+    dmadev
     rte_security
     rawdev
     link_bonding_poll_mode_drv_lib
doc/guides/rel_notes/release_21_11.rst +5 -0
index d5c762df62..1c7c1d6eab 100644
@@ -76,6 +76,11 @@ New Features
   * ``RTE_ETH_RX_METADATA_USER_MARK``
   * ``RTE_ETH_RX_METADATA_TUNNEL_ID``
 
+* **Added dmadev library.**
+
+  * Added a DMA device framework for management and provision of
+    hardware and software DMA devices.
+
 * **Updated af_packet ethdev driver.**
 
   * Default VLAN strip behavior was changed. VLAN tag won't be stripped
drivers/dma/meson.build new file +4 -0
new file mode 100644
index 0000000000..a24c56d8ff
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2021 HiSilicon Limited
+
+drivers = []
drivers/meson.build +1 -0
index 3d08540581..b7d680868a 100644
@@ -18,6 +18,7 @@ subdirs = [
         'vdpa',           # depends on common, bus and mempool.
         'event',          # depends on common, bus, mempool and net.
         'baseband',       # depends on common and bus.
+        'dma',            # depends on common and bus.
 ]
 
 if meson.is_cross_build()
lib/dmadev/meson.build new file +6 -0
new file mode 100644
index 0000000000..f8d54c6e74
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited.
+
+sources = files('rte_dmadev.c')
+headers = files('rte_dmadev.h')
+driver_sdk_headers += files('rte_dmadev_pmd.h')
lib/dmadev/rte_dmadev.c new file +246 -0

246 lines not on this page · raw message

lib/dmadev/rte_dmadev.h new file +133 -0

133 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h new file +90 -0

90 lines not on this page · raw message

lib/dmadev/version.map new file +20 -0
new file mode 100644
index 0000000000..f8a0076468
@@ -0,0 +1,20 @@
+EXPERIMENTAL {
+	global:
+
+	rte_dma_count_avail;
+	rte_dma_dev_max;
+	rte_dma_get_dev_id_by_name;
+	rte_dma_is_valid;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	rte_dma_devices;
+	rte_dma_pmd_allocate;
+	rte_dma_pmd_release;
+
+	local: *;
+};
lib/meson.build +1 -0
index b2ba7258d8..3b8b099820 100644
@@ -45,6 +45,7 @@ libraries = [
         'pdump',
         'rawdev',
         'regexdev',
+        'dmadev',
         'rib',
         'reorder',
         'sched',
Signature · 1 line
2.33.0
#333 lore raw

PATCH v26 2/6 dmadev: add control plane API support

In reply to Chengwen Feng

This patch add control plane API for dmadev.

---
 doc/guides/prog_guide/dmadev.rst       |  10 +
 doc/guides/rel_notes/release_21_11.rst |   2 +
 lib/dmadev/rte_dmadev.c                | 361 +++++++++++++++++++
 lib/dmadev/rte_dmadev.h                | 465 +++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_pmd.h            |  61 ++++
 lib/dmadev/version.map                 |   9 +
 6 files changed, 908 insertions(+)
doc/guides/prog_guide/dmadev.rst +10 -0
index 46b85ce217..b1b5bff639 100644
@@ -56,3 +56,13 @@ identifiers:
 
 - A device name used to designate the DMA device in console messages, for
   administration or debugging purposes.
+
+
+Device Features and Capabilities
+--------------------------------
+
+DMA devices may support different feature sets. The ``rte_dma_info_get`` API
+can be used to get the device info and supported features.
+
+Silent mode is a special device capability which does not require the
+application to invoke dequeue APIs.
doc/guides/rel_notes/release_21_11.rst +2 -0
index 1c7c1d6eab..2953bc3e97 100644
@@ -80,6 +80,8 @@ New Features
 
   * Added a DMA device framework for management and provision of
     hardware and software DMA devices.
+  * Added generic API which support a number of different DMA
+    operations.
 
 * **Updated af_packet ethdev driver.**
 
lib/dmadev/rte_dmadev.c +361 -0

370 lines not on this page · raw message

lib/dmadev/rte_dmadev.h +465 -0

477 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h +61 -0

74 lines not on this page · raw message

lib/dmadev/version.map +9 -0
index f8a0076468..e925dfcd6d 100644
@@ -1,10 +1,19 @@
 EXPERIMENTAL {
 	global:
 
+	rte_dma_close;
+	rte_dma_configure;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
+	rte_dma_dump;
 	rte_dma_get_dev_id_by_name;
+	rte_dma_info_get;
 	rte_dma_is_valid;
+	rte_dma_start;
+	rte_dma_stats_get;
+	rte_dma_stats_reset;
+	rte_dma_stop;
+	rte_dma_vchan_setup;
 
 	local: *;
 };
Signature · 1 line
2.33.0
#334 lore raw

PATCH v26 3/6 dmadev: add data plane API support

In reply to Chengwen Feng

This patch add data plane API for dmadev.

---
 doc/guides/prog_guide/dmadev.rst |  22 ++
 lib/dmadev/meson.build           |   1 +
 lib/dmadev/rte_dmadev.c          | 112 ++++++++
 lib/dmadev/rte_dmadev.h          | 451 +++++++++++++++++++++++++++++++
 lib/dmadev/rte_dmadev_core.h     |  76 ++++++
 lib/dmadev/rte_dmadev_pmd.h      |   2 +
 lib/dmadev/version.map           |   7 +
 7 files changed, 671 insertions(+)
 create mode 100644 lib/dmadev/rte_dmadev_core.h
doc/guides/prog_guide/dmadev.rst +22 -0
index b1b5bff639..32f7147862 100644
@@ -66,3 +66,25 @@ can be used to get the device info and supported features.
 
 Silent mode is a special device capability which does not require the
 application to invoke dequeue APIs.
+
+
+Enqueue / Dequeue APIs
+~~~~~~~~~~~~~~~~~~~~~~
+
+Enqueue APIs such as ``rte_dma_copy`` and ``rte_dma_fill`` can be used to
+enqueue operations to hardware. If an enqueue is successful, a ``ring_idx`` is
+returned. This ``ring_idx`` can be used by applications to track per operation
+metadata in an application-defined circular ring.
+
+The ``rte_dma_submit`` API is used to issue doorbell to hardware.
+Alternatively the ``RTE_DMA_OP_FLAG_SUBMIT`` flag can be passed to the enqueue
+APIs to also issue the doorbell to hardware.
+
+There are two dequeue APIs ``rte_dma_completed`` and
+``rte_dma_completed_status``, these are used to obtain the results of the
+enqueue requests. ``rte_dma_completed`` will return the number of successfully
+completed operations. ``rte_dma_completed_status`` will return the number of
+completed operations along with the status of each operation (filled into the
+``status`` array passed by user). These two APIs can also return the last
+completed operation's ``ring_idx`` which could help user track operations within
+their own application-defined rings.
lib/dmadev/meson.build +1 -0
index f8d54c6e74..d2fc85e8c7 100644
@@ -3,4 +3,5 @@
 
 sources = files('rte_dmadev.c')
 headers = files('rte_dmadev.h')
+indirect_headers += files('rte_dmadev_core.h')
 driver_sdk_headers += files('rte_dmadev_pmd.h')
lib/dmadev/rte_dmadev.c +112 -0

147 lines not on this page · raw message

lib/dmadev/rte_dmadev.h +451 -0

463 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h new file +76 -0

76 lines not on this page · raw message

lib/dmadev/rte_dmadev_pmd.h +2 -0
index 5fcf0f60b8..d6d2161306 100644
@@ -100,6 +100,8 @@ struct rte_dma_dev {
 	void *dev_private; /**< PMD-specific private data. */
 	/** Device info which supplied during device initialization. */
 	struct rte_device *device;
+	/**< Fast-path functions and related data. */
+	struct rte_dma_fp_object *fp_obj;
 	/** Functions implemented by PMD. */
 	const struct rte_dma_dev_ops *dev_ops;
 	struct rte_dma_conf dev_conf; /**< DMA device configuration. */
lib/dmadev/version.map +7 -0
index e925dfcd6d..e17207b212 100644
@@ -2,10 +2,15 @@ EXPERIMENTAL {
 	global:
 
 	rte_dma_close;
+	rte_dma_completed;
+	rte_dma_completed_status;
 	rte_dma_configure;
+	rte_dma_copy;
+	rte_dma_copy_sg;
 	rte_dma_count_avail;
 	rte_dma_dev_max;
 	rte_dma_dump;
+	rte_dma_fill;
 	rte_dma_get_dev_id_by_name;
 	rte_dma_info_get;
 	rte_dma_is_valid;
@@ -13,6 +18,7 @@ EXPERIMENTAL {
 	rte_dma_stats_get;
 	rte_dma_stats_reset;
 	rte_dma_stop;
+	rte_dma_submit;
 	rte_dma_vchan_setup;
 
 	local: *;
@@ -22,6 +28,7 @@ INTERNAL {
 	global:
 
 	rte_dma_devices;
+	rte_dma_fp_objs;
 	rte_dma_pmd_allocate;
 	rte_dma_pmd_release;
 
Signature · 1 line
2.33.0
#335 lore raw

PATCH v26 4/6 dmadev: add multi-process support

In reply to Chengwen Feng

This patch add multi-process support for dmadev.

---
 doc/guides/rel_notes/release_21_11.rst |   1 +
 lib/dmadev/rte_dmadev.c                | 183 ++++++++++++++++++++-----
 lib/dmadev/rte_dmadev_core.h           |   2 +-
 lib/dmadev/rte_dmadev_pmd.h            |  29 +++-
 4 files changed, 169 insertions(+), 46 deletions(-)
doc/guides/rel_notes/release_21_11.rst +1 -0
index 2953bc3e97..888f123237 100644
@@ -82,6 +82,7 @@ New Features
     hardware and software DMA devices.
   * Added generic API which support a number of different DMA
     operations.
+  * Added multi-process support.
 
 * **Updated af_packet ethdev driver.**
 
lib/dmadev/rte_dmadev.c +145 -38

330 lines not on this page · raw message

lib/dmadev/rte_dmadev_core.h +1 -1
index a6946052db..236d9d38e5 100644
@@ -60,7 +60,7 @@ typedef uint16_t (*rte_dma_completed_status_t)(void *dev_private,
  */
 struct rte_dma_fp_object {
 	/** PMD-specific private data. The driver should copy
-	 * rte_dma_dev.dev_private to this field during initialization.
+	 * rte_dma_dev.data->dev_private to this field during initialization.
 	 */
 	void *dev_private;
 	rte_dma_copy_t             copy;
lib/dmadev/rte_dmadev_pmd.h +22 -7
index d6d2161306..23b07a4e1c 100644
@@ -76,6 +76,27 @@ struct rte_dma_dev_ops {
 
 	rte_dma_dump_t             dev_dump;
 };
+
+/**
+ * @internal
+ * The data part, with no function pointers, associated with each DMA device.
+ *
+ * This structure is safe to place in shared memory to be common among different
+ * processes in a multi-process configuration.
+ *
+ * @see struct rte_dma_dev::data
+ */
+struct rte_dma_dev_data {
+	char dev_name[RTE_DEV_NAME_MAX_LEN]; /**< Unique identifier name */
+	int16_t dev_id; /**< Device [external] identifier. */
+	int16_t numa_node; /**< Local NUMA memory ID. -1 if unknown. */
+	void *dev_private; /**< PMD-specific private data. */
+	struct rte_dma_conf dev_conf; /**< DMA device configuration. */
+	__extension__
+	uint8_t dev_started : 1; /**< Device state: STARTED(1)/STOPPED(0). */
+	uint64_t reserved[2]; /**< Reserved for future fields */
+} __rte_cache_aligned;
+
 /**
  * Possible states of a DMA device.
  *
@@ -94,20 +115,14 @@ enum rte_dma_dev_state {
  * The generic data structure associated with each DMA device.
  */
 struct rte_dma_dev {
-	char dev_name[RTE_DEV_NAME_MAX_LEN]; /**< Unique identifier name */
-	int16_t dev_id; /**< Device [external] identifier. */
-	int16_t numa_node; /**< Local NUMA memory ID. -1 if unknown. */
-	void *dev_private; /**< PMD-specific private data. */
 	/** Device info which supplied during device initialization. */
 	struct rte_device *device;
+	struct rte_dma_dev_data *data; /**< Pointer to shared device data. */
 	/**< Fast-path functions and related data. */
 	struct rte_dma_fp_object *fp_obj;
 	/** Functions implemented by PMD. */
 	const struct rte_dma_dev_ops *dev_ops;
-	struct rte_dma_conf dev_conf; /**< DMA device configuration. */
 	enum rte_dma_dev_state state; /**< Flag indicating the device state. */
-	__extension__
-	uint8_t dev_started : 1; /**< Device state: STARTED(1)/STOPPED(0). */
 	uint64_t reserved[2]; /**< Reserved for future fields. */
 } __rte_cache_aligned;
 
Signature · 1 line
2.33.0
#336 lore raw

PATCH v26 5/6 dma/skeleton: introduce skeleton dmadev driver

In reply to Chengwen Feng

Skeleton dmadevice driver, on the lines of rawdev skeleton, is for showcasing of the dmadev library.

Design of skeleton involves a virtual device which is plugged into VDEV bus on initialization.

Also, enable compilation of dmadev skeleton drivers.

---
 MAINTAINERS                            |   1 +
 drivers/dma/meson.build                |   4 +-
 drivers/dma/skeleton/meson.build       |   7 +
 drivers/dma/skeleton/skeleton_dmadev.c | 571 +++++++++++++++++++++++++
 drivers/dma/skeleton/skeleton_dmadev.h |  61 +++
 drivers/dma/skeleton/version.map       |   3 +
 6 files changed, 646 insertions(+), 1 deletion(-)
 create mode 100644 drivers/dma/skeleton/meson.build
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.c
 create mode 100644 drivers/dma/skeleton/skeleton_dmadev.h
 create mode 100644 drivers/dma/skeleton/version.map
MAINTAINERS +1 -0
index 119cfaa04e..ec887ac49f 100644
@@ -457,6 +457,7 @@ F: doc/guides/regexdevs/features/default.ini
 DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
+F: drivers/dma/skeleton/
 F: doc/guides/prog_guide/dmadev.rst
 
 Eventdev API
drivers/dma/meson.build +3 -1
index a24c56d8ff..d9c7ede32f 100644
@@ -1,4 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2021 HiSilicon Limited
 
-drivers = []
+drivers = [
+        'skeleton',
+]
drivers/dma/skeleton/meson.build new file +7 -0
new file mode 100644
index 0000000000..8871b80956
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited
+
+deps += ['dmadev', 'kvargs', 'ring', 'bus_vdev']
+sources = files(
+        'skeleton_dmadev.c',
+)
drivers/dma/skeleton/skeleton_dmadev.c new file +571 -0

571 lines not on this page · raw message

drivers/dma/skeleton/skeleton_dmadev.h new file +61 -0

61 lines not on this page · raw message

drivers/dma/skeleton/version.map new file +3 -0
new file mode 100644
index 0000000000..c2e0723b4c
@@ -0,0 +1,3 @@
+DPDK_22 {
+	local: *;
+};
Signature · 1 line
2.33.0
#337 lore raw

PATCH v26 6/6 app/test: add dmadev API test

In reply to Chengwen Feng

This patch add dmadev API test which based on 'dma_skeleton' vdev. The test cases could be executed using 'dmadev_autotest' command in test framework.

---
 MAINTAINERS                |   1 +
 app/test/meson.build       |   4 +
 app/test/test_dmadev.c     |  41 +++
 app/test/test_dmadev_api.c | 574 +++++++++++++++++++++++++++++++++++++
 app/test/test_dmadev_api.h |   5 +
 5 files changed, 625 insertions(+)
 create mode 100644 app/test/test_dmadev.c
 create mode 100644 app/test/test_dmadev_api.c
 create mode 100644 app/test/test_dmadev_api.h
MAINTAINERS +1 -0
index ec887ac49f..d329873465 100644
@@ -458,6 +458,7 @@ DMA device API - EXPERIMENTAL
 M: Chengwen Feng <[email protected]>
 F: lib/dmadev/
 F: drivers/dma/skeleton/
+F: app/test/test_dmadev*
 F: doc/guides/prog_guide/dmadev.rst
 
 Eventdev API
app/test/meson.build +4 -0
index f144d8b8ed..a16374b7a1 100644
@@ -44,6 +44,8 @@ test_sources = files(
         'test_debug.c',
         'test_distributor.c',
         'test_distributor_perf.c',
+        'test_dmadev.c',
+        'test_dmadev_api.c',
         'test_eal_flags.c',
         'test_eal_fs.c',
         'test_efd.c',
@@ -163,6 +165,7 @@ test_deps = [
         'cmdline',
         'cryptodev',
         'distributor',
+        'dmadev',
         'efd',
         'ethdev',
         'eventdev',
@@ -334,6 +337,7 @@ driver_test_names = [
         'cryptodev_sw_mvsam_autotest',
         'cryptodev_sw_snow3g_autotest',
         'cryptodev_sw_zuc_autotest',
+        'dmadev_autotest',
         'eventdev_selftest_octeontx',
         'eventdev_selftest_sw',
         'rawdev_autotest',
app/test/test_dmadev.c new file +41 -0

41 lines not on this page · raw message

app/test/test_dmadev_api.c new file +574 -0

574 lines not on this page · raw message

app/test/test_dmadev_api.h new file +5 -0
new file mode 100644
index 0000000000..33fbc5bd41
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ */
+
+int test_dma_api(uint16_t dev_id);
Signature · 1 line
2.33.0
#338 lore raw

Re: [dpdk-dev] [PATCH v26 0/6] support dmadev

In reply to Chengwen Feng

13/10/2021 14:24, Chengwen Feng:

Quoted 9 lines

This patch set contains six patch for new add dmadev.

Chengwen Feng (6):
dmadev: introduce DMA device library
dmadev: add control plane API support
dmadev: add data plane API support
dmadev: add multi-process support
dma/skeleton: introduce skeleton dmadev driver
app/test: add dmadev API test

Applied, thanks for the big work.