Skip to contentSkip to conversation

Add DLB2 PMD

PATCH Superseded Timothy McDaniel 10 participants 366 messages Sep 11, 2020 – Nov 2, 2020 Arm/Amlogic Meson SoC support Page 8 of 8

lore
Share & tools
lore
Reviewed-by on v1 19/22
#352 lore raw

Re: [dpdk-dev] [PATCH 19/22] event/dlb2: add PMD self-tests

In reply to Timothy McDaniel
Quoted 28 lines

-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
To: Jerin Jacob <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>
Subject: [PATCH 19/22] event/dlb2: add PMD self-tests

Add a variety of self-tests for both ldb and directed ports/queues, as well as configure, start, stop, link, etc...

---
 app/test/test_eventdev.c           |    9 +
 drivers/event/dlb2/dlb2.c          |    1 +
 drivers/event/dlb2/dlb2_selftest.c | 1570
++++++++++++++++++++++++++++++++++++
 drivers/event/dlb2/meson.build     |    3 +-
 4 files changed, 1582 insertions(+), 1 deletion(-)
 create mode 100644 drivers/event/dlb2/dlb2_selftest.c
app/test/test_eventdev.c +0 -0
index 43ccb1c..b8d8df8 100644

app/test/test_eventdev.c test_eventdev_selftest_dpaa2(void) · line 1039

Quoted 10 lines
app/test/test_eventdev.c +7 -0
@@ -1030,6 +1030,13 @@ test_eventdev_selftest_dpaa2(void)
 	return test_eventdev_selftest_impl("event_dpaa2", "");
 }
 
+static int
+test_eventdev_selftest_dlb2(void)
+{
+	return test_eventdev_selftest_impl("dlb2_event", "");
+}
+
+

Nit: extra newline

Quoted 10 lines
app/test/test_eventdev.c +0 -0
 REGISTER_TEST_COMMAND(eventdev_common_autotest,
 test_eventdev_common);
 REGISTER_TEST_COMMAND(eventdev_selftest_sw,

test_eventdev_selftest_sw);
REGISTER_TEST_COMMAND(eventdev_selftest_octeontx,

app/test/test_eventdev.c +0 -0
@@ -1037,3 +1044,5 @@
 REGISTER_TEST_COMMAND(eventdev_selftest_octeontx,
 REGISTER_TEST_COMMAND(eventdev_selftest_octeontx2,
 		test_eventdev_selftest_octeontx2);
 REGISTER_TEST_COMMAND(eventdev_selftest_dpaa2,
Quoted 4 lines

test_eventdev_selftest_dpaa2);
+REGISTER_TEST_COMMAND(eventdev_selftest_dlb2,
test_eventdev_selftest_dlb2);
+

Nit: newline at the end of the file

Quoted 4 lines
drivers/event/dlb2/dlb2.c +0 -0
index 43b85d7..620a0a5 100644

[...]

drivers/event/dlb2/dlb2.c

Quoted 3 lines
drivers/event/dlb2/dlb2.c +3 -0
+/* destruction */
+static inline void
+cleanup(struct test *t __rte_unused)

Since 't' is unused, just drop the parameter.

With that and the whitespace issues fixed:

Thanks,
Gage

#353 lore raw

PATCH 20/22 event/dlb2: add queue and port release

In reply to Timothy McDaniel

These entry points are NO-OPS. DLB does not support reconfiguring individual queues or ports. The entire device must be reconfigured.

---
 drivers/event/dlb2/dlb2.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
drivers/event/dlb2/dlb2.c +25 -0
index 620a0a5..daf9b9f 100644
@@ -3756,6 +3756,29 @@ dlb2_eventdev_close(struct rte_eventdev *dev)
 }
 
 static void
+dlb2_eventdev_queue_release(struct rte_eventdev *dev, uint8_t id)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(id);
+
+	/* This function intentionally left blank. dlb2 does not support
+	 * reconfiguring individual queues or ports -- the entire device
+	 * must be reconfigured.
+	 */
+}
+
+static void
+dlb2_eventdev_port_release(void *port)
+{
+	RTE_SET_USED(port);
+
+	/* This function intentionally left blank. dlb2 does not support
+	 * reconfiguring individual queues or ports -- the entire device
+	 * must be reconfigured.
+	 */
+}
+
+static void
 dlb2_entry_points_init(struct rte_eventdev *dev)
 {
 	struct dlb2_eventdev *dlb2;
@@ -3769,8 +3792,10 @@ dlb2_entry_points_init(struct rte_eventdev *dev)
 		.dev_close        = dlb2_eventdev_close,
 		.queue_def_conf   = dlb2_eventdev_queue_default_conf_get,
 		.queue_setup      = dlb2_eventdev_queue_setup,
+		.queue_release    = dlb2_eventdev_queue_release,
 		.port_def_conf    = dlb2_eventdev_port_default_conf_get,
 		.port_setup       = dlb2_eventdev_port_setup,
+		.port_release     = dlb2_eventdev_port_release,
 		.port_link        = dlb2_eventdev_port_link,
 		.port_unlink      = dlb2_eventdev_port_unlink,
 		.port_unlinks_in_progress =
Signature · 1 line
2.6.4
#354 lore raw

Re: [dpdk-dev] [PATCH 20/22] event/dlb2: add queue and port release

drivers/event/dlb2/dlb2.c dlb2_eventdev_close(struct rte_eventdev *dev) · line 3767

Quoted 10 lines

static void

drivers/event/dlb2/dlb2.c +9 -0
+dlb2_eventdev_queue_release(struct rte_eventdev *dev, uint8_t id)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(id);
+
+	/* This function intentionally left blank. dlb2 does not support
+	 * reconfiguring individual queues or ports -- the entire device
+	 * must be reconfigured.
+	 */

I think the 2nd sentence in the comment need to be reconsidered (or just removed). The release callbacks are called (for every port and queue) when rte_event_dev_configure() is called a 2nd/3rd/etc. time to reconfigure the device, and the callbacks give the PMD the opportunity clean-up the individual port/queue. Implementing the logic can be useful for entire-device-reconfiguration.

drivers/event/dlb2/dlb2.c dlb2_eventdev_close(struct rte_eventdev *dev) · line 3778

Quoted 11 lines
drivers/event/dlb2/dlb2.c +11 -0
+}
+
+static void
+dlb2_eventdev_port_release(void *port)
+{
+	RTE_SET_USED(port);
+
+	/* This function intentionally left blank. dlb2 does not support
+	 * reconfiguring individual queues or ports -- the entire device
+	 * must be reconfigured.
+	 */

This might be the appropriate place to free each port's memzone (see my comments in patch 11).

Thanks,
Gage

#355 lore raw

PATCH 21/22 event/dlb2: add timeout ticks entry point

In reply to Timothy McDaniel

Adds the timeout ticks conversion function.

---
 drivers/event/dlb2/dlb2.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/event/dlb2/dlb2.c +14 -0
index daf9b9f..50fb4b9 100644
@@ -3778,6 +3778,19 @@ dlb2_eventdev_port_release(void *port)
 	 */
 }
 
+static int
+dlb2_eventdev_timeout_ticks(struct rte_eventdev *dev, uint64_t ns,
+			    uint64_t *timeout_ticks)
+{
+	RTE_SET_USED(dev);
+	uint64_t cycles_per_ns;
+
+	cycles_per_ns = rte_get_timer_hz() / 1E9;
+	*timeout_ticks = ns * cycles_per_ns;
+
+	return 0;
+}
+
 static void
 dlb2_entry_points_init(struct rte_eventdev *dev)
 {
@@ -3800,6 +3813,7 @@ dlb2_entry_points_init(struct rte_eventdev *dev)
 		.port_unlink      = dlb2_eventdev_port_unlink,
 		.port_unlinks_in_progress =
 				    dlb2_eventdev_port_unlinks_in_progress,
+		.timeout_ticks    = dlb2_eventdev_timeout_ticks,
 		.dump             = dlb2_eventdev_dump,
 		.xstats_get       = dlb2_eventdev_xstats_get,
 		.xstats_get_names = dlb2_eventdev_xstats_get_names,
Signature · 1 line
2.6.4
Reviewed-by on v1 21/22
#356 lore raw

Re: [dpdk-dev] [PATCH 21/22] event/dlb2: add timeout ticks entry point

Quoted 10 lines

-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 21/22] event/dlb2: add timeout ticks entry point

Quoted 3 lines

Adds the timeout ticks conversion function.

#357 lore raw

PATCH 22/22 doc: add new DLB2 eventdev driver to relnotes

In reply to Timothy McDaniel

Added announcement of availabililty for the new driver for Intel Dynamic Load Balancer V2.0 hardware.

---
 doc/guides/rel_notes/release_20_11.rst | 5 +++++
 1 file changed, 5 insertions(+)
doc/guides/rel_notes/release_20_11.rst +5 -0
index df227a1..f80df21 100644
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added a new driver for the Intel Dynamic Load Balancer v2.0 device.**
+
+  Added the new ``dlb2`` eventdev driver for the Intel DLB V2.0 device. See the
+  :doc:`../eventdevs/dlb2` eventdev guide for more details on this new driver.
+
 
 Removed Items
 -------------
Signature · 1 line
2.6.4
#358 lore raw

Re: [dpdk-dev] [PATCH 22/22] doc: add new DLB2 eventdev driver to relnotes

Quoted 12 lines

-----Original Message-----
From: McDaniel, Timothy <[email protected]>
Sent: Friday, September 11, 2020 3:26 PM
To: Mcnamara, John <[email protected]>; Kovacevic, Marko <[email protected]>
Cc: [email protected]; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; [email protected]
Subject: [PATCH 22/22] doc: add new DLB2 eventdev driver to relnotes

Quoted 2 lines

Added announcement of availabililty for the new driver for Intel Dynamic Load Balancer V2.0 hardware.

There's some inconsistency in the version numbering (2.0, V2.0, v2.0) in this series -- I believe 2.0 is the marketing-approved one.

Quoted 12 lines
---
 doc/guides/rel_notes/release_20_11.rst | 5 +++++
 1 file changed, 5 insertions(+)
doc/guides/rel_notes/release_20_11.rst +0 -0
index df227a1..f80df21 100644

doc/guides/rel_notes/release_20_11.rst New Features · line 61

Quoted 7 lines
doc/guides/rel_notes/release_20_11.rst +4 -0
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added a new driver for the Intel Dynamic Load Balancer v2.0 device.**
+
+  Added the new ``dlb2`` eventdev driver for the Intel DLB V2.0 device. See the
+  :doc:`../eventdevs/dlb2` eventdev guide for more details on this new driver.

Looks like the guide didn't make it into this series ;)

Please also add an entry MAINTAINERS entry for the PMD.

Thanks,
Gage

#359 lore raw
In reply to Timothy McDaniel
Quoted 33 lines · Timothy McDaniel

The following patch series adds support for a new eventdev PMD. The DLB2 PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2) hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.

The DLB2 hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB2 ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue. Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware.

While reviewing the code, please be aware that this PMD has full control over the DLB2 hardware. Intel will be extending the DLB2 PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so datapath operations occur purely in user-space. Note that the DLB2 hardware is a successor of the DLB hardware, and as such is structured similarly, both in terms of code layout and implementation.

Quoted 8 lines

The framework to support both the PF PMD and bifurcated PMD exists in this patchset, and is why the iface.[ch] layer is present.

Depends-on: patch-77466 ("eventdev: add PCI probe named convenience function")
Depends-on: series-12160 ("Eventdev ABI changes")
Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")

Timothy McDaniel (22):

There is two version of driver(DLB and DLB2)[1]
Both are needed?
Is two separate set of HW?

http://patches.dpdk.org/project/dpdk/list/?submitter=826

Quoted 82 lines

event/dlb2: add meson build infrastructure event/dlb2: add dynamic logging
event/dlb2: add private data structures and constants event/dlb2: add definitions shared with LKM or shared code event/dlb2: add inline functions
event/dlb2: add probe
event/dlb2: add xstats
event/dlb2: add infos get and configure event/dlb2: add queue and port default conf event/dlb2: add queue setup
event/dlb2: add port setup
event/dlb2: add port link
event/dlb2: add port unlink and port unlinks in progress event/dlb2: add eventdev start
event/dlb2: add enqueue and its burst variants event/dlb2: add dequeue and its burst variants event/dlb2: add eventdev stop and close event/dlb2: add PMD's token pop public interface event/dlb2: add PMD self-tests
event/dlb2: add queue and port release event/dlb2: add timeout ticks entry point
doc: add new DLB2 eventdev driver to relnotes

 app/test/test_eventdev.c                          |    9 +
 config/rte_config.h                               |    7 +
 doc/guides/rel_notes/release_20_11.rst            |    5 +
 drivers/event/dlb2/dlb2.c                         | 4046 ++++++++++++++
 drivers/event/dlb2/dlb2_iface.c                   |   88 +
 drivers/event/dlb2/dlb2_iface.h                   |   75 +
 drivers/event/dlb2/dlb2_inline_fns.h              |   85 +
 drivers/event/dlb2/dlb2_log.h                     |   25 +
 drivers/event/dlb2/dlb2_priv.h                    |  619 +++
 drivers/event/dlb2/dlb2_selftest.c                | 1570 ++++++
 drivers/event/dlb2/dlb2_user.h                    |  883 +++
 drivers/event/dlb2/dlb2_xstats.c                  | 1269 +++++
 drivers/event/dlb2/meson.build                    |   16 +
 drivers/event/dlb2/pf/base/dlb2_hw_types.h        |  367 ++
 drivers/event/dlb2/pf/base/dlb2_mbox.h            |  596 ++
 drivers/event/dlb2/pf/base/dlb2_osdep.h           |  248 +
 drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h    |  447 ++
 drivers/event/dlb2/pf/base/dlb2_osdep_list.h      |  131 +
 drivers/event/dlb2/pf/base/dlb2_osdep_types.h     |   31 +
 drivers/event/dlb2/pf/base/dlb2_regs.h            | 2527 +++++++++
 drivers/event/dlb2/pf/base/dlb2_resource.c        | 6023 +++++++++++++++++++++
 drivers/event/dlb2/pf/base/dlb2_resource.h        | 1913 +++++++
 drivers/event/dlb2/pf/dlb2_main.c                 |  692 +++
 drivers/event/dlb2/pf/dlb2_main.h                 |  107 +
 drivers/event/dlb2/pf/dlb2_pf.c                   |  734 +++
 drivers/event/dlb2/rte_pmd_dlb2.c                 |   39 +
 drivers/event/dlb2/rte_pmd_dlb2.h                 |   59 +
 drivers/event/dlb2/rte_pmd_dlb2_event_version.map |    9 +
 drivers/event/meson.build                         |    4 +
 29 files changed, 22624 insertions(+)
 create mode 100644 drivers/event/dlb2/dlb2.c
 create mode 100644 drivers/event/dlb2/dlb2_iface.c
 create mode 100644 drivers/event/dlb2/dlb2_iface.h
 create mode 100644 drivers/event/dlb2/dlb2_inline_fns.h
 create mode 100644 drivers/event/dlb2/dlb2_log.h
 create mode 100644 drivers/event/dlb2/dlb2_priv.h
 create mode 100644 drivers/event/dlb2/dlb2_selftest.c
 create mode 100644 drivers/event/dlb2/dlb2_user.h
 create mode 100644 drivers/event/dlb2/dlb2_xstats.c
 create mode 100644 drivers/event/dlb2/meson.build
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_hw_types.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_mbox.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_list.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_types.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_regs.h
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.c
 create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.h
 create mode 100644 drivers/event/dlb2/pf/dlb2_main.c
 create mode 100644 drivers/event/dlb2/pf/dlb2_main.h
 create mode 100644 drivers/event/dlb2/pf/dlb2_pf.c
 create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.c
 create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.h
 create mode 100644 drivers/event/dlb2/rte_pmd_dlb2_event_version.map

--
2.6.4

#360 lore raw
Quoted 54 lines

-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, September 21, 2020 12:12 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD

Quoted 33 lines · Timothy McDaniel

The following patch series adds support for a new eventdev PMD. The DLB2 PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2) hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.

The DLB2 hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB2 ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue. Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware.

While reviewing the code, please be aware that this PMD has full control over the DLB2 hardware. Intel will be extending the DLB2 PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so datapath operations occur purely in user-space. Note that the DLB2 hardware is a successor of the DLB hardware, and as such is structured similarly, both in terms of code layout and implementation.

Quoted 4 lines

The framework to support both the PF PMD and bifurcated PMD exists in this patchset, and is why the iface.[ch] layer is present.

Depends-on: patch-77466 ("eventdev: add PCI probe named convenience

function")

Quoted 3 lines

Depends-on: series-12160 ("Eventdev ABI changes")
Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")

Quoted 7 lines
Quoted 1 line

Timothy McDaniel (22):

There is two version of driver(DLB and DLB2)[1]
Both are needed?
Is two separate set of HW?

Yes, the two PMDs, DLB and DLB2 are both needed. They support two different hardware devices.

#361 lore raw
Quoted 20 lines · Timothy
Quoted 20 lines

-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, September 21, 2020 12:12 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD

Quoted 4 lines · Timothy McDaniel

Timothy McDaniel (22):

Quoted 6 lines
Quoted 4 lines

There is two version of driver(DLB and DLB2)[1]
Both are needed?
Is two separate set of HW?

Yes, the two PMDs, DLB and DLB2 are both needed. They support two different hardware devices.

There are plenty of build errors. Please see
http://mails.dpdk.org/archives/test-report/2020-September/152143.html

Also, I would like close the eventdev spec first before we merge this driver as the driver is depended on
http://patches.dpdk.org/patch/77465/

Quoted 2 lines
#362 lore raw
Quoted 40 lines

-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, September 29, 2020 1:41 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD

Quoted 8 lines · Timothy
Quoted 8 lines

-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, September 21, 2020 12:12 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>;

Eads,

Quoted 12 lines
Quoted 12 lines

Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD

Quoted 4 lines · Timothy McDaniel

Timothy McDaniel (22):

Quoted 6 lines
Quoted 4 lines

There is two version of driver(DLB and DLB2)[1] Both are needed?
Is two separate set of HW?

Yes, the two PMDs, DLB and DLB2 are both needed. They support two

Quoted 9 lines

different hardware devices.

There are plenty of build errors. Please see
http://mails.dpdk.org/archives/test-report/2020-September/152143.html

Also, I would like close the eventdev spec first before we merge this driver as the driver is depended on
http://patches.dpdk.org/patch/77465/

Thank you, Jerin.

I am working on addressing the build breaks and your other review comments. I hope to upload those changes tomorrow, and that will include the performance comparison that you requested.

#363 lore raw
In reply to Jerin Jacob
Quoted 39 lines

-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, September 29, 2020 1:41 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD

Quoted 8 lines · Timothy
Quoted 8 lines

-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Monday, September 21, 2020 12:12 PM
To: McDaniel, Timothy <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>;

Eads,

Quoted 17 lines
Quoted 16 lines

Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD

Quoted 4 lines · Timothy McDaniel

Timothy McDaniel (22):

There is two version of driver(DLB and DLB2)[1] Both are needed?
Is two separate set of HW?

Quoted 6 lines
Quoted 1 line

Yes, the two PMDs, DLB and DLB2 are both needed. They support two

different hardware devices.

There are plenty of build errors. Please see
http://mails.dpdk.org/archives/test-report/2020-September/152143.html

The build errors appear to be a result of the depends-on patches not having been applied. Also, note that there is a 2-way dependency on the ABI changes patch and the test/apps patch. They both depend on each other.

#364 lore raw
In reply to Timothy McDaniel
Quoted 33 lines · Timothy McDaniel

The following patch series adds support for a new eventdev PMD. The DLB2 PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2) hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.

The DLB2 hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB2 ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue. Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware.

While reviewing the code, please be aware that this PMD has full control over the DLB2 hardware. Intel will be extending the DLB2 PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so datapath operations occur purely in user-space. Note that the DLB2 hardware is a successor of the DLB hardware, and as such is structured similarly, both in terms of code layout and implementation.

Quoted 6 lines

The framework to support both the PF PMD and bifurcated PMD exists in this patchset, and is why the iface.[ch] layer is present.

Depends-on: patch-77466 ("eventdev: add PCI probe named convenience function")
Depends-on: series-12160 ("Eventdev ABI changes")
Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")

Cc Thomas and David

There is a duplicate for patch-77460 as
https://patches.dpdk.org/patch/76554/ from different authors, and both are open comments to be closed.
@McDaniel, Timothy and @Liang Ma , We need to have this eal patch on master before I merge this
patch series in eventdev-next. Please check.

Quoted 1 line
#365 lore raw
Quoted 56 lines

-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Tuesday, September 29, 2020 1:46 PM
To: McDaniel, Timothy <[email protected]>; Thomas Monjalon <[email protected]>; David Marchand <[email protected]>; Ma, Liang J <[email protected]>
Cc: dpdk-dev <[email protected]>; Carrillo, Erik G <[email protected]>; Eads, Gage <[email protected]>; Van Haaren, Harry <[email protected]>; Jerin Jacob <[email protected]>
Subject: Re: [dpdk-dev] [PATCH 00/22] Add DLB2 PMD

Quoted 33 lines · Timothy McDaniel

The following patch series adds support for a new eventdev PMD. The DLB2 PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2) hardware.
The DLB2 is a PCIe device that provides load-balanced, prioritized scheduling of core-to-core communication. The device consists of queues and arbiters that connect producer and consumer cores, and implements load-balanced queueing features including:
- Lock-free multi-producer/multi-consumer operation.
- Multiple priority levels for varying traffic types.
- 'Direct' traffic (i.e. multi-producer/single-consumer)
- Simple unordered load-balanced distribution.
- Atomic lock-free load balancing across multiple consumers.
- Queue element reordering feature allowing ordered load-balanced distribution.

The DLB2 hardware supports both load balanced and directed ports and queues. Unlike other eventdev devices already in the repo, not all DLB2 ports and queues are equally capable. In particular, directed ports are limited to a single link, and must be connected to a directed queue. Additionally, even though LDB ports may link multiple queues, the number of queues that may be linked is limited by hardware.

While reviewing the code, please be aware that this PMD has full control over the DLB2 hardware. Intel will be extending the DLB2 PMD in the future (not as part of this first series) with a mode that we refer to as the bifurcated PMD. The bifurcated PMD communicates with a kernel driver to configure the device, ports, and queues, and memory maps device MMIO so datapath operations occur purely in user-space. Note that the DLB2 hardware is a successor of the DLB hardware, and as such is structured similarly, both in terms of code layout and implementation.

Quoted 4 lines

The framework to support both the PF PMD and bifurcated PMD exists in this patchset, and is why the iface.[ch] layer is present.

Depends-on: patch-77466 ("eventdev: add PCI probe named convenience

function")

Quoted 2 lines

Depends-on: series-12160 ("Eventdev ABI changes")
Depends-on: patch-77460 ("eal: add umonitor umwait to x86 cpuflags")

Quoted 10 lines
Cc Thomas and David

There is a duplicate for patch-77460 as
https://patches.dpdk.org/patch/76554/ from different authors, and both are open comments to be closed.
@McDaniel, Timothy and @Liang Ma , We need to have this eal patch on master before I merge this
patch series in eventdev-next. Please check.

We have discussed this duplication with @Liang Ma, who split out just the
bit definitions into its own patch. DLB/DLB2 have been updated to use his new patch going forward, but will duplicate it here until it is merged.