Skip to contentSkip to conversation

eth_ring: perf test and usability improvements

PATCH Applied Bruce Richardson 4 participants 8 messages Sep 30, 2015 – Nov 3, 2015 Network drivers

lore
Share & tools
lore
#0 lore raw

cover eth_ring: perf test and usability improvements

This patchset makes it easier to create ring pmd instances from code, by providing a simple ring->ethdev wrapper function and also ensuring that any created rings are ready for use immediately, without having to call configure and rx/tx queue setup.

This set also contains a set of unit tests to compare the performance of basic ring operations against the same operations via the ring ethdev. This shows how the perf penalty can be significant for small bursts, but is much less so for larger bursts of 32 packets.

Bruce Richardson (4):
ring: enhance rte_eth_from_rings
rte_ring: store memzone pointer inside ring
ring: add rte_eth_from_ring function
test: Add perf test for ring pmd

 app/test/Makefile                         |   1 +
 app/test/test_pmd_ring_perf.c             | 188 ++++++++++++++++++++++++++++++
 drivers/net/ring/rte_eth_ring.c           |  55 ++++++++-
 drivers/net/ring/rte_eth_ring.h           |  14 +++
 drivers/net/ring/rte_eth_ring_version.map |   5 +
 lib/librte_ring/rte_ring.c                |   1 +
 lib/librte_ring/rte_ring.h                |   4 +
 7 files changed, 262 insertions(+), 6 deletions(-)
 create mode 100644 app/test/test_pmd_ring_perf.c
Signature · 1 line
2.4.3
#1 lore raw

PATCH 1/4 ring: enhance rte_eth_from_rings

The ring ethdev creation function creates an ethdev, but does not actually set it up for use. Even if it's just a single ring, the user still needs to create a mempool, call rte_eth_dev_configure, then call rx and tx setup functions before the ethdev can be used.

This patch changes things so that the ethdev is fully set up after the call to create the ethdev. The above-mentionned functions can still be called - as will be the case, for instance, if the NIC is created via commandline parameters - but they no longer are essential.

The function now also sets rte_errno appropriately on error, so the caller can get a better indication of why a call may have failed.

---
 drivers/net/ring/rte_eth_ring.c | 47 +++++++++++++++++++++++++++++++++++------
 1 file changed, 41 insertions(+), 6 deletions(-)
drivers/net/ring/rte_eth_ring.c +41 -6
index 0ba36d5..bfd6f4e 100644
@@ -39,6 +39,7 @@
 #include <rte_string_fns.h>
 #include <rte_dev.h>
 #include <rte_kvargs.h>
+#include <rte_errno.h>
 
 #define ETH_RING_NUMA_NODE_ACTION_ARG	"nodeaction"
 #define ETH_RING_ACTION_CREATE		"CREATE"
@@ -276,10 +277,18 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],
 	unsigned i;
 
 	/* do some parameter checking */
-	if (rx_queues == NULL && nb_rx_queues > 0)
+	if (rx_queues == NULL && nb_rx_queues > 0) {
+		rte_errno = EINVAL;
 		goto error;
-	if (tx_queues == NULL && nb_tx_queues > 0)
+	}
+	if (tx_queues == NULL && nb_tx_queues > 0) {
+		rte_errno = EINVAL;
+		goto error;
+	}
+	if (nb_rx_queues > RTE_PMD_RING_MAX_RX_RINGS) {
+		rte_errno = EINVAL;
 		goto error;
+	}
 
 	RTE_LOG(INFO, PMD, "Creating rings-backed ethdev on numa socket %u\n",
 			numa_node);
@@ -288,21 +297,43 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],
 	 * and internal (private) data
 	 */
 	data = rte_zmalloc_socket(name, sizeof(*data), 0, numa_node);
-	if (data == NULL)
+	if (data == NULL) {
+		rte_errno = ENOMEM;
 		goto error;
+	}
+
+	data->rx_queues = rte_zmalloc_socket(name, sizeof(void *) * nb_rx_queues,
+			0, numa_node);
+	if (data->rx_queues == NULL) {
+		rte_errno = ENOMEM;
+		goto error;
+	}
+
+	data->tx_queues = rte_zmalloc_socket(name, sizeof(void *) * nb_tx_queues,
+			0, numa_node);
+	if (data->tx_queues == NULL) {
+		rte_errno = ENOMEM;
+		goto error;
+	}
 
 	pci_dev = rte_zmalloc_socket(name, sizeof(*pci_dev), 0, numa_node);
-	if (pci_dev == NULL)
+	if (pci_dev == NULL) {
+		rte_errno = ENOMEM;
 		goto error;
+	}
 
 	internals = rte_zmalloc_socket(name, sizeof(*internals), 0, numa_node);
-	if (internals == NULL)
+	if (internals == NULL) {
+		rte_errno = ENOMEM;
 		goto error;
+	}
 
 	/* reserve an ethdev entry */
 	eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL);
-	if (eth_dev == NULL)
+	if (eth_dev == NULL) {
+		rte_errno = ENOSPC;
 		goto error;
+	}
 
 
 	/* now put it all together
@@ -318,9 +349,11 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],
 	internals->nb_tx_queues = nb_tx_queues;
 	for (i = 0; i < nb_rx_queues; i++) {
 		internals->rx_ring_queues[i].rng = rx_queues[i];
+		data->rx_queues[i] = &internals->rx_ring_queues[i];
 	}
 	for (i = 0; i < nb_tx_queues; i++) {
 		internals->tx_ring_queues[i].rng = tx_queues[i];
+		data->tx_queues[i] = &internals->tx_ring_queues[i];
 	}
 
 	rte_ring_pmd.pci_drv.name = ring_ethdev_driver_name;
@@ -350,6 +383,8 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],
 	return data->port_id;
 
 error:
+	rte_free(data->rx_queues);
+	rte_free(data->tx_queues);
 	rte_free(data);
 	rte_free(pci_dev);
 	rte_free(internals);
Signature · 1 line
2.4.3
#2 lore raw

PATCH 2/4 rte_ring: store memzone pointer inside ring

In reply to Bruce Richardson

Add a new field to the rte_ring structure to store the memzone pointer which contains the ring. For rings created using rte_ring_create(), the field will be set automatically.

This new field will allow users of the ring to query the numa node a ring is allocated on, or to get the physical address of the ring, if so needed.

The rte_ring structure will also maintain ABI compatibility, as the structure members, after the new one, are set to be cache line aligned, so leaving a space.

---
 lib/librte_ring/rte_ring.c | 1 +
 lib/librte_ring/rte_ring.h | 4 ++++
 2 files changed, 5 insertions(+)
lib/librte_ring/rte_ring.c +1 -0
index c9e59d4..4e78e14 100644
@@ -196,6 +196,7 @@ rte_ring_create(const char *name, unsigned count, int socket_id,
 		rte_ring_init(r, name, count, flags);
 
 		te->data = (void *) r;
+		r->memzone = mz;
 
 		TAILQ_INSERT_TAIL(ring_list, te, next);
 	} else {
lib/librte_ring/rte_ring.h +4 -0
index af68888..df45f3f 100644
@@ -134,6 +134,8 @@ struct rte_ring_debug_stats {
                                     *   if RTE_RING_PAUSE_REP not defined. */
 #endif
 
+struct rte_memzone; /* forward declaration, so as not to require memzone.h */
+
 /**
  * An RTE ring structure.
  *
@@ -147,6 +149,8 @@ struct rte_ring_debug_stats {
 struct rte_ring {
 	char name[RTE_RING_NAMESIZE];    /**< Name of the ring. */
 	int flags;                       /**< Flags supplied at creation. */
+	const struct rte_memzone *memzone;
+			/**< Memzone, if any, containing the rte_ring */
 
 	/** Ring producer status. */
 	struct prod {
Signature · 1 line
2.4.3
Acked-by on v1 2/4
#3 lore raw

Re: [PATCH 2/4] rte_ring: store memzone pointer inside ring

Hi Bruce,

Quoted 4 lines · Bruce Richardson

Add a new field to the rte_ring structure to store the memzone pointer which contains the ring. For rings created using rte_ring_create(), the field will be set automatically.

Quoted 8 lines

This new field will allow users of the ring to query the numa node a ring is allocated on, or to get the physical address of the ring, if so needed.

The rte_ring structure will also maintain ABI compatibility, as the structure members, after the new one, are set to be cache line aligned, so leaving a space.

#4 lore raw

PATCH 3/4 ring: add rte_eth_from_ring function

In reply to Bruce Richardson

Add a one-parameter function to take an existing rte_ring and wrap it as an ethdev, returning the port id of the new ethdev instance.

---
 drivers/net/ring/rte_eth_ring.c           |  8 ++++++++
 drivers/net/ring/rte_eth_ring.h           | 14 ++++++++++++++
 drivers/net/ring/rte_eth_ring_version.map |  5 +++++
 3 files changed, 27 insertions(+)
drivers/net/ring/rte_eth_ring.c +8 -0
index bfd6f4e..d851f9e 100644
@@ -36,6 +36,7 @@
 #include <rte_ethdev.h>
 #include <rte_malloc.h>
 #include <rte_memcpy.h>
+#include <rte_memzone.h>
 #include <rte_string_fns.h>
 #include <rte_dev.h>
 #include <rte_kvargs.h>
@@ -392,6 +393,13 @@ error:
 	return -1;
 }
 
+int
+rte_eth_from_ring(struct rte_ring *r)
+{
+	return rte_eth_from_rings(r->name, &r, 1, &r, 1,
+			r->memzone ? r->memzone->socket_id : SOCKET_ID_ANY);
+}
+
 enum dev_action{
 	DEV_CREATE,
 	DEV_ATTACH
drivers/net/ring/rte_eth_ring.h +14 -0
index 5a69bff..4ff83ec 100644
@@ -65,6 +65,20 @@ int rte_eth_from_rings(const char *name,
 		const unsigned nb_tx_queues,
 		const unsigned numa_node);
 
+/**
+ * Create a new ethdev port from a ring
+ *
+ * This function is a shortcut call for rte_eth_from_rings for the
+ * case where one wants to take a single rte_ring and use it as though
+ * it were an ethdev
+ *
+ * @param ring
+ *    the ring to be used as an ethdev
+ * @return
+ *    the port number of the newly created ethdev, or -1 on error
+ */
+int rte_eth_from_ring(struct rte_ring *r);
+
 #ifdef __cplusplus
 }
 #endif
drivers/net/ring/rte_eth_ring_version.map +5 -0
index 0875e25..8e113a4 100644
@@ -5,3 +5,8 @@ DPDK_2.0 {
 
 	local: *;
 };
+
+DPDK_2.2 {
+	global:
+	rte_eth_from_ring;
+} DPDK_2.0;
Signature · 1 line
2.4.3
#5 lore raw

PATCH 4/4 test: Add perf test for ring pmd

In reply to Bruce Richardson

Add a performance test for ring pmd, comparing performance of the pmd compared to the basic rte_ring APIs.

---
 app/test/Makefile             |   1 +
 app/test/test_pmd_ring_perf.c | 188 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 189 insertions(+)
 create mode 100644 app/test/test_pmd_ring_perf.c
app/test/Makefile +1 -0
index 294618f..14fa502 100644
@@ -141,6 +141,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c
 endif
 
 SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring_perf.c
 SRCS-$(CONFIG_RTE_LIBRTE_KVARGS) += test_kvargs.c
 
 CFLAGS += -O3
app/test/test_pmd_ring_perf.c new file +188 -0

188 lines not on this page · raw message

Signature · 1 line
2.4.3
#6 lore raw
In reply to Bruce Richardson
Quoted 36 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Bruce Richardson
Sent: Wednesday, September 30, 2015 1:12 PM
To: [email protected]
Subject: [dpdk-dev] [PATCH 0/4] eth_ring: perf test and usability improvements

This patchset makes it easier to create ring pmd instances from code, by providing a simple ring->ethdev wrapper function and also ensuring that any created rings are ready for use immediately, without having to call configure and rx/tx queue setup.

This set also contains a set of unit tests to compare the performance of basic ring operations against the same operations via the ring ethdev. This shows how the perf penalty can be significant for small bursts, but is much less so for larger bursts of 32 packets.

Bruce Richardson (4):
ring: enhance rte_eth_from_rings
rte_ring: store memzone pointer inside ring
ring: add rte_eth_from_ring function
test: Add perf test for ring pmd

 app/test/Makefile                         |   1 +
 app/test/test_pmd_ring_perf.c             | 188
++++++++++++++++++++++++++++++
 drivers/net/ring/rte_eth_ring.c           |  55 ++++++++-
 drivers/net/ring/rte_eth_ring.h           |  14 +++
 drivers/net/ring/rte_eth_ring_version.map |   5 +
 lib/librte_ring/rte_ring.c                |   1 +
 lib/librte_ring/rte_ring.h                |   4 +
 7 files changed, 262 insertions(+), 6 deletions(-)
 create mode 100644 app/test/test_pmd_ring_perf.c
Quoted 2 lines

--
2.4.3

#7 lore raw
Quoted 10 lines
Quoted 10 lines

This patchset makes it easier to create ring pmd instances from code, by providing a simple ring->ethdev wrapper function and also ensuring that any created rings are ready for use immediately, without having to call configure and rx/tx queue setup.

This set also contains a set of unit tests to compare the performance of basic ring operations against the same operations via the ring ethdev. This shows how the perf penalty can be significant for small bursts, but is much less so for larger bursts of 32 packets.

Quoted 7 lines
Quoted 5 lines

Bruce Richardson (4):
ring: enhance rte_eth_from_rings
rte_ring: store memzone pointer inside ring
ring: add rte_eth_from_ring function
test: Add perf test for ring pmd

Applied, thanks