ethdev: introduce shared Rx queue

RFC In review Xueming Li 14 participants 266 messages Jul 27, 2021 – Nov 5, 2021 Page 3 of 3

Subsystems (2)

Network drivers Arm/Amlogic Meson SoC support

lore

Share & tools

lore

Xueming Li

author

2021-10-19 15:28 UTC #165 lore raw

PATCH v10 1/7 ethdev: introduce shared Rx queue

In reply to Xueming Li

In current DPDK framework, each Rx queue is pre-loaded with mbufs to save incoming packets. For some PMDs, when number of representors scale out in a switch domain, the memory consumption became significant. Polling all ports also leads to high cache miss, high latency and low throughput.

This patch introduces shared Rx queue. Ports in same Rx domain and switch domain could share Rx queue set by specifying non-zero sharing group in Rx queue configuration.

Shared Rx queue is identified by share_rxq field of Rx queue configuration. Port A RxQ X can share RxQ with Port B RxQ Y by using same shared Rx queue ID.

No special API is defined to receive packets from shared Rx queue. Polling any member port of a shared Rx queue receives packets of that queue for all member ports, port_id is identified by mbuf->port. PMD is responsible to resolve shared Rx queue from device and queue data.

Shared Rx queue must be polled in same thread or core, polling a queue ID of any member port is essentially same.

Multiple share groups are supported. PMD should support mixed configuration by allowing multiple share groups and non-shared Rx queue on one port.

Example grouping and polling model to reflect service priority:
Group1, 2 shared Rx queues per port: PF, rep0, rep1 Group2, 1 shared Rx queue per port: rep2, rep3, ... rep127
Core0: poll PF queue0
Core1: poll PF queue1
Core2: poll rep2 queue0

PMD advertise shared Rx queue capability via RTE_ETH_DEV_CAPA_RXQ_SHARE.

PMD is responsible for shared Rx queue consistency checks to avoid member port's configuration contradict each other.

Signed-off-by Xueming Li <[email protected]>

Reviewed-by Andrew Rybchenko <[email protected]>

---
 doc/guides/nics/features.rst                  | 13 ++++++++++
 doc/guides/nics/features/default.ini          |  1 +
 .../prog_guide/switch_representation.rst      | 11 +++++++++
 doc/guides/rel_notes/release_21_11.rst        |  6 +++++
 lib/ethdev/rte_ethdev.c                       |  8 +++++++
 lib/ethdev/rte_ethdev.h                       | 24 +++++++++++++++++++
 6 files changed, 63 insertions(+)

doc/guides/nics/features.rst +13 -0

index e346018e4b8..89f9accbca1 100644
@@ -615,6 +615,19 @@ Supports inner packet L4 checksum.
   ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_OUTER_UDP_CKSUM``.
 
 
+.. _nic_features_shared_rx_queue:
+
+Shared Rx queue
+---------------
+
+Supports shared Rx queue for ports in same Rx domain of a switch domain.
+
+* **[uses]     rte_eth_dev_info**: ``dev_capa:RTE_ETH_DEV_CAPA_RXQ_SHARE``.
+* **[uses]     rte_eth_dev_info，rte_eth_switch_info**: ``rx_domain``, ``domain_id``.
+* **[uses]     rte_eth_rxconf**: ``share_group``, ``share_qid``.
+* **[provides] mbuf**: ``mbuf.port``.
+
+
 .. _nic_features_packet_type_parsing:
 
 Packet type parsing

doc/guides/nics/features/default.ini +1 -0

index d473b94091a..93f5d1b46f4 100644
@@ -19,6 +19,7 @@ Free Tx mbuf on demand =
 Queue start/stop     =
 Runtime Rx queue setup =
 Runtime Tx queue setup =
+Shared Rx queue      =
 Burst mode info      =
 Power mgmt address monitor =
 MTU update           =

doc/guides/prog_guide/switch_representation.rst +11 -0

index ff6aa91c806..4f2532a91ea 100644
@@ -123,6 +123,17 @@ thought as a software "patch panel" front-end for applications.
 .. [1] `Ethernet switch device driver model (switchdev)
        <https://www.kernel.org/doc/Documentation/networking/switchdev.txt>`_
 
+- For some PMDs, memory usage of representors is huge when number of
+  representor grows, mbufs are allocated for each descriptor of Rx queue.
+  Polling large number of ports brings more CPU load, cache miss and
+  latency. Shared Rx queue can be used to share Rx queue between PF and
+  representors among same Rx domain. ``RTE_ETH_DEV_CAPA_RXQ_SHARE`` in
+  device info is used to indicate the capability. Setting non-zero share
+  group in Rx queue configuration to enable share, share_qid is used to
+  identify the shared Rx queue in group. Polling any member port can
+  receive packets of all member ports in the group, port ID is saved in
+  ``mbuf.port``.
+
 Basic SR-IOV
 ------------

doc/guides/rel_notes/release_21_11.rst +6 -0

index d5435a64aa1..b34d9776a15 100644
@@ -75,6 +75,12 @@ New Features
     operations.
   * Added multi-process support.
 
+* **Added ethdev shared Rx queue support.**
+
+  * Added new device capability flag and Rx domain field to switch info.
+  * Added share group and share queue ID to Rx queue configuration.
+  * Added testpmd support and dedicate forwarding engine.
+
 * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.**
 
   Added macros ETH_RSS_IPV4_CHKSUM and ETH_RSS_L4_CHKSUM, now IPv4 and

lib/ethdev/rte_ethdev.c +8 -0

index 028907bc4b9..bc55f899f72 100644
@@ -2159,6 +2159,14 @@ rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id,
 		return -EINVAL;
 	}
 
+	if (local_conf.share_group > 0 &&
+	    (dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE) == 0) {
+		RTE_ETHDEV_LOG(ERR,
+			"Ethdev port_id=%d rx_queue_id=%d, enabled share_group=%hu while device doesn't support Rx queue share\n",
+			port_id, rx_queue_id, local_conf.share_group);
+		return -EINVAL;
+	}
+
 	/*
 	 * If LRO is enabled, check that the maximum aggregated packet
 	 * size is supported by the configured device.

lib/ethdev/rte_ethdev.h +24 -0

index 6d80514ba7a..34acc91273d 100644
@@ -1044,6 +1044,14 @@ struct rte_eth_rxconf {
 	uint8_t rx_drop_en; /**< Drop packets if no descriptors are available. */
 	uint8_t rx_deferred_start; /**< Do not start queue with rte_eth_dev_start(). */
 	uint16_t rx_nseg; /**< Number of descriptions in rx_seg array. */
+	/**
+	 * Share group index in Rx domain and switch domain.
+	 * Non-zero value to enable Rx queue share, zero value disable share.
+	 * PMD is responsible for Rx queue consistency checks to avoid member
+	 * port's configuration contradict to each other.
+	 */
+	uint16_t share_group;
+	uint16_t share_qid; /**< Shared Rx queue ID in group. */
 	/**
 	 * Per-queue Rx offloads to be set using DEV_RX_OFFLOAD_* flags.
 	 * Only offloads set on rx_queue_offload_capa or rx_offload_capa

@@ -1445,6 +1453,16 @@ struct rte_eth_conf {
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /** Device supports Tx queue setup after device started. */
 #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
+/**
+ * Device supports shared Rx queue among ports within Rx domain and
+ * switch domain. Mbufs are consumed by shared Rx queue instead of
+ * each queue. Multiple groups are supported by share_group of Rx
+ * queue configuration. Shared Rx queue is identified by PMD using
+ * share_qid of Rx queue configuration. Polling any port in the group
+ * receive packets of all member ports, source port identified by
+ * mbuf->port field.
+ */
+#define RTE_ETH_DEV_CAPA_RXQ_SHARE              RTE_BIT64(2)
 /**@}*/
 
 /*

@@ -1488,6 +1506,12 @@ struct rte_eth_switch_info {
 	 * but each driver should explicitly define the mapping of switch
 	 * port identifier to that physical interconnect/switch
 	 */
+	/**
+	 * Shared Rx queue sub-domain boundary. Only ports in same Rx domain
+	 * and switch domain can share Rx queue. Valid only if device advertised
+	 * RTE_ETH_DEV_CAPA_RXQ_SHARE capability.
+	 */
+	uint16_t rx_domain;
 };
 
 /**

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-19 15:28 UTC #166 lore raw

PATCH v10 2/7 ethdev: new API to resolve device capability name

In reply to Xueming Li

This patch adds API to return name of device capability.

Signed-off-by Xueming Li <[email protected]>

---
 lib/ethdev/rte_ethdev.c | 30 ++++++++++++++++++++++++++++++
 lib/ethdev/rte_ethdev.h | 14 ++++++++++++++
 lib/ethdev/version.map  |  3 +++
 3 files changed, 47 insertions(+)

lib/ethdev/rte_ethdev.c +30 -0

index bc55f899f72..97217529449 100644
@@ -165,6 +165,20 @@ static const struct {
 
 #undef RTE_TX_OFFLOAD_BIT2STR
 
+#define RTE_ETH_DEV_CAPA_BIT2STR(_name)	\
+	{ RTE_ETH_DEV_CAPA_##_name, #_name }
+
+static const struct {
+	uint64_t offload;
+	const char *name;
+} rte_eth_dev_capa_names[] = {
+	RTE_ETH_DEV_CAPA_BIT2STR(RUNTIME_RX_QUEUE_SETUP),
+	RTE_ETH_DEV_CAPA_BIT2STR(RUNTIME_TX_QUEUE_SETUP),
+	RTE_ETH_DEV_CAPA_BIT2STR(RXQ_SHARE),
+};
+
+#undef RTE_ETH_DEV_CAPA_BIT2STR
+
 /**
  * The user application callback description.
  *

@@ -1260,6 +1274,22 @@ rte_eth_dev_tx_offload_name(uint64_t offload)
 	return name;
 }
 
+const char *
+rte_eth_dev_capability_name(uint64_t capability)
+{
+	const char *name = "UNKNOWN";
+	unsigned int i;
+
+	for (i = 0; i < RTE_DIM(rte_eth_dev_capa_names); ++i) {
+		if (capability == rte_eth_dev_capa_names[i].offload) {
+			name = rte_eth_dev_capa_names[i].name;
+			break;
+		}
+	}
+
+	return name;
+}
+
 static inline int
 eth_dev_check_lro_pkt_size(uint16_t port_id, uint32_t config_size,
 		   uint32_t max_rx_pkt_len, uint32_t dev_info_size)

lib/ethdev/rte_ethdev.h +14 -0

index 34acc91273d..df8ef9382a9 100644
@@ -2109,6 +2109,20 @@ const char *rte_eth_dev_rx_offload_name(uint64_t offload);
  */
 const char *rte_eth_dev_tx_offload_name(uint64_t offload);
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get RTE_ETH_DEV_CAPA_* flag name.
+ *
+ * @param capability
+ *   Capability flag.
+ * @return
+ *   Capability name or 'UNKNOWN' if the flag cannot be recognized.
+ */
+__rte_experimental
+const char *rte_eth_dev_capability_name(uint64_t capability);
+
 /**
  * Configure an Ethernet device.
  * This function must be invoked first before any other function in the

lib/ethdev/version.map +3 -0

index efd729c0f2d..e1d403dd357 100644
@@ -245,6 +245,9 @@ EXPERIMENTAL {
 	rte_mtr_meter_policy_delete;
 	rte_mtr_meter_policy_update;
 	rte_mtr_meter_policy_validate;
+
+	# added in 21.11
+	rte_eth_dev_capability_name;
 };
 
 INTERNAL {

Signature · 1 line

2.33.0

Andrew Rybchenko

2021-10-19 17:57 UTC #167 lore raw

Re: [dpdk-dev] [PATCH v10 2/7] ethdev: new API to resolve device capability name

Quoted 3 lines · Xueming Li

This patch adds API to return name of device capability.
Signed-off-by Xueming Li <[email protected]>

[snip]

lib/ethdev/rte_ethdev.c static const struct { · line 169

Quoted 10 lines

lib/ethdev/rte_ethdev.c +2 -0

index bc55f899f72..97217529449 100644
@@ -165,6 +165,20 @@ static const struct {
 
  #undef RTE_TX_OFFLOAD_BIT2STR
 
+#define RTE_ETH_DEV_CAPA_BIT2STR(_name)	\
+	{ RTE_ETH_DEV_CAPA_##_name, #_name }

In fact, such macros make more harm than add value. It complicates grep by capability name. So, it is better to drop the macro and just duplicate few symbols below.

Quoted 15 lines

lib/ethdev/rte_ethdev.c +12 -0

+
+static const struct {
+	uint64_t offload;
+	const char *name;
+} rte_eth_dev_capa_names[] = {
+	RTE_ETH_DEV_CAPA_BIT2STR(RUNTIME_RX_QUEUE_SETUP),
+	RTE_ETH_DEV_CAPA_BIT2STR(RUNTIME_TX_QUEUE_SETUP),
+	RTE_ETH_DEV_CAPA_BIT2STR(RXQ_SHARE),
+};
+
+#undef RTE_ETH_DEV_CAPA_BIT2STR
+
  /**
   * The user application callback description.
   *

Xueming(Steven) Li

author

2021-10-20 07:47 UTC #168 lore raw

Re: [dpdk-dev] [PATCH v10 2/7] ethdev: new API to resolve device capability name

Quoted 13 lines · Andrew Rybchenko

Quoted 3 lines · Xueming Li
This patch adds API to return name of device capability.
Signed-off-by Xueming Li <[email protected]>
[snip]
Quoted 6 lines
lib/ethdev/rte_ethdev.c +0 -0
index bc55f899f72..97217529449 100644
@@ -165,6 +165,20 @@ static const struct {
 

Quoted 8 lines

lib/ethdev/rte_ethdev.c static const struct { · line 169
Quoted 4 lines
lib/ethdev/rte_ethdev.c +2 -0
  #undef RTE_TX_OFFLOAD_BIT2STR
 
+#define RTE_ETH_DEV_CAPA_BIT2STR(_name)	\
+	{ RTE_ETH_DEV_CAPA_##_name, #_name }
In fact, such macros make more harm than add value. It complicates grep by capability name. So, it is better to drop the macro and just duplicate few symbols below.

Will update in next version. Eclipse resolves macros and search into expanded macros.

BTW, do you plan to review other patches today? If so I will hold new version a little bit to avoid explode maillist.

Quoted 16 lines

lib/ethdev/rte_ethdev.c +12 -0

+
+static const struct {
+	uint64_t offload;
+	const char *name;
+} rte_eth_dev_capa_names[] = {
+	RTE_ETH_DEV_CAPA_BIT2STR(RUNTIME_RX_QUEUE_SETUP),
+	RTE_ETH_DEV_CAPA_BIT2STR(RUNTIME_TX_QUEUE_SETUP),
+	RTE_ETH_DEV_CAPA_BIT2STR(RXQ_SHARE),
+};
+
+#undef RTE_ETH_DEV_CAPA_BIT2STR
+
  /**
   * The user application callback description.
   *

Andrew Rybchenko

2021-10-20 07:48 UTC #169 lore raw

Re: [dpdk-dev] [PATCH v10 2/7] ethdev: new API to resolve device capability name

Quoted 23 lines · Xueming(Steven) Li

Quoted 13 lines · Andrew Rybchenko
Quoted 3 lines · Xueming Li
This patch adds API to return name of device capability.
Signed-off-by Xueming Li <[email protected]>
[snip]
Quoted 6 lines
lib/ethdev/rte_ethdev.c +0 -0
index bc55f899f72..97217529449 100644
@@ -165,6 +165,20 @@ static const struct {
 
Quoted 8 lines
lib/ethdev/rte_ethdev.c static const struct { · line 169
Quoted 4 lines
lib/ethdev/rte_ethdev.c +2 -0
  #undef RTE_TX_OFFLOAD_BIT2STR
 
+#define RTE_ETH_DEV_CAPA_BIT2STR(_name)	\
+	{ RTE_ETH_DEV_CAPA_##_name, #_name }
In fact, such macros make more harm than add value. It complicates grep by capability name. So, it is better to drop the macro and just duplicate few symbols below.

Quoted 5 lines

Will update in next version. Eclipse resolves macros and search into expanded macros.
BTW, do you plan to review other patches today? If so I will hold new version a little bit to avoid explode maillist.

Sorry, I have no time to review testpmd patches today. ethdev part LGTM.

Xueming Li

author

2021-10-19 15:29 UTC #170 lore raw

PATCH v10 3/7 app/testpmd: dump device capability and Rx domain info

In reply to Xueming Li

Dump device capability and Rx domain ID if shared Rx queue is supported by device.

Signed-off-by Xueming Li <[email protected]>

Acked-by Andrew Rybchenko <[email protected]>

---
 app/test-pmd/config.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

app/test-pmd/config.c +29 -0

index 9c66329e96e..2c1b06c544d 100644
@@ -582,6 +582,29 @@ device_infos_display(const char *identifier)
 	rte_devargs_reset(&da);
 }
 
+static void
+print_dev_capabilities(uint64_t capabilities)
+{
+	uint64_t single_capa;
+	int begin;
+	int end;
+	int bit;
+
+	if (capabilities == 0)
+		return;
+
+	begin = __builtin_ctzll(capabilities);
+	end = sizeof(capabilities) * CHAR_BIT - __builtin_clzll(capabilities);
+
+	single_capa = 1ULL << begin;
+	for (bit = begin; bit < end; bit++) {
+		if (capabilities & single_capa)
+			printf(" %s",
+			       rte_eth_dev_capability_name(single_capa));
+		single_capa <<= 1;
+	}
+}
+
 void
 port_infos_display(portid_t port_id)
 {

@@ -733,6 +756,9 @@ port_infos_display(portid_t port_id)
 	printf("Max segment number per MTU/TSO: %hu\n",
 		dev_info.tx_desc_lim.nb_mtu_seg_max);
 
+	printf("Device capabilities: 0x%"PRIx64"(", dev_info.dev_capa);
+	print_dev_capabilities(dev_info.dev_capa);
+	printf(" )\n");
 	/* Show switch info only if valid switch domain and port id is set */
 	if (dev_info.switch_info.domain_id !=
 		RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID) {

@@ -743,6 +769,9 @@ port_infos_display(portid_t port_id)
 			dev_info.switch_info.domain_id);
 		printf("Switch Port Id: %u\n",
 			dev_info.switch_info.port_id);
+		if ((dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE) != 0)
+			printf("Switch Rx domain: %u\n",
+			       dev_info.switch_info.rx_domain);
 	}
 }

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-19 15:29 UTC #171 lore raw

PATCH v10 4/7 app/testpmd: new parameter to enable shared Rx queue

In reply to Xueming Li

Adds "--rxq-share=X" parameter to enable shared RxQ, share if device supports, otherwise fallback to standard RxQ.

Share group number grows per X ports. X defaults to MAX, implies all ports join share group 1. Queue ID is mapped equally with shared Rx queue ID.

Forwarding engine "shared-rxq" should be used which Rx only and update stream statistics correctly.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/config.c                 |  7 ++++++-
 app/test-pmd/parameters.c             | 13 +++++++++++++
 app/test-pmd/testpmd.c                | 20 +++++++++++++++++---
 app/test-pmd/testpmd.h                |  2 ++
 doc/guides/testpmd_app_ug/run_app.rst |  7 +++++++
 5 files changed, 45 insertions(+), 4 deletions(-)

app/test-pmd/config.c +6 -1

index 2c1b06c544d..fa951a86704 100644
@@ -2738,7 +2738,12 @@ rxtx_config_display(void)
 			printf("      RX threshold registers: pthresh=%d hthresh=%d "
 				" wthresh=%d\n",
 				pthresh_tmp, hthresh_tmp, wthresh_tmp);
-			printf("      RX Offloads=0x%"PRIx64"\n", offloads_tmp);
+			printf("      RX Offloads=0x%"PRIx64, offloads_tmp);
+			if (rx_conf->share_group > 0)
+				printf(" share_group=%u share_qid=%u",
+				       rx_conf->share_group,
+				       rx_conf->share_qid);
+			printf("\n");
 		}
 
 		/* per tx queue config only for first queue to be less verbose */

app/test-pmd/parameters.c +13 -0

index 3f94a82e321..30dae326310 100644
@@ -167,6 +167,7 @@ usage(char* progname)
 	printf("  --tx-ip=src,dst: IP addresses in Tx-only mode\n");
 	printf("  --tx-udp=src[,dst]: UDP ports in Tx-only mode\n");
 	printf("  --eth-link-speed: force link speed.\n");
+	printf("  --rxq-share: number of ports per shared rxq groups, defaults to MAX(1 group)\n");
 	printf("  --disable-link-check: disable check on link status when "
 	       "starting/stopping ports.\n");
 	printf("  --disable-device-start: do not automatically start port\n");

@@ -607,6 +608,7 @@ launch_args_parse(int argc, char** argv)
 		{ "rxpkts",			1, 0, 0 },
 		{ "txpkts",			1, 0, 0 },
 		{ "txonly-multi-flow",		0, 0, 0 },
+		{ "rxq-share",			2, 0, 0 },
 		{ "eth-link-speed",		1, 0, 0 },
 		{ "disable-link-check",		0, 0, 0 },
 		{ "disable-device-start",	0, 0, 0 },

@@ -1271,6 +1273,17 @@ launch_args_parse(int argc, char** argv)
 			}
 			if (!strcmp(lgopts[opt_idx].name, "txonly-multi-flow"))
 				txonly_multi_flow = 1;
+			if (!strcmp(lgopts[opt_idx].name, "rxq-share")) {
+				if (optarg == NULL) {
+					rxq_share = UINT32_MAX;
+				} else {
+					n = atoi(optarg);
+					if (n >= 0)
+						rxq_share = (uint32_t)n;
+					else
+						rte_exit(EXIT_FAILURE, "rxq-share must be >= 0\n");
+				}
+			}
 			if (!strcmp(lgopts[opt_idx].name, "no-flush-rx"))
 				no_flush_rx = 1;
 			if (!strcmp(lgopts[opt_idx].name, "eth-link-speed")) {

app/test-pmd/testpmd.c +17 -3

index 97ae52e17ec..123142ed110 100644
@@ -498,6 +498,11 @@ uint8_t record_core_cycles;
  */
 uint8_t record_burst_stats;
 
+/*
+ * Number of ports per shared Rx queue group, 0 disable.
+ */
+uint32_t rxq_share;
+
 unsigned int num_sockets = 0;
 unsigned int socket_ids[RTE_MAX_NUMA_NODES];

@@ -3393,14 +3398,23 @@ dev_event_callback(const char *device_name, enum rte_dev_event_type type,
 }
 
 static void
-rxtx_port_config(struct rte_port *port)
+rxtx_port_config(portid_t pid)
 {
 	uint16_t qid;
 	uint64_t offloads;
+	struct rte_port *port = &ports[pid];
 
 	for (qid = 0; qid < nb_rxq; qid++) {
 		offloads = port->rx_conf[qid].offloads;
 		port->rx_conf[qid] = port->dev_info.default_rxconf;
+
+		if (rxq_share > 0 &&
+		    (port->dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)) {
+			/* Non-zero share group to enable RxQ share. */
+			port->rx_conf[qid].share_group = pid / rxq_share + 1;
+			port->rx_conf[qid].share_qid = qid; /* Equal mapping. */
+		}
+
 		if (offloads != 0)
 			port->rx_conf[qid].offloads = offloads;

@@ -3558,7 +3572,7 @@ init_port_config(void)
 				port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_NONE;
 		}
 
-		rxtx_port_config(port);
+		rxtx_port_config(pid);
 
 		ret = eth_macaddr_get_print_err(pid, &port->eth_addr);
 		if (ret != 0)

@@ -3772,7 +3786,7 @@ init_port_dcb_config(portid_t pid,
 
 	memcpy(&rte_port->dev_conf, &port_conf, sizeof(struct rte_eth_conf));
 
-	rxtx_port_config(rte_port);
+	rxtx_port_config(pid);
 	/* VLAN filter */
 	rte_port->dev_conf.rxmode.offloads |= DEV_RX_OFFLOAD_VLAN_FILTER;
 	for (i = 0; i < RTE_DIM(vlan_tags); i++)

app/test-pmd/testpmd.h +2 -0

index 5863b2f43f3..3dfaaad94c0 100644
@@ -477,6 +477,8 @@ extern enum tx_pkt_split tx_pkt_split;
 
 extern uint8_t txonly_multi_flow;
 
+extern uint32_t rxq_share;
+
 extern uint16_t nb_pkt_per_burst;
 extern uint16_t nb_pkt_flowgen_clones;
 extern int nb_flows_flowgen;

doc/guides/testpmd_app_ug/run_app.rst +7 -0

index 640eadeff73..ff5908dcd50 100644
@@ -389,6 +389,13 @@ The command line options are:
 
     Generate multiple flows in txonly mode.
 
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Group number grows per X ports. X defaults to MAX, implies all ports
+    join share group 1. Forwarding engine "shared-rxq" should be used
+    which Rx only and update stream statistics correctly.
+
 *   ``--eth-link-speed``
 
     Set a forced link speed to the ethernet port::

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-19 15:29 UTC #172 lore raw

PATCH v10 5/7 app/testpmd: dump port info for shared Rx queue

In reply to Xueming Li

In case of shared Rx queue, polling any member port returns mbufs for all members. This patch dumps mbuf->port for each packet.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/util.c | 3 +++
 1 file changed, 3 insertions(+)

app/test-pmd/util.c +3 -0

index 51506e49404..e98f136d5ed 100644
@@ -100,6 +100,9 @@ dump_pkt_burst(uint16_t port_id, uint16_t queue, struct rte_mbuf *pkts[],
 		struct rte_flow_restore_info info = { 0, };
 
 		mb = pkts[i];
+		if (rxq_share > 0)
+			MKDUMPSTR(print_buf, buf_size, cur_len, "port %u, ",
+				  mb->port);
 		eth_hdr = rte_pktmbuf_read(mb, 0, sizeof(_eth_hdr), &_eth_hdr);
 		eth_type = RTE_BE_TO_CPU_16(eth_hdr->ether_type);
 		packet_type = mb->packet_type;

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-19 15:29 UTC #173 lore raw

PATCH v10 6/7 app/testpmd: force shared Rx queue polled on same core

In reply to Xueming Li

Shared Rx queue must be polled on same core. This patch checks and stops forwarding if shared RxQ being scheduled on multiple
cores.

It's suggested to use same number of Rx queues and polling cores.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/config.c  | 103 +++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.c |   4 +-
 app/test-pmd/testpmd.h |   2 +
 3 files changed, 108 insertions(+), 1 deletion(-)

app/test-pmd/config.c +103 -0

index fa951a86704..1f1307178be 100644
@@ -2915,6 +2915,109 @@ port_rss_hash_key_update(portid_t port_id, char rss_type[], uint8_t *hash_key,
 	}
 }
 
+/*
+ * Check whether a shared rxq scheduled on other lcores.
+ */
+static bool
+fwd_stream_on_other_lcores(uint16_t domain_id, lcoreid_t src_lc,
+			   portid_t src_port, queueid_t src_rxq,
+			   uint32_t share_group, queueid_t share_rxq)
+{
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/* Check remaining cores. */
+	for (lc_id = src_lc + 1; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)
+			    == 0)
+				/* Not shared rxq. */
+				continue;
+			if (domain_id != port->dev_info.switch_info.domain_id)
+				continue;
+			if (rxq_conf->share_group != share_group)
+				continue;
+			if (rxq_conf->share_qid != share_rxq)
+				continue;
+			printf("Shared Rx queue group %u queue %hu can't be scheduled on different cores:\n",
+			       share_group, share_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       src_lc, src_port, src_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       lc_id, fs->rx_port, fs->rx_queue);
+			printf("Please use --nb-cores=%hu to limit number of forwarding cores\n",
+			       nb_rxq);
+			return true;
+		}
+	}
+	return false;
+}
+
+/*
+ * Check shared rxq configuration.
+ *
+ * Shared group must not being scheduled on different core.
+ */
+bool
+pkt_fwd_shared_rxq_check(void)
+{
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	uint16_t domain_id;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/*
+	 * Check streams on each core, make sure the same switch domain +
+	 * group + queue doesn't get scheduled on other cores.
+	 */
+	for (lc_id = 0; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			/* Update lcore info stream being scheduled. */
+			fs->lcore = fwd_lcores[lc_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)
+			    == 0)
+				/* Not shared rxq. */
+				continue;
+			/* Check shared rxq not scheduled on remaining cores. */
+			domain_id = port->dev_info.switch_info.domain_id;
+			if (fwd_stream_on_other_lcores(domain_id, lc_id,
+						       fs->rx_port,
+						       fs->rx_queue,
+						       rxq_conf->share_group,
+						       rxq_conf->share_qid))
+				return false;
+		}
+	}
+	return true;
+}
+
 /*
  * Setup forwarding configuration for each logical core.
  */

app/test-pmd/testpmd.c +3 -1

index 123142ed110..f3f81ef561f 100644
@@ -2236,10 +2236,12 @@ start_packet_forwarding(int with_tx_first)
 
 	fwd_config_setup();
 
+	pkt_fwd_config_display(&cur_fwd_config);
+	if (!pkt_fwd_shared_rxq_check())
+		return;
 	if(!no_flush_rx)
 		flush_fwd_rx_queues();
 
-	pkt_fwd_config_display(&cur_fwd_config);
 	rxtx_config_display();
 
 	fwd_stats_reset();

app/test-pmd/testpmd.h +2 -0

index 3dfaaad94c0..f121a2da90c 100644
@@ -144,6 +144,7 @@ struct fwd_stream {
 	uint64_t     core_cycles; /**< used for RX and TX processing */
 	struct pkt_burst_stats rx_burst_stats;
 	struct pkt_burst_stats tx_burst_stats;
+	struct fwd_lcore *lcore; /**< Lcore being scheduled. */
 };
 
 /**

@@ -795,6 +796,7 @@ void port_summary_header_display(void);
 void rx_queue_infos_display(portid_t port_idi, uint16_t queue_id);
 void tx_queue_infos_display(portid_t port_idi, uint16_t queue_id);
 void fwd_lcores_config_display(void);
+bool pkt_fwd_shared_rxq_check(void);
 void pkt_fwd_config_display(struct fwd_config *cfg);
 void rxtx_config_display(void);
 void fwd_config_setup(void);

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-19 15:29 UTC #174 lore raw

PATCH v10 7/7 app/testpmd: add forwarding engine for shared Rx queue

In reply to Xueming Li

To support shared Rx queue, this patch introduces dedicate forwarding engine. The engine groups received packets by mbuf->port into sub-group, updates stream statistics and simply frees packets.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/meson.build                    |   1 +
 app/test-pmd/shared_rxq_fwd.c               | 148 ++++++++++++++++++++
 app/test-pmd/testpmd.c                      |   1 +
 app/test-pmd/testpmd.h                      |   1 +
 doc/guides/testpmd_app_ug/run_app.rst       |   1 +
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +-
 6 files changed, 156 insertions(+), 1 deletion(-)
 create mode 100644 app/test-pmd/shared_rxq_fwd.c

app/test-pmd/meson.build +1 -0

index 98f3289bdfa..07042e45b12 100644
@@ -21,6 +21,7 @@ sources = files(
         'noisy_vnf.c',
         'parameters.c',
         'rxonly.c',
+        'shared_rxq_fwd.c',
         'testpmd.c',
         'txonly.c',
         'util.c',

app/test-pmd/shared_rxq_fwd.c new file +148 -0

148 lines not on this page · raw message

app/test-pmd/testpmd.c +1 -0

index f3f81ef561f..11a85d92d9a 100644
@@ -188,6 +188,7 @@ struct fwd_engine * fwd_engines[] = {
 #ifdef RTE_LIBRTE_IEEE1588
 	&ieee1588_fwd_engine,
 #endif
+	&shared_rxq_engine,
 	NULL,
 };

app/test-pmd/testpmd.h +1 -0

index f121a2da90c..f1fd607e365 100644
@@ -299,6 +299,7 @@ extern struct fwd_engine five_tuple_swap_fwd_engine;
 #ifdef RTE_LIBRTE_IEEE1588
 extern struct fwd_engine ieee1588_fwd_engine;
 #endif
+extern struct fwd_engine shared_rxq_engine;
 
 extern struct fwd_engine * fwd_engines[]; /**< NULL terminated array. */
 extern cmdline_parse_inst_t cmd_set_raw;

doc/guides/testpmd_app_ug/run_app.rst +1 -0

index ff5908dcd50..e4b97844ced 100644
@@ -252,6 +252,7 @@ The command line options are:
        tm
        noisy
        5tswap
+       shared-rxq
 
 *   ``--rss-ip``

doc/guides/testpmd_app_ug/testpmd_funcs.rst +4 -1

index 8ead7a4a712..499874187f2 100644
@@ -314,7 +314,7 @@ set fwd
 Set the packet forwarding mode::
 
    testpmd> set fwd (io|mac|macswap|flowgen| \
-                     rxonly|txonly|csum|icmpecho|noisy|5tswap) (""|retry)
+                     rxonly|txonly|csum|icmpecho|noisy|5tswap|shared-rxq) (""|retry)
 
 ``retry`` can be specified for forwarding engines except ``rx_only``.

@@ -357,6 +357,9 @@ The available information categories are:
 
   L4 swaps the source port and destination port of transport layer (TCP and UDP).
 
+* ``shared-rxq``: Receive only for shared Rx queue.
+  Resolve packet source port from mbuf and update stream statistics accordingly.
+
 Example::
 
    testpmd> set fwd rxonly

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-20 07:54 UTC #175 lore raw

v11 cover ethdev: introduce shared Rx queue

In reply to Xueming Li

In current DPDK framework, all Rx queues is pre-loaded with mbufs for incoming packets. When number of representors scale out in a switch domain, the memory consumption became significant. Further more, polling all ports leads to high cache miss, high latency and low throughputs.

This patch introduces shared Rx queue. PF and representors in same Rx domain and switch domain could share Rx queue set by specifying non-zero share group value in Rx queue configuration.

All ports that share Rx queue actually shares hardware descriptor queue and feed all Rx queues with one descriptor supply, memory is saved.

Polling any queue using same shared Rx queue receives packets from all member ports. Source port is identified by mbuf->port.

Multiple groups is supported by group ID. Port queue number in a shared group should be identical. Queue index is 1:1 mapped in shared group. An example of two share groups:
Group1, 4 shared Rx queues per member port: PF, repr0, repr1 Group2, 2 shared Rx queues per member port: repr2, repr3, ... repr127 Poll first port for each group:
core port queue
0 0 0
1 0 1
2 0 2
3 0 3
4 2 0
5 2 1

Shared Rx queue must be polled on single thread or core. If both PF0 and representor0 joined same share group, can't poll pf0rxq0 on core1 and rep0rxq0 on core2. Actually, polling one port within share group is sufficient since polling any port in group will return packets for any port in group.

There was some discussion to aggregate member ports in same group into a dummy port, several ways to achieve it. Since it optional, need to collect more feedback and requirement from user, make better decision later.

v1:
- initial version
v2:
- add testpmd patches
v3:
- change common forwarding api to macro for performance, thanks Jerin.
- save global variable accessed in forwarding to flowstream to minimize cache miss
- combined patches for each forwarding engine
- support multiple groups in testpmd "--share-rxq" parameter
- new api to aggregate shared rxq group
v4:
- spelling fixes
- remove shared-rxq support for all forwarding engines
- add dedicate shared-rxq forwarding engine
v5:
- fix grammars
- remove aggregate api and leave it for later discussion
- add release notes
- add deployment example
v6:
- replace RxQ offload flag with device offload capability flag
- add Rx domain
- RxQ is shared when share group > 0
- update testpmd accordingly
v7:
- fix testpmd share group id allocation
- change rx_domain to 16bits
v8:
- add new patch for testpmd to show device Rx domain ID and capability
- new share_qid in RxQ configuration
v9:
- fix some spelling
v10:
- add device capability name api
v11:
- remove macro from device capability name list

Xueming Li (7):
ethdev: introduce shared Rx queue
ethdev: new API to resolve device capability name app/testpmd: dump device capability and Rx domain info app/testpmd: new parameter to enable shared Rx queue app/testpmd: dump port info for shared Rx queue app/testpmd: force shared Rx queue polled on same core app/testpmd: add forwarding engine for shared Rx queue

 app/test-pmd/config.c                         | 139 +++++++++++++++-
 app/test-pmd/meson.build                      |   1 +
 app/test-pmd/parameters.c                     |  13 ++
 app/test-pmd/shared_rxq_fwd.c                 | 148 ++++++++++++++++++
 app/test-pmd/testpmd.c                        |  25 ++-
 app/test-pmd/testpmd.h                        |   5 +
 app/test-pmd/util.c                           |   3 +
 doc/guides/nics/features.rst                  |  13 ++
 doc/guides/nics/features/default.ini          |   1 +
 .../prog_guide/switch_representation.rst      |  11 ++
 doc/guides/rel_notes/release_21_11.rst        |   6 +
 doc/guides/testpmd_app_ug/run_app.rst         |   8 +
 doc/guides/testpmd_app_ug/testpmd_funcs.rst   |   5 +-
 lib/ethdev/rte_ethdev.c                       |  33 ++++
 lib/ethdev/rte_ethdev.h                       |  38 +++++
 lib/ethdev/version.map                        |   3 +
 16 files changed, 446 insertions(+), 6 deletions(-)
 create mode 100644 app/test-pmd/shared_rxq_fwd.c

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-20 07:54 UTC #176 lore raw

PATCH v11 1/7 ethdev: introduce shared Rx queue

This patch introduces shared Rx queue. Ports in same Rx domain and switch domain could share Rx queue set by specifying non-zero sharing group in Rx queue configuration.

Shared Rx queue is identified by share_rxq field of Rx queue configuration. Port A RxQ X can share RxQ with Port B RxQ Y by using same shared Rx queue ID.

Shared Rx queue must be polled in same thread or core, polling a queue ID of any member port is essentially same.

Multiple share groups are supported. PMD should support mixed configuration by allowing multiple share groups and non-shared Rx queue on one port.

PMD advertise shared Rx queue capability via RTE_ETH_DEV_CAPA_RXQ_SHARE.

PMD is responsible for shared Rx queue consistency checks to avoid member port's configuration contradict each other.

Signed-off-by Xueming Li <[email protected]>

Reviewed-by Andrew Rybchenko <[email protected]>

---
 doc/guides/nics/features.rst                  | 13 ++++++++++
 doc/guides/nics/features/default.ini          |  1 +
 .../prog_guide/switch_representation.rst      | 11 +++++++++
 doc/guides/rel_notes/release_21_11.rst        |  6 +++++
 lib/ethdev/rte_ethdev.c                       |  8 +++++++
 lib/ethdev/rte_ethdev.h                       | 24 +++++++++++++++++++
 6 files changed, 63 insertions(+)

doc/guides/nics/features.rst +13 -0

index e346018e4b8..89f9accbca1 100644
@@ -615,6 +615,19 @@ Supports inner packet L4 checksum.
   ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_OUTER_UDP_CKSUM``.
 
 
+.. _nic_features_shared_rx_queue:
+
+Shared Rx queue
+---------------
+
+Supports shared Rx queue for ports in same Rx domain of a switch domain.
+
+* **[uses]     rte_eth_dev_info**: ``dev_capa:RTE_ETH_DEV_CAPA_RXQ_SHARE``.
+* **[uses]     rte_eth_dev_info，rte_eth_switch_info**: ``rx_domain``, ``domain_id``.
+* **[uses]     rte_eth_rxconf**: ``share_group``, ``share_qid``.
+* **[provides] mbuf**: ``mbuf.port``.
+
+
 .. _nic_features_packet_type_parsing:
 
 Packet type parsing

doc/guides/nics/features/default.ini +1 -0

index d473b94091a..93f5d1b46f4 100644
@@ -19,6 +19,7 @@ Free Tx mbuf on demand =
 Queue start/stop     =
 Runtime Rx queue setup =
 Runtime Tx queue setup =
+Shared Rx queue      =
 Burst mode info      =
 Power mgmt address monitor =
 MTU update           =

doc/guides/prog_guide/switch_representation.rst +11 -0

index ff6aa91c806..4f2532a91ea 100644
@@ -123,6 +123,17 @@ thought as a software "patch panel" front-end for applications.
 .. [1] `Ethernet switch device driver model (switchdev)
        <https://www.kernel.org/doc/Documentation/networking/switchdev.txt>`_
 
+- For some PMDs, memory usage of representors is huge when number of
+  representor grows, mbufs are allocated for each descriptor of Rx queue.
+  Polling large number of ports brings more CPU load, cache miss and
+  latency. Shared Rx queue can be used to share Rx queue between PF and
+  representors among same Rx domain. ``RTE_ETH_DEV_CAPA_RXQ_SHARE`` in
+  device info is used to indicate the capability. Setting non-zero share
+  group in Rx queue configuration to enable share, share_qid is used to
+  identify the shared Rx queue in group. Polling any member port can
+  receive packets of all member ports in the group, port ID is saved in
+  ``mbuf.port``.
+
 Basic SR-IOV
 ------------

doc/guides/rel_notes/release_21_11.rst +6 -0

index 3362c52a738..caf82242f2e 100644
@@ -75,6 +75,12 @@ New Features
     operations.
   * Added multi-process support.
 
+* **Added ethdev shared Rx queue support.**
+
+  * Added new device capability flag and Rx domain field to switch info.
+  * Added share group and share queue ID to Rx queue configuration.
+  * Added testpmd support and dedicate forwarding engine.
+
 * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.**
 
   Added macros ETH_RSS_IPV4_CHKSUM and ETH_RSS_L4_CHKSUM, now IPv4 and

lib/ethdev/rte_ethdev.c +8 -0

index 028907bc4b9..bc55f899f72 100644
@@ -2159,6 +2159,14 @@ rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id,
 		return -EINVAL;
 	}
 
+	if (local_conf.share_group > 0 &&
+	    (dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE) == 0) {
+		RTE_ETHDEV_LOG(ERR,
+			"Ethdev port_id=%d rx_queue_id=%d, enabled share_group=%hu while device doesn't support Rx queue share\n",
+			port_id, rx_queue_id, local_conf.share_group);
+		return -EINVAL;
+	}
+
 	/*
 	 * If LRO is enabled, check that the maximum aggregated packet
 	 * size is supported by the configured device.

lib/ethdev/rte_ethdev.h +24 -0

index 6d80514ba7a..34acc91273d 100644
@@ -1044,6 +1044,14 @@ struct rte_eth_rxconf {
 	uint8_t rx_drop_en; /**< Drop packets if no descriptors are available. */
 	uint8_t rx_deferred_start; /**< Do not start queue with rte_eth_dev_start(). */
 	uint16_t rx_nseg; /**< Number of descriptions in rx_seg array. */
+	/**
+	 * Share group index in Rx domain and switch domain.
+	 * Non-zero value to enable Rx queue share, zero value disable share.
+	 * PMD is responsible for Rx queue consistency checks to avoid member
+	 * port's configuration contradict to each other.
+	 */
+	uint16_t share_group;
+	uint16_t share_qid; /**< Shared Rx queue ID in group. */
 	/**
 	 * Per-queue Rx offloads to be set using DEV_RX_OFFLOAD_* flags.
 	 * Only offloads set on rx_queue_offload_capa or rx_offload_capa

@@ -1445,6 +1453,16 @@ struct rte_eth_conf {
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /** Device supports Tx queue setup after device started. */
 #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
+/**
+ * Device supports shared Rx queue among ports within Rx domain and
+ * switch domain. Mbufs are consumed by shared Rx queue instead of
+ * each queue. Multiple groups are supported by share_group of Rx
+ * queue configuration. Shared Rx queue is identified by PMD using
+ * share_qid of Rx queue configuration. Polling any port in the group
+ * receive packets of all member ports, source port identified by
+ * mbuf->port field.
+ */
+#define RTE_ETH_DEV_CAPA_RXQ_SHARE              RTE_BIT64(2)
 /**@}*/
 
 /*

@@ -1488,6 +1506,12 @@ struct rte_eth_switch_info {
 	 * but each driver should explicitly define the mapping of switch
 	 * port identifier to that physical interconnect/switch
 	 */
+	/**
+	 * Shared Rx queue sub-domain boundary. Only ports in same Rx domain
+	 * and switch domain can share Rx queue. Valid only if device advertised
+	 * RTE_ETH_DEV_CAPA_RXQ_SHARE capability.
+	 */
+	uint16_t rx_domain;
 };
 
 /**

Signature · 1 line

2.33.0

Ajit Khaparde

Acked-by on v11 1/7

2021-10-20 17:15 UTC #177 lore raw

Quoted 34 lines · Xueming Li

In current DPDK framework, each Rx queue is pre-loaded with mbufs to save incoming packets. For some PMDs, when number of representors scale out in a switch domain, the memory consumption became significant. Polling all ports also leads to high cache miss, high latency and low throughput.
This patch introduces shared Rx queue. Ports in same Rx domain and switch domain could share Rx queue set by specifying non-zero sharing group in Rx queue configuration.
Shared Rx queue is identified by share_rxq field of Rx queue configuration. Port A RxQ X can share RxQ with Port B RxQ Y by using same shared Rx queue ID.
No special API is defined to receive packets from shared Rx queue. Polling any member port of a shared Rx queue receives packets of that queue for all member ports, port_id is identified by mbuf->port. PMD is responsible to resolve shared Rx queue from device and queue data.
Shared Rx queue must be polled in same thread or core, polling a queue ID of any member port is essentially same.
Multiple share groups are supported. PMD should support mixed configuration by allowing multiple share groups and non-shared Rx queue on one port.
Example grouping and polling model to reflect service priority:
Group1, 2 shared Rx queues per port: PF, rep0, rep1 Group2, 1 shared Rx queue per port: rep2, rep3, ... rep127
Core0: poll PF queue0
Core1: poll PF queue1
Core2: poll rep2 queue0

Quoted 7 lines

PMD advertise shared Rx queue capability via RTE_ETH_DEV_CAPA_RXQ_SHARE.
PMD is responsible for shared Rx queue consistency checks to avoid member port's configuration contradict each other.
Signed-off-by Xueming Li <[email protected]>
Reviewed-by Andrew Rybchenko <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

Quoted 155 lines

---
 doc/guides/nics/features.rst                  | 13 ++++++++++
 doc/guides/nics/features/default.ini          |  1 +
 .../prog_guide/switch_representation.rst      | 11 +++++++++
 doc/guides/rel_notes/release_21_11.rst        |  6 +++++
 lib/ethdev/rte_ethdev.c                       |  8 +++++++
 lib/ethdev/rte_ethdev.h                       | 24 +++++++++++++++++++
 6 files changed, 63 insertions(+)

doc/guides/nics/features.rst +13 -0

index e346018e4b8..89f9accbca1 100644
@@ -615,6 +615,19 @@ Supports inner packet L4 checksum.
   ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_OUTER_UDP_CKSUM``.
 
 
+.. _nic_features_shared_rx_queue:
+
+Shared Rx queue
+---------------
+
+Supports shared Rx queue for ports in same Rx domain of a switch domain.
+
+* **[uses]     rte_eth_dev_info**: ``dev_capa:RTE_ETH_DEV_CAPA_RXQ_SHARE``.
+* **[uses]     rte_eth_dev_info，rte_eth_switch_info**: ``rx_domain``, ``domain_id``.
+* **[uses]     rte_eth_rxconf**: ``share_group``, ``share_qid``.
+* **[provides] mbuf**: ``mbuf.port``.
+
+
 .. _nic_features_packet_type_parsing:
 
 Packet type parsing

doc/guides/nics/features/default.ini +1 -0

index d473b94091a..93f5d1b46f4 100644
@@ -19,6 +19,7 @@ Free Tx mbuf on demand =
 Queue start/stop     =
 Runtime Rx queue setup =
 Runtime Tx queue setup =
+Shared Rx queue      =
 Burst mode info      =
 Power mgmt address monitor =
 MTU update           =

doc/guides/prog_guide/switch_representation.rst +11 -0

index ff6aa91c806..4f2532a91ea 100644
@@ -123,6 +123,17 @@ thought as a software "patch panel" front-end for applications.
 .. [1] `Ethernet switch device driver model (switchdev)
        <https://www.kernel.org/doc/Documentation/networking/switchdev.txt>`_
 
+- For some PMDs, memory usage of representors is huge when number of
+  representor grows, mbufs are allocated for each descriptor of Rx queue.
+  Polling large number of ports brings more CPU load, cache miss and
+  latency. Shared Rx queue can be used to share Rx queue between PF and
+  representors among same Rx domain. ``RTE_ETH_DEV_CAPA_RXQ_SHARE`` in
+  device info is used to indicate the capability. Setting non-zero share
+  group in Rx queue configuration to enable share, share_qid is used to
+  identify the shared Rx queue in group. Polling any member port can
+  receive packets of all member ports in the group, port ID is saved in
+  ``mbuf.port``.
+
 Basic SR-IOV
 ------------

doc/guides/rel_notes/release_21_11.rst +6 -0

index 3362c52a738..caf82242f2e 100644
@@ -75,6 +75,12 @@ New Features
     operations.
   * Added multi-process support.
 
+* **Added ethdev shared Rx queue support.**
+
+  * Added new device capability flag and Rx domain field to switch info.
+  * Added share group and share queue ID to Rx queue configuration.
+  * Added testpmd support and dedicate forwarding engine.
+
 * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.**
 
   Added macros ETH_RSS_IPV4_CHKSUM and ETH_RSS_L4_CHKSUM, now IPv4 and

lib/ethdev/rte_ethdev.c +8 -0

index 028907bc4b9..bc55f899f72 100644
@@ -2159,6 +2159,14 @@ rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id,
                return -EINVAL;
        }
 
+       if (local_conf.share_group > 0 &&
+           (dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE) == 0) {
+               RTE_ETHDEV_LOG(ERR,
+                       "Ethdev port_id=%d rx_queue_id=%d, enabled share_group=%hu while device doesn't support Rx queue share\n",
+                       port_id, rx_queue_id, local_conf.share_group);
+               return -EINVAL;
+       }
+
        /*
         * If LRO is enabled, check that the maximum aggregated packet
         * size is supported by the configured device.

lib/ethdev/rte_ethdev.h +24 -1

index 6d80514ba7a..34acc91273d 100644
@@ -1044,6 +1044,14 @@ struct rte_eth_rxconf {
        uint8_t rx_drop_en; /**< Drop packets if no descriptors are available. */
        uint8_t rx_deferred_start; /**< Do not start queue with rte_eth_dev_start(). */
        uint16_t rx_nseg; /**< Number of descriptions in rx_seg array. */
+       /**
+        * Share group index in Rx domain and switch domain.
+        * Non-zero value to enable Rx queue share, zero value disable share.
+        * PMD is responsible for Rx queue consistency checks to avoid member
+        * port's configuration contradict to each other.
+        */
+       uint16_t share_group;
+       uint16_t share_qid; /**< Shared Rx queue ID in group. */
        /**
         * Per-queue Rx offloads to be set using DEV_RX_OFFLOAD_* flags.
         * Only offloads set on rx_queue_offload_capa or rx_offload_capa

@@ -1445,6 +1453,16 @@ struct rte_eth_conf {
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /** Device supports Tx queue setup after device started. */
 #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
+/**
+ * Device supports shared Rx queue among ports within Rx domain and
+ * switch domain. Mbufs are consumed by shared Rx queue instead of
+ * each queue. Multiple groups are supported by share_group of Rx
+ * queue configuration. Shared Rx queue is identified by PMD using
+ * share_qid of Rx queue configuration. Polling any port in the group
+ * receive packets of all member ports, source port identified by
+ * mbuf->port field.
+ */
+#define RTE_ETH_DEV_CAPA_RXQ_SHARE              RTE_BIT64(2)
 /**@}*/
 
 /*

@@ -1488,6 +1506,12 @@ struct rte_eth_switch_info {
         * but each driver should explicitly define the mapping of switch
         * port identifier to that physical interconnect/switch
         */
+       /**
+        * Shared Rx queue sub-domain boundary. Only ports in same Rx domain
+        * and switch domain can share Rx queue. Valid only if device advertised
+        * RTE_ETH_DEV_CAPA_RXQ_SHARE capability.
+        */
+       uint16_t rx_domain;
 };
 
 /**
--

2.33.0

Xueming Li

author

2021-10-20 07:54 UTC #178 lore raw

PATCH v11 2/7 ethdev: new API to resolve device capability name

In reply to Xueming Li

This patch adds API to return name of device capability.

Signed-off-by Xueming Li <[email protected]>

---
 lib/ethdev/rte_ethdev.c | 25 +++++++++++++++++++++++++
 lib/ethdev/rte_ethdev.h | 14 ++++++++++++++
 lib/ethdev/version.map  |  3 +++
 3 files changed, 42 insertions(+)

lib/ethdev/rte_ethdev.c +25 -0

index bc55f899f72..d1a4a0405d6 100644
@@ -165,6 +165,15 @@ static const struct {
 
 #undef RTE_TX_OFFLOAD_BIT2STR
 
+static const struct {
+	uint64_t offload;
+	const char *name;
+} rte_eth_dev_capa_names[] = {
+	{RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP, "RUNTIME_RX_QUEUE_SETUP"},
+	{RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP, "RUNTIME_TX_QUEUE_SETUP"},
+	{RTE_ETH_DEV_CAPA_RXQ_SHARE, "RXQ_SHARE"},
+};
+
 /**
  * The user application callback description.
  *

@@ -1260,6 +1269,22 @@ rte_eth_dev_tx_offload_name(uint64_t offload)
 	return name;
 }
 
+const char *
+rte_eth_dev_capability_name(uint64_t capability)
+{
+	const char *name = "UNKNOWN";
+	unsigned int i;
+
+	for (i = 0; i < RTE_DIM(rte_eth_dev_capa_names); ++i) {
+		if (capability == rte_eth_dev_capa_names[i].offload) {
+			name = rte_eth_dev_capa_names[i].name;
+			break;
+		}
+	}
+
+	return name;
+}
+
 static inline int
 eth_dev_check_lro_pkt_size(uint16_t port_id, uint32_t config_size,
 		   uint32_t max_rx_pkt_len, uint32_t dev_info_size)

lib/ethdev/rte_ethdev.h +14 -0

index 34acc91273d..df8ef9382a9 100644
@@ -2109,6 +2109,20 @@ const char *rte_eth_dev_rx_offload_name(uint64_t offload);
  */
 const char *rte_eth_dev_tx_offload_name(uint64_t offload);
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get RTE_ETH_DEV_CAPA_* flag name.
+ *
+ * @param capability
+ *   Capability flag.
+ * @return
+ *   Capability name or 'UNKNOWN' if the flag cannot be recognized.
+ */
+__rte_experimental
+const char *rte_eth_dev_capability_name(uint64_t capability);
+
 /**
  * Configure an Ethernet device.
  * This function must be invoked first before any other function in the

lib/ethdev/version.map +3 -0

index efd729c0f2d..e1d403dd357 100644
@@ -245,6 +245,9 @@ EXPERIMENTAL {
 	rte_mtr_meter_policy_delete;
 	rte_mtr_meter_policy_update;
 	rte_mtr_meter_policy_validate;
+
+	# added in 21.11
+	rte_eth_dev_capability_name;
 };
 
 INTERNAL {

Signature · 1 line

2.33.0

Andrew Rybchenko

Reviewed-by on v11 2/7

2021-10-20 10:52 UTC #179 lore raw

Re: [dpdk-dev] [PATCH v11 2/7] ethdev: new API to resolve device capability name

Quoted 3 lines · Xueming Li

This patch adds API to return name of device capability.
Signed-off-by Xueming Li <[email protected]>

Reviewed-by Andrew Rybchenko <[email protected]>

Ajit Khaparde

Acked-by

2021-10-20 17:16 UTC #180 lore raw

Re: [dpdk-dev] [PATCH v11 2/7] ethdev: new API to resolve device capability name

Quoted 7 lines · Andrew Rybchenko

Quoted 3 lines · Xueming Li
This patch adds API to return name of device capability.
Signed-off-by Xueming Li <[email protected]>
Reviewed-by Andrew Rybchenko <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

Quoted 1 line

Thomas Monjalon

Acked-by on v11 2/7

2021-10-20 18:42 UTC #181 lore raw

Re: [dpdk-dev] [PATCH v11 2/7] ethdev: new API to resolve device capability name

In reply to Xueming Li

20/10/2021 09:53, Xueming Li:

Quoted 3 lines

This patch adds API to return name of device capability.
Signed-off-by Xueming Li <[email protected]>

The title of this patch should be:
"ethdev: get device capability name as string"

Acked-by Thomas Monjalon <[email protected]>

Xueming Li

author

2021-10-20 07:54 UTC #182 lore raw

PATCH v11 3/7 app/testpmd: dump device capability and Rx domain info

In reply to Xueming Li

Dump device capability and Rx domain ID if shared Rx queue is supported by device.

Signed-off-by Xueming Li <[email protected]>

Acked-by Andrew Rybchenko <[email protected]>

---
 app/test-pmd/config.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

app/test-pmd/config.c +29 -0

index 9c66329e96e..2c1b06c544d 100644
@@ -582,6 +582,29 @@ device_infos_display(const char *identifier)
 	rte_devargs_reset(&da);
 }
 
+static void
+print_dev_capabilities(uint64_t capabilities)
+{
+	uint64_t single_capa;
+	int begin;
+	int end;
+	int bit;
+
+	if (capabilities == 0)
+		return;
+
+	begin = __builtin_ctzll(capabilities);
+	end = sizeof(capabilities) * CHAR_BIT - __builtin_clzll(capabilities);
+
+	single_capa = 1ULL << begin;
+	for (bit = begin; bit < end; bit++) {
+		if (capabilities & single_capa)
+			printf(" %s",
+			       rte_eth_dev_capability_name(single_capa));
+		single_capa <<= 1;
+	}
+}
+
 void
 port_infos_display(portid_t port_id)
 {

@@ -733,6 +756,9 @@ port_infos_display(portid_t port_id)
 	printf("Max segment number per MTU/TSO: %hu\n",
 		dev_info.tx_desc_lim.nb_mtu_seg_max);
 
+	printf("Device capabilities: 0x%"PRIx64"(", dev_info.dev_capa);
+	print_dev_capabilities(dev_info.dev_capa);
+	printf(" )\n");
 	/* Show switch info only if valid switch domain and port id is set */
 	if (dev_info.switch_info.domain_id !=
 		RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID) {

@@ -743,6 +769,9 @@ port_infos_display(portid_t port_id)
 			dev_info.switch_info.domain_id);
 		printf("Switch Port Id: %u\n",
 			dev_info.switch_info.port_id);
+		if ((dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE) != 0)
+			printf("Switch Rx domain: %u\n",
+			       dev_info.switch_info.rx_domain);
 	}
 }

Signature · 1 line

2.33.0

Li, Xiaoyun

Acked-by on v11 3/7

2021-10-21 03:24 UTC #183 lore raw

Re: [dpdk-dev] [PATCH v11 3/7] app/testpmd: dump device capability and Rx domain info

Quoted 14 lines

-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Wednesday, October 20, 2021 15:53
To: [email protected]; Zhang, Yuying <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>; Li, Xiaoyun <[email protected]>
Subject: [PATCH v11 3/7] app/testpmd: dump device capability and Rx domain info

Quoted 9 lines

Dump device capability and Rx domain ID if shared Rx queue is supported by device.
Signed-off-by Xueming Li <[email protected]>
Acked-by Andrew Rybchenko <[email protected]>
---
 app/test-pmd/config.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

Acked-by Xiaoyun Li <[email protected]>

Ajit Khaparde

Acked-by

2021-10-21 03:29 UTC #184 lore raw

Re: [dpdk-dev] [PATCH v11 3/7] app/testpmd: dump device capability and Rx domain info

Quoted 18 lines · Xiaoyun

Quoted 18 lines
-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Wednesday, October 20, 2021 15:53
To: [email protected]; Zhang, Yuying <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>; Li, Xiaoyun <[email protected]>
Subject: [PATCH v11 3/7] app/testpmd: dump device capability and Rx domain info
Dump device capability and Rx domain ID if shared Rx queue is supported by device.

Quoted 8 lines

Quoted 6 lines
Signed-off-by Xueming Li <[email protected]>
Acked-by Andrew Rybchenko <[email protected]>
---
 app/test-pmd/config.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
Acked-by Xiaoyun Li <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

Xueming Li

author

2021-10-20 07:54 UTC #185 lore raw

PATCH v11 4/7 app/testpmd: new parameter to enable shared Rx queue

In reply to Xueming Li

Adds "--rxq-share=X" parameter to enable shared RxQ, share if device supports, otherwise fallback to standard RxQ.

Share group number grows per X ports. X defaults to MAX, implies all ports join share group 1. Queue ID is mapped equally with shared Rx queue ID.

Forwarding engine "shared-rxq" should be used which Rx only and update stream statistics correctly.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/config.c                 |  7 ++++++-
 app/test-pmd/parameters.c             | 13 +++++++++++++
 app/test-pmd/testpmd.c                | 20 +++++++++++++++++---
 app/test-pmd/testpmd.h                |  2 ++
 doc/guides/testpmd_app_ug/run_app.rst |  7 +++++++
 5 files changed, 45 insertions(+), 4 deletions(-)

app/test-pmd/config.c +6 -1

index 2c1b06c544d..fa951a86704 100644
@@ -2738,7 +2738,12 @@ rxtx_config_display(void)
 			printf("      RX threshold registers: pthresh=%d hthresh=%d "
 				" wthresh=%d\n",
 				pthresh_tmp, hthresh_tmp, wthresh_tmp);
-			printf("      RX Offloads=0x%"PRIx64"\n", offloads_tmp);
+			printf("      RX Offloads=0x%"PRIx64, offloads_tmp);
+			if (rx_conf->share_group > 0)
+				printf(" share_group=%u share_qid=%u",
+				       rx_conf->share_group,
+				       rx_conf->share_qid);
+			printf("\n");
 		}
 
 		/* per tx queue config only for first queue to be less verbose */

app/test-pmd/parameters.c +13 -0

index 3f94a82e321..30dae326310 100644
@@ -167,6 +167,7 @@ usage(char* progname)
 	printf("  --tx-ip=src,dst: IP addresses in Tx-only mode\n");
 	printf("  --tx-udp=src[,dst]: UDP ports in Tx-only mode\n");
 	printf("  --eth-link-speed: force link speed.\n");
+	printf("  --rxq-share: number of ports per shared rxq groups, defaults to MAX(1 group)\n");
 	printf("  --disable-link-check: disable check on link status when "
 	       "starting/stopping ports.\n");
 	printf("  --disable-device-start: do not automatically start port\n");

@@ -607,6 +608,7 @@ launch_args_parse(int argc, char** argv)
 		{ "rxpkts",			1, 0, 0 },
 		{ "txpkts",			1, 0, 0 },
 		{ "txonly-multi-flow",		0, 0, 0 },
+		{ "rxq-share",			2, 0, 0 },
 		{ "eth-link-speed",		1, 0, 0 },
 		{ "disable-link-check",		0, 0, 0 },
 		{ "disable-device-start",	0, 0, 0 },

@@ -1271,6 +1273,17 @@ launch_args_parse(int argc, char** argv)
 			}
 			if (!strcmp(lgopts[opt_idx].name, "txonly-multi-flow"))
 				txonly_multi_flow = 1;
+			if (!strcmp(lgopts[opt_idx].name, "rxq-share")) {
+				if (optarg == NULL) {
+					rxq_share = UINT32_MAX;
+				} else {
+					n = atoi(optarg);
+					if (n >= 0)
+						rxq_share = (uint32_t)n;
+					else
+						rte_exit(EXIT_FAILURE, "rxq-share must be >= 0\n");
+				}
+			}
 			if (!strcmp(lgopts[opt_idx].name, "no-flush-rx"))
 				no_flush_rx = 1;
 			if (!strcmp(lgopts[opt_idx].name, "eth-link-speed")) {

app/test-pmd/testpmd.c +17 -3

index 97ae52e17ec..123142ed110 100644
@@ -498,6 +498,11 @@ uint8_t record_core_cycles;
  */
 uint8_t record_burst_stats;
 
+/*
+ * Number of ports per shared Rx queue group, 0 disable.
+ */
+uint32_t rxq_share;
+
 unsigned int num_sockets = 0;
 unsigned int socket_ids[RTE_MAX_NUMA_NODES];

@@ -3393,14 +3398,23 @@ dev_event_callback(const char *device_name, enum rte_dev_event_type type,
 }
 
 static void
-rxtx_port_config(struct rte_port *port)
+rxtx_port_config(portid_t pid)
 {
 	uint16_t qid;
 	uint64_t offloads;
+	struct rte_port *port = &ports[pid];
 
 	for (qid = 0; qid < nb_rxq; qid++) {
 		offloads = port->rx_conf[qid].offloads;
 		port->rx_conf[qid] = port->dev_info.default_rxconf;
+
+		if (rxq_share > 0 &&
+		    (port->dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)) {
+			/* Non-zero share group to enable RxQ share. */
+			port->rx_conf[qid].share_group = pid / rxq_share + 1;
+			port->rx_conf[qid].share_qid = qid; /* Equal mapping. */
+		}
+
 		if (offloads != 0)
 			port->rx_conf[qid].offloads = offloads;

@@ -3558,7 +3572,7 @@ init_port_config(void)
 				port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_NONE;
 		}
 
-		rxtx_port_config(port);
+		rxtx_port_config(pid);
 
 		ret = eth_macaddr_get_print_err(pid, &port->eth_addr);
 		if (ret != 0)

@@ -3772,7 +3786,7 @@ init_port_dcb_config(portid_t pid,
 
 	memcpy(&rte_port->dev_conf, &port_conf, sizeof(struct rte_eth_conf));
 
-	rxtx_port_config(rte_port);
+	rxtx_port_config(pid);
 	/* VLAN filter */
 	rte_port->dev_conf.rxmode.offloads |= DEV_RX_OFFLOAD_VLAN_FILTER;
 	for (i = 0; i < RTE_DIM(vlan_tags); i++)

app/test-pmd/testpmd.h +2 -0

index 5863b2f43f3..3dfaaad94c0 100644
@@ -477,6 +477,8 @@ extern enum tx_pkt_split tx_pkt_split;
 
 extern uint8_t txonly_multi_flow;
 
+extern uint32_t rxq_share;
+
 extern uint16_t nb_pkt_per_burst;
 extern uint16_t nb_pkt_flowgen_clones;
 extern int nb_flows_flowgen;

doc/guides/testpmd_app_ug/run_app.rst +7 -0

index 640eadeff73..ff5908dcd50 100644
@@ -389,6 +389,13 @@ The command line options are:
 
     Generate multiple flows in txonly mode.
 
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Group number grows per X ports. X defaults to MAX, implies all ports
+    join share group 1. Forwarding engine "shared-rxq" should be used
+    which Rx only and update stream statistics correctly.
+
 *   ``--eth-link-speed``
 
     Set a forced link speed to the ethernet port::

Signature · 1 line

2.33.0

Ajit Khaparde

2021-10-20 17:30 UTC #186 lore raw

Re: [dpdk-dev] [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue

Quoted 11 lines · Xueming Li

Adds "--rxq-share=X" parameter to enable shared RxQ, share if device supports, otherwise fallback to standard RxQ.
Share group number grows per X ports. X defaults to MAX, implies all ports join share group 1. Queue ID is mapped equally with shared Rx queue ID.
Forwarding engine "shared-rxq" should be used which Rx only and update stream statistics correctly.

Quoted 9 lines

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/config.c                 |  7 ++++++-
 app/test-pmd/parameters.c             | 13 +++++++++++++
 app/test-pmd/testpmd.c                | 20 +++++++++++++++++---
 app/test-pmd/testpmd.h                |  2 ++
 doc/guides/testpmd_app_ug/run_app.rst |  7 +++++++
 5 files changed, 45 insertions(+), 4 deletions(-)

:::snip::::

Quoted 9 lines

app/test-pmd/testpmd.h +1 -0

+
 extern uint16_t nb_pkt_per_burst;
 extern uint16_t nb_pkt_flowgen_clones;
 extern int nb_flows_flowgen;

doc/guides/testpmd_app_ug/run_app.rst +0 -0

index 640eadeff73..ff5908dcd50 100644

doc/guides/testpmd_app_ug/run_app.rst The command line options are: · line 397

Quoted 9 lines

doc/guides/testpmd_app_ug/run_app.rst +6 -0

@@ -389,6 +389,13 @@ The command line options are:
 
     Generate multiple flows in txonly mode.
 
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Group number grows per X ports. X defaults to MAX, implies all ports
+    join share group 1. Forwarding engine "shared-rxq" should be used
+    which Rx only and update stream statistics correctly.

Did you mean "with Rx only"?
Something like this?
"shared-rxq" should be used in Rx only mode.

If you say - "the Forwarding engine should update stream statistics correctly", I think that is expected anyway? So there is no need to mention that in the guide.

Quoted 7 lines

doc/guides/testpmd_app_ug/run_app.rst +1 -1

+
 *   ``--eth-link-speed``
 
     Set a forced link speed to the ethernet port::
--

2.33.0

Thomas Monjalon

2021-10-20 19:14 UTC #187 lore raw

Re: [dpdk-dev] [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue

20/10/2021 19:29, Ajit Khaparde:

Quoted 3 lines · Xueming Li

Quoted 2 lines
Adds "--rxq-share=X" parameter to enable shared RxQ,

You should end the sentence here.

Quoted 4 lines

Quoted 4 lines
share if device
supports, otherwise fallback to standard RxQ.
Share group number grows per X ports.

Do you mean "Shared queues are grouped per X ports." ?

Quoted 6 lines

Quoted 6 lines
X defaults to MAX, implies all
ports join share group 1. Queue ID is mapped equally with shared Rx queue ID.
Forwarding engine "shared-rxq" should be used which Rx only and update stream statistics correctly.

I suggest this wording:
"
A new forwarding engine "shared-rxq" should be used for shared Rx queues. This engine does Rx only and update stream statistics accordingly. "

Quoted 1 line

Quoted 1 line
Signed-off-by Xueming Li <[email protected]>

[...]

app/test-pmd/parameters.c usage(char* progname) · line 170

Quoted 1 line

app/test-pmd/parameters.c +1 -0

+	printf("  --rxq-share: number of ports per shared rxq groups, defaults to MAX(1 group)\n");

rxq -> Rx queue
Is MAX a special value? or should it be "all queues"?
Note: space is missing before the parenthesis.

[...]

Quoted 4 lines

doc/guides/testpmd_app_ug/run_app.rst The command line options are:
Quoted 4 lines
doc/guides/testpmd_app_ug/run_app.rst +4 -0
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Group number grows per X ports.

Again I suggest "Shared queues are grouped per X ports."

Quoted 11 lines

doc/guides/testpmd_app_ug/run_app.rst The command line options are: · line 397
Quoted 3 lines
+ X defaults to MAX, implies all ports
doc/guides/testpmd_app_ug/run_app.rst +2 -0
+    join share group 1. Forwarding engine "shared-rxq" should be used
+    which Rx only and update stream statistics correctly.
Did you mean "with Rx only"?
Something like this?
"shared-rxq" should be used in Rx only mode.
If you say - "the Forwarding engine should update stream statistics correctly", I think that is expected anyway? So there is no need to mention that in the guide.

I suggested a wording above.

Xueming(Steven) Li

author

2021-10-21 04:09 UTC #188 lore raw

Re: [dpdk-dev] [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue

Quoted 52 lines · Thomas Monjalon

20/10/2021 19:29, Ajit Khaparde:
Quoted 3 lines · Xueming Li
Quoted 2 lines
Adds "--rxq-share=X" parameter to enable shared RxQ,
You should end the sentence here.
Quoted 4 lines
Quoted 4 lines
share if device
supports, otherwise fallback to standard RxQ.
Share group number grows per X ports.
Do you mean "Shared queues are grouped per X ports." ?
Quoted 6 lines
Quoted 6 lines
X defaults to MAX, implies all
ports join share group 1. Queue ID is mapped equally with shared Rx queue ID.
Forwarding engine "shared-rxq" should be used which Rx only and update stream statistics correctly.
I suggest this wording:
"
A new forwarding engine "shared-rxq" should be used for shared Rx queues. This engine does Rx only and update stream statistics accordingly. "
Quoted 1 line
Quoted 1 line
Signed-off-by Xueming Li <[email protected]>
[...]
app/test-pmd/parameters.c usage(char* progname) · line 170
Quoted 1 line
app/test-pmd/parameters.c +1 -0
+	printf("  --rxq-share: number of ports per shared rxq groups, defaults to MAX(1 group)\n");
rxq -> Rx queue
Is MAX a special value? or should it be "all queues"?
Note: space is missing before the parenthesis.
[...]
Quoted 4 lines
doc/guides/testpmd_app_ug/run_app.rst The command line options are:
Quoted 4 lines
doc/guides/testpmd_app_ug/run_app.rst +4 -0
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Group number grows per X ports.
Again I suggest "Shared queues are grouped per X ports."
Quoted 8 lines
doc/guides/testpmd_app_ug/run_app.rst The command line options are: · line 397
Quoted 3 lines
+ X defaults to MAX, implies all ports
doc/guides/testpmd_app_ug/run_app.rst +2 -0
+    join share group 1. Forwarding engine "shared-rxq" should be used
+    which Rx only and update stream statistics correctly.
Did you mean "with Rx only"?
Something like this?
"shared-rxq" should be used in Rx only mode.

Quoted 6 lines

Quoted 3 lines
If you say - "the Forwarding engine should update stream statistics correctly", I think that is expected anyway? So there is no need to mention that in the guide.
I suggested a wording above.

Looks good, thanks Ajit and Thomas!

Quoted 2 lines

Xueming(Steven) Li

author

2021-10-21 03:49 UTC #189 lore raw

Re: [dpdk-dev] [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue

In reply to Ajit Khaparde

Quoted 45 lines · Ajit Khaparde

Quoted 11 lines · Xueming Li
Adds "--rxq-share=X" parameter to enable shared RxQ, share if device supports, otherwise fallback to standard RxQ.
Share group number grows per X ports. X defaults to MAX, implies all ports join share group 1. Queue ID is mapped equally with shared Rx queue ID.
Forwarding engine "shared-rxq" should be used which Rx only and update stream statistics correctly.
Quoted 9 lines
Signed-off-by Xueming Li <[email protected]>
---
 app/test-pmd/config.c                 |  7 ++++++-
 app/test-pmd/parameters.c             | 13 +++++++++++++
 app/test-pmd/testpmd.c                | 20 +++++++++++++++++---
 app/test-pmd/testpmd.h                |  2 ++
 doc/guides/testpmd_app_ug/run_app.rst |  7 +++++++
 5 files changed, 45 insertions(+), 4 deletions(-)
:::snip::::
Quoted 9 lines
app/test-pmd/testpmd.h +1 -0
+
 extern uint16_t nb_pkt_per_burst;
 extern uint16_t nb_pkt_flowgen_clones;
 extern int nb_flows_flowgen;
doc/guides/testpmd_app_ug/run_app.rst +0 -0
index 640eadeff73..ff5908dcd50 100644
doc/guides/testpmd_app_ug/run_app.rst The command line options are: · line 397
Quoted 9 lines
doc/guides/testpmd_app_ug/run_app.rst +6 -0
@@ -389,6 +389,13 @@ The command line options are:
 
     Generate multiple flows in txonly mode.
 
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Group number grows per X ports. X defaults to MAX, implies all ports
+    join share group 1. Forwarding engine "shared-rxq" should be used
+    which Rx only and update stream statistics correctly.
Did you mean "with Rx only"?
Something like this?
"shared-rxq" should be used in Rx only mode.

Quoted 3 lines

If you say - "the Forwarding engine should update stream statistics correctly", I think that is expected anyway? So there is no need to mention that in the guide.

I will change like this:
"shared-rxq" should be used, other forwarding engines can't resolve source stream correctly, statistics and forwarding target could be wrong.

Quoted 9 lines

Quoted 9 lines
doc/guides/testpmd_app_ug/run_app.rst +1 -1
+
 *   ``--eth-link-speed``
 
     Set a forced link speed to the ethernet port::
--
2.33.0

Li, Xiaoyun

2021-10-21 03:24 UTC #190 lore raw

Re: [dpdk-dev] [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue

In reply to Xueming Li

Quoted 14 lines

-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Wednesday, October 20, 2021 15:53
To: [email protected]; Zhang, Yuying <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>; Li, Xiaoyun <[email protected]>
Subject: [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue

Quoted 4 lines

Adds "--rxq-share=X" parameter to enable shared RxQ, share if device supports, otherwise fallback to standard RxQ.
Share group number grows per X ports. X defaults to MAX, implies all ports join

X defaults to number of probed ports.

Quoted 18 lines

share group 1. Queue ID is mapped equally with shared Rx queue ID.
Forwarding engine "shared-rxq" should be used which Rx only and update stream statistics correctly.
Signed-off-by Xueming Li <[email protected]>
---
 app/test-pmd/config.c                 |  7 ++++++-
 app/test-pmd/parameters.c             | 13 +++++++++++++
 app/test-pmd/testpmd.c                | 20 +++++++++++++++++---
 app/test-pmd/testpmd.h                |  2 ++
 doc/guides/testpmd_app_ug/run_app.rst |  7 +++++++
 5 files changed, 45 insertions(+), 4 deletions(-)
app/test-pmd/config.c index renamed from app/test-pmd/config.c +0 -0
2c1b06c544d..fa951a86704 100644
app/test-pmd/config.c +0 -0

<snip>

app/test-pmd/config.c launch_args_parse(int argc, char** argv) · line 1278

Quoted 7 lines

app/test-pmd/config.c +3 -0

@@ -1271,6 +1273,17 @@ launch_args_parse(int argc, char** argv)
 			}
 			if (!strcmp(lgopts[opt_idx].name, "txonly-multi-flow"))
 				txonly_multi_flow = 1;
+			if (!strcmp(lgopts[opt_idx].name, "rxq-share")) {
+				if (optarg == NULL) {
+					rxq_share = UINT32_MAX;

Why not use "nb_ports" here? nb_ports is the number of probed ports.

Quoted 12 lines

app/test-pmd/config.c +8 -0

+				} else {
+					n = atoi(optarg);
+					if (n >= 0)
+						rxq_share = (uint32_t)n;
+					else
+						rte_exit(EXIT_FAILURE, "rxq-
 share must be >= 0\n");
+				}
+			}
 			if (!strcmp(lgopts[opt_idx].name, "no-flush-rx"))
 				no_flush_rx = 1;

if (!strcmp(lgopts[opt_idx].name, "eth-link-speed"))

<snip>

app/test-pmd/config.c

Quoted 5 lines

app/test-pmd/config.c +4 -0

+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Group number grows per X ports. X defaults to MAX, implies all ports

X defaults to number of probed ports.
I suppose this is what you mean? Also, I agree with other comments with the wording part.

Quoted 8 lines

app/test-pmd/config.c +3 -1

+    join share group 1. Forwarding engine "shared-rxq" should be used
+    which Rx only and update stream statistics correctly.
+
 *   ``--eth-link-speed``
 
     Set a forced link speed to the ethernet port::
--

2.33.0

Xueming(Steven) Li

author

2021-10-21 03:58 UTC #191 lore raw

Re: [dpdk-dev] [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue

Quoted 16 lines · Xiaoyun

Hi
Quoted 14 lines
-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Wednesday, October 20, 2021 15:53
To: [email protected]; Zhang, Yuying <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>; Li, Xiaoyun <[email protected]>
Subject: [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue

Quoted 6 lines

Quoted 4 lines
Adds "--rxq-share=X" parameter to enable shared RxQ, share if device supports, otherwise fallback to standard RxQ.
Share group number grows per X ports. X defaults to MAX, implies all ports join
X defaults to number of probed ports.

I will change to UINT32_MAX, thanks.

Quoted 20 lines

Quoted 19 lines
share group 1. Queue ID is mapped equally with shared Rx queue ID.
Forwarding engine "shared-rxq" should be used which Rx only and update stream statistics correctly.
Signed-off-by Xueming Li <[email protected]>
---
 app/test-pmd/config.c                 |  7 ++++++-
 app/test-pmd/parameters.c             | 13 +++++++++++++
 app/test-pmd/testpmd.c                | 20 +++++++++++++++++---
 app/test-pmd/testpmd.h                |  2 ++
 doc/guides/testpmd_app_ug/run_app.rst |  7 +++++++
 5 files changed, 45 insertions(+), 4 deletions(-)
app/test-pmd/config.c index renamed from app/test-pmd/config.c +0 -0
2c1b06c544d..fa951a86704 100644
app/test-pmd/config.c +0 -0
<snip>

Quoted 9 lines

app/test-pmd/config.c launch_args_parse(int argc, char** argv) · line 1278
Quoted 7 lines
app/test-pmd/config.c +3 -0
@@ -1271,6 +1273,17 @@ launch_args_parse(int argc, char** argv)
 			}
 			if (!strcmp(lgopts[opt_idx].name, "txonly-multi-flow"))
 				txonly_multi_flow = 1;
+			if (!strcmp(lgopts[opt_idx].name, "rxq-share")) {
+				if (optarg == NULL) {
+					rxq_share = UINT32_MAX;
Why not use "nb_ports" here? nb_ports is the number of probed ports.

Considering hotplug, nb_ports could grow later, I think UINT32_MAX is safe.

Quoted 32 lines

Quoted 13 lines
app/test-pmd/config.c +8 -0
+				} else {
+					n = atoi(optarg);
+					if (n >= 0)
+						rxq_share = (uint32_t)n;
+					else
+						rte_exit(EXIT_FAILURE, "rxq-
 share must be >= 0\n");
+				}
+			}
 			if (!strcmp(lgopts[opt_idx].name, "no-flush-rx"))
 				no_flush_rx = 1;
if (!strcmp(lgopts[opt_idx].name, "eth-link-speed"))
<snip>
app/test-pmd/config.c
Quoted 5 lines
app/test-pmd/config.c +4 -0
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Group number grows per X ports. X defaults to MAX, implies all ports
X defaults to number of probed ports.
I suppose this is what you mean? Also, I agree with other comments with the wording part
Quoted 8 lines
app/test-pmd/config.c +3 -1
+    join share group 1. Forwarding engine "shared-rxq" should be used
+    which Rx only and update stream statistics correctly.
+
 *   ``--eth-link-speed``
 
     Set a forced link speed to the ethernet port::
--
2.33.0

Li, Xiaoyun

2021-10-21 05:15 UTC #192 lore raw

Re: [dpdk-dev] [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue

Quoted 70 lines

-----Original Message-----
From: Xueming(Steven) Li <[email protected]>
Sent: Thursday, October 21, 2021 11:59
To: Li, Xiaoyun <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; [email protected]; NBU-Contact-Thomas Monjalon <[email protected]>; Slava Ovsiienko <[email protected]>; [email protected]; Yigit, Ferruh <[email protected]>; [email protected]; Lior Margalit <[email protected]>
Subject: Re: [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue
Quoted 16 lines · Xiaoyun
Hi
Quoted 14 lines
-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Wednesday, October 20, 2021 15:53
To: [email protected]; Zhang, Yuying <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>; Li, Xiaoyun <[email protected]>
Subject: [PATCH v11 4/7] app/testpmd: new parameter to enable shared Rx queue
Quoted 7 lines
Quoted 5 lines
Adds "--rxq-share=X" parameter to enable shared RxQ, share if device supports, otherwise fallback to standard RxQ.
Share group number grows per X ports. X defaults to MAX, implies all ports join
X defaults to number of probed ports.
I will change to UINT32_MAX, thanks.
Quoted 20 lines
Quoted 19 lines
share group 1. Queue ID is mapped equally with shared Rx queue ID.
Forwarding engine "shared-rxq" should be used which Rx only and update stream statistics correctly.
Signed-off-by Xueming Li <[email protected]>
---
 app/test-pmd/config.c                 |  7 ++++++-
 app/test-pmd/parameters.c             | 13 +++++++++++++
 app/test-pmd/testpmd.c                | 20 +++++++++++++++++---
 app/test-pmd/testpmd.h                |  2 ++
 doc/guides/testpmd_app_ug/run_app.rst |  7 +++++++
 5 files changed, 45 insertions(+), 4 deletions(-)
app/test-pmd/config.c index renamed from app/test-pmd/config.c +0 -0
2c1b06c544d..fa951a86704 100644
app/test-pmd/config.c +0 -0
<snip>
Quoted 9 lines
app/test-pmd/config.c launch_args_parse(int argc, char** argv) · line 1278
Quoted 7 lines
app/test-pmd/config.c +3 -0
@@ -1271,6 +1273,17 @@ launch_args_parse(int argc, char** argv)
 			}
 			if (!strcmp(lgopts[opt_idx].name, "txonly-multi-flow"))
 				txonly_multi_flow = 1;
+			if (!strcmp(lgopts[opt_idx].name, "rxq-share")) {
+				if (optarg == NULL) {
+					rxq_share = UINT32_MAX;
Why not use "nb_ports" here? nb_ports is the number of probed ports.

Quoted 1 line

Considering hotplug, nb_ports could grow later, I think UINT32_MAX is safe.

Yes. It will be safer if there's hotplug.
But I thought you won’t consider this case since if you consider about hotplug, your calculation for share_group using port_id is not correct.

		port->rx_conf[qid].share_group = pid / rxq_share + 1;

Quoted 35 lines

Quoted 35 lines
Quoted 14 lines
app/test-pmd/config.c +8 -0
+				} else {
+					n = atoi(optarg);
+					if (n >= 0)
+						rxq_share = (uint32_t)n;
+					else
+						rte_exit(EXIT_FAILURE, "rxq-
 share must be >= 0\n");
+				}
+			}
 			if (!strcmp(lgopts[opt_idx].name, "no-flush-rx"))
 				no_flush_rx = 1;
if (!strcmp(lgopts[opt_idx].name, "eth-link-speed"))
<snip>
app/test-pmd/config.c
Quoted 6 lines
app/test-pmd/config.c +5 -0
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Group number grows per X ports. X defaults to MAX, implies all
+ ports
X defaults to number of probed ports.
I suppose this is what you mean? Also, I agree with other comments with the wording part
Quoted 8 lines
app/test-pmd/config.c +3 -1
+    join share group 1. Forwarding engine "shared-rxq" should be used
+    which Rx only and update stream statistics correctly.
+
 *   ``--eth-link-speed``
 
     Set a forced link speed to the ethernet port::
--
2.33.0

Xueming Li

author

2021-10-20 07:55 UTC #193 lore raw

PATCH v11 5/7 app/testpmd: dump port info for shared Rx queue

In reply to Xueming Li

In case of shared Rx queue, polling any member port returns mbufs for all members. This patch dumps mbuf->port for each packet.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/util.c | 3 +++
 1 file changed, 3 insertions(+)

app/test-pmd/util.c +3 -0

index 51506e49404..e98f136d5ed 100644
@@ -100,6 +100,9 @@ dump_pkt_burst(uint16_t port_id, uint16_t queue, struct rte_mbuf *pkts[],
 		struct rte_flow_restore_info info = { 0, };
 
 		mb = pkts[i];
+		if (rxq_share > 0)
+			MKDUMPSTR(print_buf, buf_size, cur_len, "port %u, ",
+				  mb->port);
 		eth_hdr = rte_pktmbuf_read(mb, 0, sizeof(_eth_hdr), &_eth_hdr);
 		eth_type = RTE_BE_TO_CPU_16(eth_hdr->ether_type);
 		packet_type = mb->packet_type;

Signature · 1 line

2.33.0

Li, Xiaoyun

Acked-by on v11 5/7

2021-10-21 03:24 UTC #194 lore raw

Re: [dpdk-dev] [PATCH v11 5/7] app/testpmd: dump port info for shared Rx queue

Quoted 38 lines

-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Wednesday, October 20, 2021 15:53
To: [email protected]; Zhang, Yuying <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>; Li, Xiaoyun <[email protected]>
Subject: [PATCH v11 5/7] app/testpmd: dump port info for shared Rx queue
In case of shared Rx queue, polling any member port returns mbufs for all members. This patch dumps mbuf->port for each packet.
Signed-off-by Xueming Li <[email protected]>
---
 app/test-pmd/util.c | 3 +++
 1 file changed, 3 insertions(+)
app/test-pmd/util.c index renamed from app/test-pmd/util.c +0 -0
51506e49404..e98f136d5ed 100644
app/test-pmd/util.c +3 -0
@@ -100,6 +100,9 @@ dump_pkt_burst(uint16_t port_id, uint16_t queue,
 struct rte_mbuf *pkts[],
 		struct rte_flow_restore_info info = { 0, };
 
 		mb = pkts[i];
+		if (rxq_share > 0)
+			MKDUMPSTR(print_buf, buf_size, cur_len, "port %u, ",
+				  mb->port);
 		eth_hdr = rte_pktmbuf_read(mb, 0, sizeof(_eth_hdr),
 &_eth_hdr);
app/test-pmd/util.c +0 -1
 		eth_type = RTE_BE_TO_CPU_16(eth_hdr->ether_type);
 		packet_type = mb->packet_type;
--

Quoted 1 line

2.33.0

Acked-by Xiaoyun Li <[email protected]>

Xueming Li

author

2021-10-20 07:55 UTC #195 lore raw

PATCH v11 6/7 app/testpmd: force shared Rx queue polled on same core

In reply to Xueming Li

Shared Rx queue must be polled on same core. This patch checks and stops forwarding if shared RxQ being scheduled on multiple
cores.

It's suggested to use same number of Rx queues and polling cores.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/config.c  | 103 +++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.c |   4 +-
 app/test-pmd/testpmd.h |   2 +
 3 files changed, 108 insertions(+), 1 deletion(-)

app/test-pmd/config.c +103 -0

index fa951a86704..1f1307178be 100644
@@ -2915,6 +2915,109 @@ port_rss_hash_key_update(portid_t port_id, char rss_type[], uint8_t *hash_key,
 	}
 }
 
+/*
+ * Check whether a shared rxq scheduled on other lcores.
+ */
+static bool
+fwd_stream_on_other_lcores(uint16_t domain_id, lcoreid_t src_lc,
+			   portid_t src_port, queueid_t src_rxq,
+			   uint32_t share_group, queueid_t share_rxq)
+{
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/* Check remaining cores. */
+	for (lc_id = src_lc + 1; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)
+			    == 0)
+				/* Not shared rxq. */
+				continue;
+			if (domain_id != port->dev_info.switch_info.domain_id)
+				continue;
+			if (rxq_conf->share_group != share_group)
+				continue;
+			if (rxq_conf->share_qid != share_rxq)
+				continue;
+			printf("Shared Rx queue group %u queue %hu can't be scheduled on different cores:\n",
+			       share_group, share_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       src_lc, src_port, src_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       lc_id, fs->rx_port, fs->rx_queue);
+			printf("Please use --nb-cores=%hu to limit number of forwarding cores\n",
+			       nb_rxq);
+			return true;
+		}
+	}
+	return false;
+}
+
+/*
+ * Check shared rxq configuration.
+ *
+ * Shared group must not being scheduled on different core.
+ */
+bool
+pkt_fwd_shared_rxq_check(void)
+{
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	uint16_t domain_id;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/*
+	 * Check streams on each core, make sure the same switch domain +
+	 * group + queue doesn't get scheduled on other cores.
+	 */
+	for (lc_id = 0; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			/* Update lcore info stream being scheduled. */
+			fs->lcore = fwd_lcores[lc_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)
+			    == 0)
+				/* Not shared rxq. */
+				continue;
+			/* Check shared rxq not scheduled on remaining cores. */
+			domain_id = port->dev_info.switch_info.domain_id;
+			if (fwd_stream_on_other_lcores(domain_id, lc_id,
+						       fs->rx_port,
+						       fs->rx_queue,
+						       rxq_conf->share_group,
+						       rxq_conf->share_qid))
+				return false;
+		}
+	}
+	return true;
+}
+
 /*
  * Setup forwarding configuration for each logical core.
  */

app/test-pmd/testpmd.c +3 -1

index 123142ed110..f3f81ef561f 100644
@@ -2236,10 +2236,12 @@ start_packet_forwarding(int with_tx_first)
 
 	fwd_config_setup();
 
+	pkt_fwd_config_display(&cur_fwd_config);
+	if (!pkt_fwd_shared_rxq_check())
+		return;
 	if(!no_flush_rx)
 		flush_fwd_rx_queues();
 
-	pkt_fwd_config_display(&cur_fwd_config);
 	rxtx_config_display();
 
 	fwd_stats_reset();

app/test-pmd/testpmd.h +2 -0

index 3dfaaad94c0..f121a2da90c 100644
@@ -144,6 +144,7 @@ struct fwd_stream {
 	uint64_t     core_cycles; /**< used for RX and TX processing */
 	struct pkt_burst_stats rx_burst_stats;
 	struct pkt_burst_stats tx_burst_stats;
+	struct fwd_lcore *lcore; /**< Lcore being scheduled. */
 };
 
 /**

@@ -795,6 +796,7 @@ void port_summary_header_display(void);
 void rx_queue_infos_display(portid_t port_idi, uint16_t queue_id);
 void tx_queue_infos_display(portid_t port_idi, uint16_t queue_id);
 void fwd_lcores_config_display(void);
+bool pkt_fwd_shared_rxq_check(void);
 void pkt_fwd_config_display(struct fwd_config *cfg);
 void rxtx_config_display(void);
 void fwd_config_setup(void);

Signature · 1 line

2.33.0

Li, Xiaoyun

2021-10-21 03:25 UTC #196 lore raw

Re: [dpdk-dev] [PATCH v11 6/7] app/testpmd: force shared Rx queue polled on same core

Quoted 126 lines

-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Wednesday, October 20, 2021 15:53
To: [email protected]; Zhang, Yuying <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>; Li, Xiaoyun <[email protected]>
Subject: [PATCH v11 6/7] app/testpmd: force shared Rx queue polled on same core

Shared Rx queue must be polled on same core. This patch checks and stops forwarding if shared RxQ being scheduled on multiple cores.

It's suggested to use same number of Rx queues and polling cores.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/config.c  | 103
+++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.c |   4 +-
 app/test-pmd/testpmd.h |   2 +
 3 files changed, 108 insertions(+), 1 deletion(-)

app/test-pmd/config.c index renamed from app/test-pmd/config.c +0 -0

fa951a86704..1f1307178be 100644

app/test-pmd/config.c +44 -0

@@ -2915,6 +2915,109 @@ port_rss_hash_key_update(portid_t port_id, char
 rss_type[], uint8_t *hash_key,
 	}
 }
 
+/*
+ * Check whether a shared rxq scheduled on other lcores.
+ */
+static bool
+fwd_stream_on_other_lcores(uint16_t domain_id, lcoreid_t src_lc,
+			   portid_t src_port, queueid_t src_rxq,
+			   uint32_t share_group, queueid_t share_rxq) {
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/* Check remaining cores. */
+	for (lc_id = src_lc + 1; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa &
 RTE_ETH_DEV_CAPA_RXQ_SHARE)
+			    == 0)
+				/* Not shared rxq. */
+				continue;
+			if (domain_id != port->dev_info.switch_info.domain_id)
+				continue;
+			if (rxq_conf->share_group != share_group)
+				continue;
+			if (rxq_conf->share_qid != share_rxq)
+				continue;
+			printf("Shared Rx queue group %u queue %hu can't be
 scheduled on different cores:\n",
+			       share_group, share_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       src_lc, src_port, src_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       lc_id, fs->rx_port, fs->rx_queue);
+			printf("Please use --nb-cores=%hu to limit number of

forwarding cores\n",

app/test-pmd/config.c +42 -0

+			       nb_rxq);
+			return true;
+		}
+	}
+	return false;
+}
+
+/*
+ * Check shared rxq configuration.
+ *
+ * Shared group must not being scheduled on different core.
+ */
+bool
+pkt_fwd_shared_rxq_check(void)
+{
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	uint16_t domain_id;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/*
+	 * Check streams on each core, make sure the same switch domain +
+	 * group + queue doesn't get scheduled on other cores.
+	 */
+	for (lc_id = 0; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			/* Update lcore info stream being scheduled. */
+			fs->lcore = fwd_lcores[lc_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa &

RTE_ETH_DEV_CAPA_RXQ_SHARE)

app/test-pmd/config.c port_rss_hash_key_update(portid_t port_id, char rss_type[], uint8_t *hash_key,

Quoted 4 lines

app/test-pmd/config.c +4 -0

+			    == 0)
+				/* Not shared rxq. */
+				continue;
+			/* Check shared rxq not scheduled on remaining cores.

The check will be done anyway just if dev has the capability of share_rxq.
But what if user wants normal queue config when they are using the dev which has the share_q capability? You should limit the check only when "rxq_share > 0".

Quoted 21 lines

app/test-pmd/config.c +12 -0

+			domain_id = port->dev_info.switch_info.domain_id;
+			if (fwd_stream_on_other_lcores(domain_id, lc_id,
+						       fs->rx_port,
+						       fs->rx_queue,
+						       rxq_conf->share_group,
+						       rxq_conf->share_qid))
+				return false;
+		}
+	}
+	return true;
+}
+
 /*
  * Setup forwarding configuration for each logical core.
  */

app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

123142ed110..f3f81ef561f 100644

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2240

Quoted 5 lines

app/test-pmd/testpmd.c +2 -0

@@ -2236,10 +2236,12 @@ start_packet_forwarding(int with_tx_first)
 
 	fwd_config_setup();
 
+	pkt_fwd_config_display(&cur_fwd_config);
+	if (!pkt_fwd_shared_rxq_check())

Same comment as above
This check should only happens if user enables "--rxq-share=[X]". You can limit the check here too.
If (rxq_share > 0 && !pkt_fwd_shared_rxq_check())

Quoted 29 lines

app/test-pmd/testpmd.c +1 -1
+		return;
 	if(!no_flush_rx)
 		flush_fwd_rx_queues();
 
-	pkt_fwd_config_display(&cur_fwd_config);
 	rxtx_config_display();
 
 	fwd_stats_reset();
app/test-pmd/testpmd.h index renamed from app/test-pmd/testpmd.h +0 -0
3dfaaad94c0..f121a2da90c 100644
app/test-pmd/testpmd.h +1 -0
@@ -144,6 +144,7 @@ struct fwd_stream {
 	uint64_t     core_cycles; /**< used for RX and TX processing */
 	struct pkt_burst_stats rx_burst_stats;
 	struct pkt_burst_stats tx_burst_stats;
+	struct fwd_lcore *lcore; /**< Lcore being scheduled. */
 };
 
 /**
@@ -795,6 +796,7 @@ void port_summary_header_display(void);
 void rx_queue_infos_display(portid_t port_idi, uint16_t queue_id);  void
tx_queue_infos_display(portid_t port_idi, uint16_t queue_id); void fwd_lcores_config_display(void);
+bool pkt_fwd_shared_rxq_check(void);
void pkt_fwd_config_display(struct fwd_config *cfg); void
rxtx_config_display(void); void fwd_config_setup(void); --
2.33.0

Xueming(Steven) Li

author

2021-10-21 04:21 UTC #197 lore raw

Re: [dpdk-dev] [PATCH v11 6/7] app/testpmd: force shared Rx queue polled on same core

Quoted 127 lines · Xiaoyun

Quoted 125 lines

Shared Rx queue must be polled on same core. This patch checks and stops forwarding if shared RxQ being scheduled on multiple cores.

It's suggested to use same number of Rx queues and polling cores.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/config.c  | 103
+++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.c |   4 +-
 app/test-pmd/testpmd.h |   2 +
 3 files changed, 108 insertions(+), 1 deletion(-)

app/test-pmd/config.c index renamed from app/test-pmd/config.c +0 -0

fa951a86704..1f1307178be 100644

app/test-pmd/config.c +44 -0

@@ -2915,6 +2915,109 @@ port_rss_hash_key_update(portid_t port_id, char
 rss_type[], uint8_t *hash_key,
 	}
 }
 
+/*
+ * Check whether a shared rxq scheduled on other lcores.
+ */
+static bool
+fwd_stream_on_other_lcores(uint16_t domain_id, lcoreid_t src_lc,
+			   portid_t src_port, queueid_t src_rxq,
+			   uint32_t share_group, queueid_t share_rxq) {
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/* Check remaining cores. */
+	for (lc_id = src_lc + 1; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa &
 RTE_ETH_DEV_CAPA_RXQ_SHARE)
+			    == 0)
+				/* Not shared rxq. */
+				continue;
+			if (domain_id != port->dev_info.switch_info.domain_id)
+				continue;
+			if (rxq_conf->share_group != share_group)
+				continue;
+			if (rxq_conf->share_qid != share_rxq)
+				continue;
+			printf("Shared Rx queue group %u queue %hu can't be
 scheduled on different cores:\n",
+			       share_group, share_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       src_lc, src_port, src_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       lc_id, fs->rx_port, fs->rx_queue);
+			printf("Please use --nb-cores=%hu to limit number of

forwarding cores\n",

app/test-pmd/config.c +42 -0

+			       nb_rxq);
+			return true;
+		}
+	}
+	return false;
+}
+
+/*
+ * Check shared rxq configuration.
+ *
+ * Shared group must not being scheduled on different core.
+ */
+bool
+pkt_fwd_shared_rxq_check(void)
+{
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	uint16_t domain_id;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/*
+	 * Check streams on each core, make sure the same switch domain +
+	 * group + queue doesn't get scheduled on other cores.
+	 */
+	for (lc_id = 0; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			/* Update lcore info stream being scheduled. */
+			fs->lcore = fwd_lcores[lc_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa &

Quoted 8 lines

app/test-pmd/config.c port_rss_hash_key_update(portid_t port_id, char rss_type[], uint8_t *hash_key,
Quoted 5 lines
RTE_ETH_DEV_CAPA_RXQ_SHARE)
app/test-pmd/config.c +4 -0
+			    == 0)
+				/* Not shared rxq. */
+				continue;
+			/* Check shared rxq not scheduled on remaining cores.
The check will be done anyway just if dev has the capability of share_rxq.
But what if user wants normal queue config when they are using the dev which has the share_q capability?

Good catch, thanks!

Quoted 1 line

You should limit the check only when "rxq_share > 0".

Yes, will add this at top of this function.

Quoted 25 lines

app/test-pmd/config.c +12 -0

+			domain_id = port->dev_info.switch_info.domain_id;
+			if (fwd_stream_on_other_lcores(domain_id, lc_id,
+						       fs->rx_port,
+						       fs->rx_queue,
+						       rxq_conf->share_group,
+						       rxq_conf->share_qid))
+				return false;
+		}
+	}
+	return true;
+}
+
 /*
  * Setup forwarding configuration for each logical core.
  */

app/test-pmd/testpmd.c index renamed from app/test-pmd/testpmd.c +0 -0

123142ed110..f3f81ef561f 100644

app/test-pmd/testpmd.c +0 -0

@@ -2236,10 +2236,12 @@ start_packet_forwarding(int with_tx_first)
 
 	fwd_config_setup();

Quoted 7 lines

app/test-pmd/testpmd.c start_packet_forwarding(int with_tx_first) · line 2240
Quoted 2 lines
app/test-pmd/testpmd.c +2 -0
+	pkt_fwd_config_display(&cur_fwd_config);
+	if (!pkt_fwd_shared_rxq_check())
Same comment as above
This check should only happens if user enables "--rxq-share=[X]". You can limit the check here too.
If (rxq_share > 0 && !pkt_fwd_shared_rxq_check())

I will add rxq_share > 0 check at begining of
pk_fwd_shared_rxq_check(), thanks!

Quoted 31 lines

Quoted 30 lines
app/test-pmd/testpmd.c +1 -1
+		return;
 	if(!no_flush_rx)
 		flush_fwd_rx_queues();
 
-	pkt_fwd_config_display(&cur_fwd_config);
 	rxtx_config_display();
 
 	fwd_stats_reset();
app/test-pmd/testpmd.h index renamed from app/test-pmd/testpmd.h +0 -0
3dfaaad94c0..f121a2da90c 100644
app/test-pmd/testpmd.h +1 -0
@@ -144,6 +144,7 @@ struct fwd_stream {
 	uint64_t     core_cycles; /**< used for RX and TX processing */
 	struct pkt_burst_stats rx_burst_stats;
 	struct pkt_burst_stats tx_burst_stats;
+	struct fwd_lcore *lcore; /**< Lcore being scheduled. */
 };
 
 /**
@@ -795,6 +796,7 @@ void port_summary_header_display(void);
 void rx_queue_infos_display(portid_t port_idi, uint16_t queue_id);  void
tx_queue_infos_display(portid_t port_idi, uint16_t queue_id); void fwd_lcores_config_display(void);
+bool pkt_fwd_shared_rxq_check(void);
void pkt_fwd_config_display(struct fwd_config *cfg); void
rxtx_config_display(void); void fwd_config_setup(void); --
2.33.0

Xueming Li

author

2021-10-20 07:55 UTC #198 lore raw

PATCH v11 7/7 app/testpmd: add forwarding engine for shared Rx queue

In reply to Xueming Li

To support shared Rx queue, this patch introduces dedicate forwarding engine. The engine groups received packets by mbuf->port into sub-group, updates stream statistics and simply frees packets.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/meson.build                    |   1 +
 app/test-pmd/shared_rxq_fwd.c               | 148 ++++++++++++++++++++
 app/test-pmd/testpmd.c                      |   1 +
 app/test-pmd/testpmd.h                      |   1 +
 doc/guides/testpmd_app_ug/run_app.rst       |   1 +
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +-
 6 files changed, 156 insertions(+), 1 deletion(-)
 create mode 100644 app/test-pmd/shared_rxq_fwd.c

app/test-pmd/meson.build +1 -0

index 98f3289bdfa..07042e45b12 100644
@@ -21,6 +21,7 @@ sources = files(
         'noisy_vnf.c',
         'parameters.c',
         'rxonly.c',
+        'shared_rxq_fwd.c',
         'testpmd.c',
         'txonly.c',
         'util.c',

app/test-pmd/shared_rxq_fwd.c new file +148 -0

148 lines not on this page · raw message

app/test-pmd/testpmd.c +1 -0

index f3f81ef561f..11a85d92d9a 100644
@@ -188,6 +188,7 @@ struct fwd_engine * fwd_engines[] = {
 #ifdef RTE_LIBRTE_IEEE1588
 	&ieee1588_fwd_engine,
 #endif
+	&shared_rxq_engine,
 	NULL,
 };

app/test-pmd/testpmd.h +1 -0

index f121a2da90c..f1fd607e365 100644
@@ -299,6 +299,7 @@ extern struct fwd_engine five_tuple_swap_fwd_engine;
 #ifdef RTE_LIBRTE_IEEE1588
 extern struct fwd_engine ieee1588_fwd_engine;
 #endif
+extern struct fwd_engine shared_rxq_engine;
 
 extern struct fwd_engine * fwd_engines[]; /**< NULL terminated array. */
 extern cmdline_parse_inst_t cmd_set_raw;

doc/guides/testpmd_app_ug/run_app.rst +1 -0

index ff5908dcd50..e4b97844ced 100644
@@ -252,6 +252,7 @@ The command line options are:
        tm
        noisy
        5tswap
+       shared-rxq
 
 *   ``--rss-ip``

doc/guides/testpmd_app_ug/testpmd_funcs.rst +4 -1

index 8ead7a4a712..499874187f2 100644
@@ -314,7 +314,7 @@ set fwd
 Set the packet forwarding mode::
 
    testpmd> set fwd (io|mac|macswap|flowgen| \
-                     rxonly|txonly|csum|icmpecho|noisy|5tswap) (""|retry)
+                     rxonly|txonly|csum|icmpecho|noisy|5tswap|shared-rxq) (""|retry)
 
 ``retry`` can be specified for forwarding engines except ``rx_only``.

@@ -357,6 +357,9 @@ The available information categories are:
 
   L4 swaps the source port and destination port of transport layer (TCP and UDP).
 
+* ``shared-rxq``: Receive only for shared Rx queue.
+  Resolve packet source port from mbuf and update stream statistics accordingly.
+
 Example::
 
    testpmd> set fwd rxonly

Signature · 1 line

2.33.0

Thomas Monjalon

2021-10-20 19:20 UTC #199 lore raw

Re: [dpdk-dev] [PATCH v11 7/7] app/testpmd: add forwarding engine for shared Rx queue

20/10/2021 09:53, Xueming Li:

Quoted 3 lines

To support shared Rx queue, this patch introduces dedicate forwarding engine. The engine groups received packets by mbuf->port into sub-group, updates stream statistics and simply frees packets.

Given this engine is mentioned in previous commits, shouldn't it be placed earlier in the series?

Quoted 24 lines

app/test-pmd/shared_rxq_fwd.c new file +24 -0

+#include <stdarg.h>
+#include <string.h>
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <unistd.h>
+#include <inttypes.h>
+
+#include <sys/queue.h>
+#include <sys/stat.h>
+
+#include <rte_common.h>
+#include <rte_byteorder.h>
+#include <rte_log.h>
+#include <rte_debug.h>
+#include <rte_cycles.h>
+#include <rte_memory.h>
+#include <rte_memcpy.h>
+#include <rte_launch.h>
+#include <rte_eal.h>
+#include <rte_per_lcore.h>
+#include <rte_lcore.h>
+#include <rte_atomic.h>
+#include <rte_branch_prediction.h>

app/test-pmd/shared_rxq_fwd.c line 38

Quoted 10 lines

app/test-pmd/shared_rxq_fwd.c new file +10 -0

+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_pci.h>
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_string_fns.h>
+#include <rte_ip.h>
+#include <rte_udp.h>
+#include <rte_net.h>
+#include <rte_flow.h>

Please do not include useless files.

Li, Xiaoyun

2021-10-21 03:26 UTC #200 lore raw

Re: [dpdk-dev] [PATCH v11 7/7] app/testpmd: add forwarding engine for shared Rx queue

Quoted 49 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Thursday, October 21, 2021 03:20
To: Xueming Li <[email protected]>
Cc: [email protected]; Zhang, Yuying <[email protected]>; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>; Li, Xiaoyun <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v11 7/7] app/testpmd: add forwarding engine for shared Rx queue
20/10/2021 09:53, Xueming Li:
Quoted 3 lines
To support shared Rx queue, this patch introduces dedicate forwarding engine. The engine groups received packets by mbuf->port into sub-group, updates stream statistics and simply frees packets.
Given this engine is mentioned in previous commits, shouldn't it be placed earlier in the series?
Quoted 28 lines
app/test-pmd/shared_rxq_fwd.c new file +28 -0
+#include <stdarg.h>
+#include <string.h>
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <unistd.h>
+#include <inttypes.h>
+
+#include <sys/queue.h>
+#include <sys/stat.h>
+
+#include <rte_common.h>
+#include <rte_byteorder.h>
+#include <rte_log.h>
+#include <rte_debug.h>
+#include <rte_cycles.h>
+#include <rte_memory.h>
+#include <rte_memcpy.h>
+#include <rte_launch.h>
+#include <rte_eal.h>
+#include <rte_per_lcore.h>
+#include <rte_lcore.h>
+#include <rte_atomic.h>
+#include <rte_branch_prediction.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_pci.h>
+#include <rte_ether.h>

Quoted 8 lines

app/test-pmd/shared_rxq_fwd.c line 38
Quoted 6 lines
app/test-pmd/shared_rxq_fwd.c new file +6 -0
+#include <rte_ethdev.h>
+#include <rte_string_fns.h>
+#include <rte_ip.h>
+#include <rte_udp.h>
+#include <rte_net.h>
+#include <rte_flow.h>
Please do not include useless files.

Xueming(Steven) Li

author

2021-10-21 04:39 UTC #201 lore raw

Re: [dpdk-dev] [PATCH v11 7/7] app/testpmd: add forwarding engine for shared Rx queue

In reply to Thomas Monjalon

Quoted 7 lines · Thomas Monjalon

20/10/2021 09:53, Xueming Li:
Quoted 3 lines
To support shared Rx queue, this patch introduces dedicate forwarding engine. The engine groups received packets by mbuf->port into sub-group, updates stream statistics and simply frees packets.
Given this engine is mentioned in previous commits, shouldn't it be placed earlier in the series?

There will be compilation issue, I'll move the document change into this patch.

Quoted 39 lines

Quoted 25 lines

app/test-pmd/shared_rxq_fwd.c new file +24 -0

+#include <stdarg.h>
+#include <string.h>
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <unistd.h>
+#include <inttypes.h>
+
+#include <sys/queue.h>
+#include <sys/stat.h>
+
+#include <rte_common.h>
+#include <rte_byteorder.h>
+#include <rte_log.h>
+#include <rte_debug.h>
+#include <rte_cycles.h>
+#include <rte_memory.h>
+#include <rte_memcpy.h>
+#include <rte_launch.h>
+#include <rte_eal.h>
+#include <rte_per_lcore.h>
+#include <rte_lcore.h>
+#include <rte_atomic.h>
+#include <rte_branch_prediction.h>

app/test-pmd/shared_rxq_fwd.c line 38

Quoted 10 lines

app/test-pmd/shared_rxq_fwd.c new file +10 -0

+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+#include <rte_pci.h>
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_string_fns.h>
+#include <rte_ip.h>
+#include <rte_udp.h>
+#include <rte_net.h>
+#include <rte_flow.h>

Please do not include useless files.

Xueming Li

author

2021-10-21 05:09 UTC #202 lore raw

v12 cover ethdev: introduce shared Rx queue

In reply to Xueming Li

This patch introduces shared Rx queue. PF and representors in same Rx domain and switch domain could share Rx queue set by specifying non-zero share group value in Rx queue configuration.

All ports that share Rx queue actually shares hardware descriptor queue and feed all Rx queues with one descriptor supply, memory is saved.

Polling any queue using same shared Rx queue receives packets from all member ports. Source port is identified by mbuf->port.

Xueming Li (7):
ethdev: introduce shared Rx queue
ethdev: get device capability name as string app/testpmd: dump device capability and Rx domain info app/testpmd: new parameter to enable shared Rx queue app/testpmd: dump port info for shared Rx queue app/testpmd: force shared Rx queue polled on same core app/testpmd: add forwarding engine for shared Rx queue

 app/test-pmd/config.c                         | 141 +++++++++++++++++-
 app/test-pmd/meson.build                      |   1 +
 app/test-pmd/parameters.c                     |  13 ++
 app/test-pmd/shared_rxq_fwd.c                 | 113 ++++++++++++++
 app/test-pmd/testpmd.c                        |  26 +++-
 app/test-pmd/testpmd.h                        |   9 ++
 app/test-pmd/util.c                           |   3 +
 doc/guides/nics/features.rst                  |  13 ++
 doc/guides/nics/features/default.ini          |   1 +
 .../prog_guide/switch_representation.rst      |  11 ++
 doc/guides/rel_notes/release_21_11.rst        |   6 +
 doc/guides/testpmd_app_ug/run_app.rst         |   9 ++
 doc/guides/testpmd_app_ug/testpmd_funcs.rst   |   5 +-
 lib/ethdev/rte_ethdev.c                       |  33 ++++
 lib/ethdev/rte_ethdev.h                       |  38 +++++
 lib/ethdev/version.map                        |   1 +
 16 files changed, 417 insertions(+), 6 deletions(-)
 create mode 100644 app/test-pmd/shared_rxq_fwd.c

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-21 05:09 UTC #203 lore raw

PATCH v12 1/7 ethdev: introduce shared Rx queue

This patch introduces shared Rx queue. Ports in same Rx domain and switch domain could share Rx queue set by specifying non-zero sharing group in Rx queue configuration.

Shared Rx queue is identified by share_rxq field of Rx queue configuration. Port A RxQ X can share RxQ with Port B RxQ Y by using same shared Rx queue ID.

Shared Rx queue must be polled in same thread or core, polling a queue ID of any member port is essentially same.

Multiple share groups are supported. PMD should support mixed configuration by allowing multiple share groups and non-shared Rx queue on one port.

PMD advertise shared Rx queue capability via RTE_ETH_DEV_CAPA_RXQ_SHARE.

PMD is responsible for shared Rx queue consistency checks to avoid member port's configuration contradict each other.

Signed-off-by Xueming Li <[email protected]>

Reviewed-by Andrew Rybchenko <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

---
 doc/guides/nics/features.rst                  | 13 ++++++++++
 doc/guides/nics/features/default.ini          |  1 +
 .../prog_guide/switch_representation.rst      | 11 +++++++++
 doc/guides/rel_notes/release_21_11.rst        |  6 +++++
 lib/ethdev/rte_ethdev.c                       |  8 +++++++
 lib/ethdev/rte_ethdev.h                       | 24 +++++++++++++++++++
 6 files changed, 63 insertions(+)

doc/guides/nics/features.rst +13 -0

index 8dd421ca013..d35751d5b5a 100644
@@ -614,6 +614,19 @@ Supports inner packet L4 checksum.
   ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_OUTER_UDP_CKSUM``.
 
 
+.. _nic_features_shared_rx_queue:
+
+Shared Rx queue
+---------------
+
+Supports shared Rx queue for ports in same Rx domain of a switch domain.
+
+* **[uses]     rte_eth_dev_info**: ``dev_capa:RTE_ETH_DEV_CAPA_RXQ_SHARE``.
+* **[uses]     rte_eth_dev_info，rte_eth_switch_info**: ``rx_domain``, ``domain_id``.
+* **[uses]     rte_eth_rxconf**: ``share_group``, ``share_qid``.
+* **[provides] mbuf**: ``mbuf.port``.
+
+
 .. _nic_features_packet_type_parsing:
 
 Packet type parsing

doc/guides/nics/features/default.ini +1 -0

index 09914b1ad32..39d21fcd379 100644
@@ -19,6 +19,7 @@ Free Tx mbuf on demand =
 Queue start/stop     =
 Runtime Rx queue setup =
 Runtime Tx queue setup =
+Shared Rx queue      =
 Burst mode info      =
 Power mgmt address monitor =
 MTU update           =

doc/guides/prog_guide/switch_representation.rst +11 -0

index ff6aa91c806..4f2532a91ea 100644
@@ -123,6 +123,17 @@ thought as a software "patch panel" front-end for applications.
 .. [1] `Ethernet switch device driver model (switchdev)
        <https://www.kernel.org/doc/Documentation/networking/switchdev.txt>`_
 
+- For some PMDs, memory usage of representors is huge when number of
+  representor grows, mbufs are allocated for each descriptor of Rx queue.
+  Polling large number of ports brings more CPU load, cache miss and
+  latency. Shared Rx queue can be used to share Rx queue between PF and
+  representors among same Rx domain. ``RTE_ETH_DEV_CAPA_RXQ_SHARE`` in
+  device info is used to indicate the capability. Setting non-zero share
+  group in Rx queue configuration to enable share, share_qid is used to
+  identify the shared Rx queue in group. Polling any member port can
+  receive packets of all member ports in the group, port ID is saved in
+  ``mbuf.port``.
+
 Basic SR-IOV
 ------------

doc/guides/rel_notes/release_21_11.rst +6 -0

index 74776ca0691..f4fb68e7408 100644
@@ -75,6 +75,12 @@ New Features
     operations.
   * Added multi-process support.
 
+* **Added ethdev shared Rx queue support.**
+
+  * Added new device capability flag and Rx domain field to switch info.
+  * Added share group and share queue ID to Rx queue configuration.
+  * Added testpmd support and dedicate forwarding engine.
+
 * **Added support to get all MAC addresses of a device.**
 
   Added ``rte_eth_macaddrs_get`` to allow user to retrieve all Ethernet

lib/ethdev/rte_ethdev.c +8 -0

index 1f18aa916cc..31a9cba065b 100644
@@ -2175,6 +2175,14 @@ rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id,
 		return -EINVAL;
 	}
 
+	if (local_conf.share_group > 0 &&
+	    (dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE) == 0) {
+		RTE_ETHDEV_LOG(ERR,
+			"Ethdev port_id=%d rx_queue_id=%d, enabled share_group=%hu while device doesn't support Rx queue share\n",
+			port_id, rx_queue_id, local_conf.share_group);
+		return -EINVAL;
+	}
+
 	/*
 	 * If LRO is enabled, check that the maximum aggregated packet
 	 * size is supported by the configured device.

lib/ethdev/rte_ethdev.h +24 -0

index 014270d3167..40f88cc3d64 100644
@@ -1045,6 +1045,14 @@ struct rte_eth_rxconf {
 	uint8_t rx_drop_en; /**< Drop packets if no descriptors are available. */
 	uint8_t rx_deferred_start; /**< Do not start queue with rte_eth_dev_start(). */
 	uint16_t rx_nseg; /**< Number of descriptions in rx_seg array. */
+	/**
+	 * Share group index in Rx domain and switch domain.
+	 * Non-zero value to enable Rx queue share, zero value disable share.
+	 * PMD is responsible for Rx queue consistency checks to avoid member
+	 * port's configuration contradict to each other.
+	 */
+	uint16_t share_group;
+	uint16_t share_qid; /**< Shared Rx queue ID in group. */
 	/**
 	 * Per-queue Rx offloads to be set using DEV_RX_OFFLOAD_* flags.
 	 * Only offloads set on rx_queue_offload_capa or rx_offload_capa

@@ -1445,6 +1453,16 @@ struct rte_eth_conf {
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /** Device supports Tx queue setup after device started. */
 #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
+/**
+ * Device supports shared Rx queue among ports within Rx domain and
+ * switch domain. Mbufs are consumed by shared Rx queue instead of
+ * each queue. Multiple groups are supported by share_group of Rx
+ * queue configuration. Shared Rx queue is identified by PMD using
+ * share_qid of Rx queue configuration. Polling any port in the group
+ * receive packets of all member ports, source port identified by
+ * mbuf->port field.
+ */
+#define RTE_ETH_DEV_CAPA_RXQ_SHARE              RTE_BIT64(2)
 /**@}*/
 
 /*

@@ -1488,6 +1506,12 @@ struct rte_eth_switch_info {
 	 * but each driver should explicitly define the mapping of switch
 	 * port identifier to that physical interconnect/switch
 	 */
+	/**
+	 * Shared Rx queue sub-domain boundary. Only ports in same Rx domain
+	 * and switch domain can share Rx queue. Valid only if device advertised
+	 * RTE_ETH_DEV_CAPA_RXQ_SHARE capability.
+	 */
+	uint16_t rx_domain;
 };
 
 /**

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-21 05:09 UTC #204 lore raw

PATCH v12 2/7 ethdev: get device capability name as string

In reply to Xueming Li

This patch adds API to return name of device capability.

Signed-off-by Xueming Li <[email protected]>

Reviewed-by Andrew Rybchenko <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

Acked-by Thomas Monjalon <[email protected]>

---
 lib/ethdev/rte_ethdev.c | 25 +++++++++++++++++++++++++
 lib/ethdev/rte_ethdev.h | 14 ++++++++++++++
 lib/ethdev/version.map  |  1 +
 3 files changed, 40 insertions(+)

lib/ethdev/rte_ethdev.c +25 -0

index 31a9cba065b..bfe5b0adbef 100644
@@ -167,6 +167,15 @@ static const struct {
 
 #undef RTE_TX_OFFLOAD_BIT2STR
 
+static const struct {
+	uint64_t offload;
+	const char *name;
+} rte_eth_dev_capa_names[] = {
+	{RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP, "RUNTIME_RX_QUEUE_SETUP"},
+	{RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP, "RUNTIME_TX_QUEUE_SETUP"},
+	{RTE_ETH_DEV_CAPA_RXQ_SHARE, "RXQ_SHARE"},
+};
+
 /**
  * The user application callback description.
  *

@@ -1236,6 +1245,22 @@ rte_eth_dev_tx_offload_name(uint64_t offload)
 	return name;
 }
 
+const char *
+rte_eth_dev_capability_name(uint64_t capability)
+{
+	const char *name = "UNKNOWN";
+	unsigned int i;
+
+	for (i = 0; i < RTE_DIM(rte_eth_dev_capa_names); ++i) {
+		if (capability == rte_eth_dev_capa_names[i].offload) {
+			name = rte_eth_dev_capa_names[i].name;
+			break;
+		}
+	}
+
+	return name;
+}
+
 static inline int
 eth_dev_check_lro_pkt_size(uint16_t port_id, uint32_t config_size,
 		   uint32_t max_rx_pkt_len, uint32_t dev_info_size)

lib/ethdev/rte_ethdev.h +14 -0

index 40f88cc3d64..9baca39e97a 100644
@@ -2109,6 +2109,20 @@ const char *rte_eth_dev_rx_offload_name(uint64_t offload);
  */
 const char *rte_eth_dev_tx_offload_name(uint64_t offload);
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get RTE_ETH_DEV_CAPA_* flag name.
+ *
+ * @param capability
+ *   Capability flag.
+ * @return
+ *   Capability name or 'UNKNOWN' if the flag cannot be recognized.
+ */
+__rte_experimental
+const char *rte_eth_dev_capability_name(uint64_t capability);
+
 /**
  * Configure an Ethernet device.
  * This function must be invoked first before any other function in the

lib/ethdev/version.map +1 -0

index d552c955c94..e1abe997290 100644
@@ -249,6 +249,7 @@ EXPERIMENTAL {
 	rte_mtr_meter_policy_validate;
 
 	# added in 21.11
+	rte_eth_dev_capability_name;
 	rte_eth_dev_conf_get;
 	rte_eth_macaddrs_get;
 	rte_eth_rx_metadata_negotiate;

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-21 05:09 UTC #205 lore raw

PATCH v12 3/7 app/testpmd: dump device capability and Rx domain info

In reply to Xueming Li

Dump device capability and Rx domain ID if shared Rx queue is supported by device.

Signed-off-by Xueming Li <[email protected]>

Acked-by Andrew Rybchenko <[email protected]>

Acked-by Xiaoyun Li <[email protected]>

---
 app/test-pmd/config.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

app/test-pmd/config.c +29 -0

index 23aa334cda0..db36ca41b94 100644
@@ -644,6 +644,29 @@ device_infos_display(const char *identifier)
 	rte_devargs_reset(&da);
 }
 
+static void
+print_dev_capabilities(uint64_t capabilities)
+{
+	uint64_t single_capa;
+	int begin;
+	int end;
+	int bit;
+
+	if (capabilities == 0)
+		return;
+
+	begin = __builtin_ctzll(capabilities);
+	end = sizeof(capabilities) * CHAR_BIT - __builtin_clzll(capabilities);
+
+	single_capa = 1ULL << begin;
+	for (bit = begin; bit < end; bit++) {
+		if (capabilities & single_capa)
+			printf(" %s",
+			       rte_eth_dev_capability_name(single_capa));
+		single_capa <<= 1;
+	}
+}
+
 void
 port_infos_display(portid_t port_id)
 {

@@ -795,6 +818,9 @@ port_infos_display(portid_t port_id)
 	printf("Max segment number per MTU/TSO: %hu\n",
 		dev_info.tx_desc_lim.nb_mtu_seg_max);
 
+	printf("Device capabilities: 0x%"PRIx64"(", dev_info.dev_capa);
+	print_dev_capabilities(dev_info.dev_capa);
+	printf(" )\n");
 	/* Show switch info only if valid switch domain and port id is set */
 	if (dev_info.switch_info.domain_id !=
 		RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID) {

@@ -805,6 +831,9 @@ port_infos_display(portid_t port_id)
 			dev_info.switch_info.domain_id);
 		printf("Switch Port Id: %u\n",
 			dev_info.switch_info.port_id);
+		if ((dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE) != 0)
+			printf("Switch Rx domain: %u\n",
+			       dev_info.switch_info.rx_domain);
 	}
 }

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-21 05:09 UTC #206 lore raw

PATCH v12 4/7 app/testpmd: new parameter to enable shared Rx queue

In reply to Xueming Li

Adds "--rxq-share=X" parameter to enable shared RxQ.

Rx queue is shared if device supports, otherwise fallback to standard RxQ.

Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX, implies all ports join share group 1. Queue ID is mapped equally with shared Rx queue ID.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/config.c                 |  7 ++++++-
 app/test-pmd/parameters.c             | 13 +++++++++++++
 app/test-pmd/testpmd.c                | 20 +++++++++++++++++---
 app/test-pmd/testpmd.h                |  2 ++
 doc/guides/testpmd_app_ug/run_app.rst |  6 ++++++
 5 files changed, 44 insertions(+), 4 deletions(-)

app/test-pmd/config.c +6 -1

index db36ca41b94..e4bbf457916 100644
@@ -2890,7 +2890,12 @@ rxtx_config_display(void)
 			printf("      RX threshold registers: pthresh=%d hthresh=%d "
 				" wthresh=%d\n",
 				pthresh_tmp, hthresh_tmp, wthresh_tmp);
-			printf("      RX Offloads=0x%"PRIx64"\n", offloads_tmp);
+			printf("      RX Offloads=0x%"PRIx64, offloads_tmp);
+			if (rx_conf->share_group > 0)
+				printf(" share_group=%u share_qid=%u",
+				       rx_conf->share_group,
+				       rx_conf->share_qid);
+			printf("\n");
 		}
 
 		/* per tx queue config only for first queue to be less verbose */

app/test-pmd/parameters.c +13 -0

index 779a721fa05..afc75f6bd21 100644
@@ -171,6 +171,7 @@ usage(char* progname)
 	printf("  --tx-ip=src,dst: IP addresses in Tx-only mode\n");
 	printf("  --tx-udp=src[,dst]: UDP ports in Tx-only mode\n");
 	printf("  --eth-link-speed: force link speed.\n");
+	printf("  --rxq-share=X: number of ports per shared Rx queue groups, defaults to UINT32_MAX (1 group)\n");
 	printf("  --disable-link-check: disable check on link status when "
 	       "starting/stopping ports.\n");
 	printf("  --disable-device-start: do not automatically start port\n");

@@ -678,6 +679,7 @@ launch_args_parse(int argc, char** argv)
 		{ "rxpkts",			1, 0, 0 },
 		{ "txpkts",			1, 0, 0 },
 		{ "txonly-multi-flow",		0, 0, 0 },
+		{ "rxq-share",			2, 0, 0 },
 		{ "eth-link-speed",		1, 0, 0 },
 		{ "disable-link-check",		0, 0, 0 },
 		{ "disable-device-start",	0, 0, 0 },

@@ -1352,6 +1354,17 @@ launch_args_parse(int argc, char** argv)
 			}
 			if (!strcmp(lgopts[opt_idx].name, "txonly-multi-flow"))
 				txonly_multi_flow = 1;
+			if (!strcmp(lgopts[opt_idx].name, "rxq-share")) {
+				if (optarg == NULL) {
+					rxq_share = UINT32_MAX;
+				} else {
+					n = atoi(optarg);
+					if (n >= 0)
+						rxq_share = (uint32_t)n;
+					else
+						rte_exit(EXIT_FAILURE, "rxq-share must be >= 0\n");
+				}
+			}
 			if (!strcmp(lgopts[opt_idx].name, "no-flush-rx"))
 				no_flush_rx = 1;
 			if (!strcmp(lgopts[opt_idx].name, "eth-link-speed")) {

app/test-pmd/testpmd.c +17 -3

index af0e79fe6d5..80337bad382 100644
@@ -502,6 +502,11 @@ uint8_t record_core_cycles;
  */
 uint8_t record_burst_stats;
 
+/*
+ * Number of ports per shared Rx queue group, 0 disable.
+ */
+uint32_t rxq_share;
+
 unsigned int num_sockets = 0;
 unsigned int socket_ids[RTE_MAX_NUMA_NODES];

@@ -3629,14 +3634,23 @@ dev_event_callback(const char *device_name, enum rte_dev_event_type type,
 }
 
 static void
-rxtx_port_config(struct rte_port *port)
+rxtx_port_config(portid_t pid)
 {
 	uint16_t qid;
 	uint64_t offloads;
+	struct rte_port *port = &ports[pid];
 
 	for (qid = 0; qid < nb_rxq; qid++) {
 		offloads = port->rx_conf[qid].offloads;
 		port->rx_conf[qid] = port->dev_info.default_rxconf;
+
+		if (rxq_share > 0 &&
+		    (port->dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)) {
+			/* Non-zero share group to enable RxQ share. */
+			port->rx_conf[qid].share_group = pid / rxq_share + 1;
+			port->rx_conf[qid].share_qid = qid; /* Equal mapping. */
+		}
+
 		if (offloads != 0)
 			port->rx_conf[qid].offloads = offloads;

@@ -3765,7 +3779,7 @@ init_port_config(void)
 			}
 		}
 
-		rxtx_port_config(port);
+		rxtx_port_config(pid);
 
 		ret = eth_macaddr_get_print_err(pid, &port->eth_addr);
 		if (ret != 0)

@@ -3977,7 +3991,7 @@ init_port_dcb_config(portid_t pid,
 
 	memcpy(&rte_port->dev_conf, &port_conf, sizeof(struct rte_eth_conf));
 
-	rxtx_port_config(rte_port);
+	rxtx_port_config(pid);
 	/* VLAN filter */
 	rte_port->dev_conf.rxmode.offloads |= DEV_RX_OFFLOAD_VLAN_FILTER;
 	for (i = 0; i < RTE_DIM(vlan_tags); i++)

app/test-pmd/testpmd.h +2 -0

index e3995d24ab5..63f9913deb6 100644
@@ -524,6 +524,8 @@ extern enum tx_pkt_split tx_pkt_split;
 
 extern uint8_t txonly_multi_flow;
 
+extern uint32_t rxq_share;
+
 extern uint16_t nb_pkt_per_burst;
 extern uint16_t nb_pkt_flowgen_clones;
 extern int nb_flows_flowgen;

doc/guides/testpmd_app_ug/run_app.rst +6 -0

index 8ff7ab85369..faa3efb902c 100644
@@ -395,6 +395,12 @@ The command line options are:
 
     Generate multiple flows in txonly mode.
 
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX,
+    implies all ports join share group 1.
+
 *   ``--eth-link-speed``
 
     Set a forced link speed to the ethernet port::

Signature · 1 line

2.33.0

Thomas Monjalon

Acked-by on v12 4/7

2021-10-21 09:20 UTC #207 lore raw

Re: [dpdk-dev] [PATCH v12 4/7] app/testpmd: new parameter to enable shared Rx queue

21/10/2021 07:08, Xueming Li:

Quoted 10 lines

Adds "--rxq-share=X" parameter to enable shared RxQ.
Rx queue is shared if device supports, otherwise fallback to standard RxQ.
Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX, implies all ports join share group 1. Queue ID is mapped equally with shared Rx queue ID.
Signed-off-by Xueming Li <[email protected]>

Acked-by Thomas Monjalon <[email protected]>

Xueming Li

author

2021-10-21 05:09 UTC #208 lore raw

PATCH v12 5/7 app/testpmd: dump port info for shared Rx queue

In reply to Xueming Li

In case of shared Rx queue, polling any member port returns mbufs for all members. This patch dumps mbuf->port for each packet.

Signed-off-by Xueming Li <[email protected]>

Acked-by Xiaoyun Li <[email protected]>

---
 app/test-pmd/util.c | 3 +++
 1 file changed, 3 insertions(+)

app/test-pmd/util.c +3 -0

index 26dc0c86406..f712f687287 100644
@@ -101,6 +101,9 @@ dump_pkt_burst(uint16_t port_id, uint16_t queue, struct rte_mbuf *pkts[],
 		struct rte_port *port = &ports[port_id];
 
 		mb = pkts[i];
+		if (rxq_share > 0)
+			MKDUMPSTR(print_buf, buf_size, cur_len, "port %u, ",
+				  mb->port);
 		eth_hdr = rte_pktmbuf_read(mb, 0, sizeof(_eth_hdr), &_eth_hdr);
 		eth_type = RTE_BE_TO_CPU_16(eth_hdr->ether_type);
 		packet_type = mb->packet_type;

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-21 05:10 UTC #209 lore raw

PATCH v12 6/7 app/testpmd: force shared Rx queue polled on same core

In reply to Xueming Li

Shared Rx queue must be polled on same core. This patch checks and stops forwarding if shared RxQ being scheduled on multiple
cores.

It's suggested to use same number of Rx queues and polling cores.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/config.c  | 105 +++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.c |   5 +-
 app/test-pmd/testpmd.h |   2 +
 3 files changed, 111 insertions(+), 1 deletion(-)

app/test-pmd/config.c +105 -0

index e4bbf457916..cad78350dcc 100644
@@ -3067,6 +3067,111 @@ port_rss_hash_key_update(portid_t port_id, char rss_type[], uint8_t *hash_key,
 	}
 }
 
+/*
+ * Check whether a shared rxq scheduled on other lcores.
+ */
+static bool
+fwd_stream_on_other_lcores(uint16_t domain_id, lcoreid_t src_lc,
+			   portid_t src_port, queueid_t src_rxq,
+			   uint32_t share_group, queueid_t share_rxq)
+{
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/* Check remaining cores. */
+	for (lc_id = src_lc + 1; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)
+			    == 0 || rxq_conf->share_group == 0)
+				/* Not shared rxq. */
+				continue;
+			if (domain_id != port->dev_info.switch_info.domain_id)
+				continue;
+			if (rxq_conf->share_group != share_group)
+				continue;
+			if (rxq_conf->share_qid != share_rxq)
+				continue;
+			printf("Shared Rx queue group %u queue %hu can't be scheduled on different cores:\n",
+			       share_group, share_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       src_lc, src_port, src_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       lc_id, fs->rx_port, fs->rx_queue);
+			printf("Please use --nb-cores=%hu to limit number of forwarding cores\n",
+			       nb_rxq);
+			return true;
+		}
+	}
+	return false;
+}
+
+/*
+ * Check shared rxq configuration.
+ *
+ * Shared group must not being scheduled on different core.
+ */
+bool
+pkt_fwd_shared_rxq_check(void)
+{
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	uint16_t domain_id;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	if (rxq_share == 0)
+		return true;
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/*
+	 * Check streams on each core, make sure the same switch domain +
+	 * group + queue doesn't get scheduled on other cores.
+	 */
+	for (lc_id = 0; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			/* Update lcore info stream being scheduled. */
+			fs->lcore = fwd_lcores[lc_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)
+			    == 0 || rxq_conf->share_group == 0)
+				/* Not shared rxq. */
+				continue;
+			/* Check shared rxq not scheduled on remaining cores. */
+			domain_id = port->dev_info.switch_info.domain_id;
+			if (fwd_stream_on_other_lcores(domain_id, lc_id,
+						       fs->rx_port,
+						       fs->rx_queue,
+						       rxq_conf->share_group,
+						       rxq_conf->share_qid))
+				return false;
+		}
+	}
+	return true;
+}
+
 /*
  * Setup forwarding configuration for each logical core.
  */

app/test-pmd/testpmd.c +4 -1

index 80337bad382..d76d298a4b9 100644
@@ -2309,6 +2309,10 @@ start_packet_forwarding(int with_tx_first)
 
 	fwd_config_setup();
 
+	pkt_fwd_config_display(&cur_fwd_config);
+	if (!pkt_fwd_shared_rxq_check())
+		return;
+
 	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
 	if (port_fwd_begin != NULL) {
 		for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {

@@ -2338,7 +2342,6 @@ start_packet_forwarding(int with_tx_first)
 	if(!no_flush_rx)
 		flush_fwd_rx_queues();
 
-	pkt_fwd_config_display(&cur_fwd_config);
 	rxtx_config_display();
 
 	fwd_stats_reset();

app/test-pmd/testpmd.h +2 -0

index 63f9913deb6..9482dab3071 100644
@@ -147,6 +147,7 @@ struct fwd_stream {
 	uint64_t     core_cycles; /**< used for RX and TX processing */
 	struct pkt_burst_stats rx_burst_stats;
 	struct pkt_burst_stats tx_burst_stats;
+	struct fwd_lcore *lcore; /**< Lcore being scheduled. */
 };
 
 /**

@@ -842,6 +843,7 @@ void port_summary_header_display(void);
 void rx_queue_infos_display(portid_t port_idi, uint16_t queue_id);
 void tx_queue_infos_display(portid_t port_idi, uint16_t queue_id);
 void fwd_lcores_config_display(void);
+bool pkt_fwd_shared_rxq_check(void);
 void pkt_fwd_config_display(struct fwd_config *cfg);
 void rxtx_config_display(void);
 void fwd_config_setup(void);

Signature · 1 line

2.33.0

Li, Xiaoyun

Acked-by on v12 6/7

2021-10-21 06:35 UTC #210 lore raw

Re: [dpdk-dev] [PATCH v12 6/7] app/testpmd: force shared Rx queue polled on same core

Quoted 20 lines

-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Thursday, October 21, 2021 13:09
To: [email protected]; Zhang, Yuying <[email protected]>; Li, Xiaoyun <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>
Subject: [PATCH v12 6/7] app/testpmd: force shared Rx queue polled on same core
Shared Rx queue must be polled on same core. This patch checks and stops forwarding if shared RxQ being scheduled on multiple cores.
It's suggested to use same number of Rx queues and polling cores.

Quoted 8 lines

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/config.c  | 105
+++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.c |   5 +-
 app/test-pmd/testpmd.h |   2 +
 3 files changed, 111 insertions(+), 1 deletion(-)

Acked-by Xiaoyun Li <[email protected]>

Xueming Li

author

2021-10-21 05:10 UTC #211 lore raw

PATCH v12 7/7 app/testpmd: add forwarding engine for shared Rx queue

In reply to Xueming Li

To support shared Rx queue, this patch introduces dedicate forwarding engine. The engine groups received packets by mbuf->port into sub-group, updates stream statistics and simply frees packets.

Signed-off-by Xueming Li <[email protected]>

Acked-by Xiaoyun Li <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

---
 app/test-pmd/meson.build                    |   1 +
 app/test-pmd/shared_rxq_fwd.c               | 113 ++++++++++++++++++++
 app/test-pmd/testpmd.c                      |   1 +
 app/test-pmd/testpmd.h                      |   5 +
 doc/guides/testpmd_app_ug/run_app.rst       |   5 +-
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +-
 6 files changed, 128 insertions(+), 2 deletions(-)
 create mode 100644 app/test-pmd/shared_rxq_fwd.c

app/test-pmd/meson.build +1 -0

index 1ad54caef2c..b5a0f7b6209 100644
@@ -22,6 +22,7 @@ sources = files(
         'noisy_vnf.c',
         'parameters.c',
         'rxonly.c',
+        'shared_rxq_fwd.c',
         'testpmd.c',
         'txonly.c',
         'util.c',

app/test-pmd/shared_rxq_fwd.c new file +113 -0

113 lines not on this page · raw message

app/test-pmd/testpmd.c +1 -0

index d76d298a4b9..6d5bbc82404 100644
@@ -188,6 +188,7 @@ struct fwd_engine * fwd_engines[] = {
 #ifdef RTE_LIBRTE_IEEE1588
 	&ieee1588_fwd_engine,
 #endif
+	&shared_rxq_engine,
 	NULL,
 };

app/test-pmd/testpmd.h +5 -0

index 9482dab3071..ef7a6199313 100644
@@ -12,6 +12,10 @@
 #include <rte_gro.h>
 #include <rte_gso.h>
 #include <rte_os_shim.h>
+#include <rte_mbuf_dyn.h>
+#include <rte_flow.h>
+#include <rte_ethdev.h>
+
 #include <cmdline.h>
 #include <sys/queue.h>
 #ifdef RTE_HAS_JANSSON

@@ -339,6 +343,7 @@ extern struct fwd_engine five_tuple_swap_fwd_engine;
 #ifdef RTE_LIBRTE_IEEE1588
 extern struct fwd_engine ieee1588_fwd_engine;
 #endif
+extern struct fwd_engine shared_rxq_engine;
 
 extern struct fwd_engine * fwd_engines[]; /**< NULL terminated array. */
 extern cmdline_parse_inst_t cmd_set_raw;

doc/guides/testpmd_app_ug/run_app.rst +4 -1

index faa3efb902c..74412bb82ca 100644
@@ -258,6 +258,7 @@ The command line options are:
        tm
        noisy
        5tswap
+       shared-rxq
 
 *   ``--rss-ip``

@@ -399,7 +400,9 @@ The command line options are:
 
     Create queues in shared Rx queue mode if device supports.
     Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX,
-    implies all ports join share group 1.
+    implies all ports join share group 1. A new forwarding engine
+    "shared-rxq" should be used for shared Rx queues. This engine does
+    Rx only and update stream statistics accordingly.
 
 *   ``--eth-link-speed``

doc/guides/testpmd_app_ug/testpmd_funcs.rst +4 -1

index 6d127d9a7bc..78d23429c42 100644
@@ -314,7 +314,7 @@ set fwd
 Set the packet forwarding mode::
 
    testpmd> set fwd (io|mac|macswap|flowgen| \
-                     rxonly|txonly|csum|icmpecho|noisy|5tswap) (""|retry)
+                     rxonly|txonly|csum|icmpecho|noisy|5tswap|shared-rxq) (""|retry)
 
 ``retry`` can be specified for forwarding engines except ``rx_only``.

@@ -357,6 +357,9 @@ The available information categories are:
 
   L4 swaps the source port and destination port of transport layer (TCP and UDP).
 
+* ``shared-rxq``: Receive only for shared Rx queue.
+  Resolve packet source port from mbuf and update stream statistics accordingly.
+
 Example::
 
    testpmd> set fwd rxonly

Signature · 1 line

2.33.0

Li, Xiaoyun

2021-10-21 06:33 UTC #212 lore raw

Re: [dpdk-dev] [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx queue

Quoted 15 lines

-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Thursday, October 21, 2021 13:09
To: [email protected]; Zhang, Yuying <[email protected]>; Li, Xiaoyun <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>
Subject: [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx queue

Quoted 6 lines

To support shared Rx queue, this patch introduces dedicate forwarding engine. The engine groups received packets by mbuf->port into sub-group, updates stream statistics and simply frees packets.
Signed-off-by Xueming Li <[email protected]>
Acked-by Xiaoyun Li <[email protected]>

I didn't ack you on this patch. I remember I added "+1" to the comment about your includes issue. It will confuse reviewers not to review new versions.

Quoted 1 line

Acked-by Ajit Khaparde <[email protected]>

I didn't see he ack this patch as well.
Please remove these acks.

Quoted 27 lines

---
 app/test-pmd/meson.build                    |   1 +
 app/test-pmd/shared_rxq_fwd.c               | 113 ++++++++++++++++++++
 app/test-pmd/testpmd.c                      |   1 +
 app/test-pmd/testpmd.h                      |   5 +
 doc/guides/testpmd_app_ug/run_app.rst       |   5 +-
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +-
 6 files changed, 128 insertions(+), 2 deletions(-)  create mode 100644 app/test-

pmd/shared_rxq_fwd.c

app/test-pmd/meson.build index renamed from app/test-pmd/meson.build +0 -0

1ad54caef2c..b5a0f7b6209 100644

app/test-pmd/meson.build +1 -0

@@ -22,6 +22,7 @@ sources = files(
         'noisy_vnf.c',
         'parameters.c',
         'rxonly.c',
+        'shared_rxq_fwd.c',
         'testpmd.c',
         'txonly.c',
         'util.c',

app/test-pmd/shared_rxq_fwd.c new file +0 -0

new file mode 100644 index 00000000000..c4684893674

app/test-pmd/shared_rxq_fwd.c line 3

Quoted 3 lines

app/test-pmd/shared_rxq_fwd.c new file +3 -0

@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 NVIDIA Corporation & Affiliates  */
+

Please add "#include <rte_ethdev.h>" here.
Your shared_rxq_fwd.c only needs this include.

Quoted 15 lines

app/test-pmd/shared_rxq_fwd.c new file +15 -0

+#include "testpmd.h"
+
+/*
+ * Rx only sub-burst forwarding.
+ */
+static void
+forward_rx_only(uint16_t nb_rx, struct rte_mbuf **pkts_burst) {
+	rte_pktmbuf_free_bulk(pkts_burst, nb_rx); }
+
+/**
+ * Get packet source stream by source port and queue.
+ * All streams of same shared Rx queue locates on same core.
+ */
+static struct fwd_stream *
+forward_stream_get(struct fwd_stream *fs, uint16_t port) {

<snip>

app/test-pmd/testpmd.h line 18

Quoted 12 lines

app/test-pmd/testpmd.h index renamed from app/test-pmd/testpmd.h +0 -0
9482dab3071..ef7a6199313 100644
app/test-pmd/testpmd.h +4 -0
@@ -12,6 +12,10 @@
 #include <rte_gro.h>
 #include <rte_gso.h>
 #include <rte_os_shim.h>
+#include <rte_mbuf_dyn.h>
+#include <rte_flow.h>
+#include <rte_ethdev.h>
+

Please remove these includes and this blank line.
You only need to add the lib you need in your file like I said above.

Quoted 62 lines

app/test-pmd/testpmd.h +1 -0

 #include <cmdline.h>
 #include <sys/queue.h>
 #ifdef RTE_HAS_JANSSON

@@ -339,6 +343,7 @@ extern struct fwd_engine five_tuple_swap_fwd_engine;
 #ifdef RTE_LIBRTE_IEEE1588  extern struct fwd_engine ieee1588_fwd_engine;
 #endif
+extern struct fwd_engine shared_rxq_engine;
 
 extern struct fwd_engine * fwd_engines[]; /**< NULL terminated array. */

extern cmdline_parse_inst_t cmd_set_raw; diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst index faa3efb902c..74412bb82ca 100644

doc/guides/testpmd_app_ug/run_app.rst +4 -1

@@ -258,6 +258,7 @@ The command line options are:
        tm
        noisy
        5tswap
+       shared-rxq
 
 *   ``--rss-ip``

@@ -399,7 +400,9 @@ The command line options are:
 
     Create queues in shared Rx queue mode if device supports.
     Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX,
-    implies all ports join share group 1.
+    implies all ports join share group 1. A new forwarding engine
+    "shared-rxq" should be used for shared Rx queues. This engine does
+    Rx only and update stream statistics accordingly.
 
 *   ``--eth-link-speed``

doc/guides/testpmd_app_ug/testpmd_funcs.rst +4 -1

index 6d127d9a7bc..78d23429c42 100644
@@ -314,7 +314,7 @@ set fwd
 Set the packet forwarding mode::
 
    testpmd> set fwd (io|mac|macswap|flowgen| \
-                     rxonly|txonly|csum|icmpecho|noisy|5tswap) (""|retry)
+
+ rxonly|txonly|csum|icmpecho|noisy|5tswap|shared-rxq) (""|retry)
 
 ``retry`` can be specified for forwarding engines except ``rx_only``.

@@ -357,6 +357,9 @@ The available information categories are:
 
   L4 swaps the source port and destination port of transport layer (TCP and UDP).
 
+* ``shared-rxq``: Receive only for shared Rx queue.
+  Resolve packet source port from mbuf and update stream statistics

accordingly.
+
Example::

testpmd> set fwd rxonly
--
2.33.0

Xueming(Steven) Li

author

2021-10-21 07:59 UTC #213 lore raw

Re: [dpdk-dev] [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx queue

Quoted 25 lines · Xiaoyun

Quoted 25 lines
-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Thursday, October 21, 2021 13:09
To: [email protected]; Zhang, Yuying <[email protected]>; Li, Xiaoyun
<[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh
<[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior
Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>
Subject: [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx
queue
To support shared Rx queue, this patch introduces dedicate forwarding engine.
The engine groups received packets by mbuf->port into sub-group, updates
stream statistics and simply frees packets.

Quoted 6 lines

Quoted 2 lines
Signed-off-by Xueming Li <[email protected]>
Acked-by Xiaoyun Li <[email protected]>
I didn't ack you on this patch. I remember I added "+1" to the comment about your includes issue.
It will confuse reviewers not to review new versions.

Yes, they there by mistake.

Quoted 35 lines

Quoted 2 lines
Acked-by Ajit Khaparde <[email protected]>
I didn't see he ack this patch as well.
Please remove these acks.
Quoted 29 lines
---
app/test-pmd/meson.build | 1 +
app/test-pmd/shared_rxq_fwd.c | 113
++++++++++++++++++++
 app/test-pmd/testpmd.c                      |   1 +
 app/test-pmd/testpmd.h                      |   5 +
 doc/guides/testpmd_app_ug/run_app.rst       |   5 +-
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +-
 6 files changed, 128 insertions(+), 2 deletions(-)  create mode
100644 app/test-
pmd/shared_rxq_fwd.c
app/test-pmd/meson.build +0 -0
index
1ad54caef2c..b5a0f7b6209 100644
app/test-pmd/meson.build +1 -0
@@ -22,6 +22,7 @@ sources = files(
          'noisy_vnf.c',
          'parameters.c',
          'rxonly.c',
+        'shared_rxq_fwd.c',
          'testpmd.c',
          'txonly.c',
          'util.c',
app/test- renamed from app/test-pmd/shared_rxq_fwd.c +0 -0
pmd/shared_rxq_fwd.c
new file mode 100644 index 00000000000..c4684893674 --- /dev/null

Quoted 8 lines

app/test- line 3
Quoted 5 lines
+++ b/app/test-pmd/shared_rxq_fwd.c
app/test- renamed from app/test-pmd/shared_rxq_fwd.c +3 -0
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 NVIDIA Corporation & Affiliates  */
+
Please add "#include <rte_ethdev.h>" here.
Your shared_rxq_fwd.c only needs this include.

As explained below, testpmd relies on rte_ethdev.h.

Quoted 17 lines

Quoted 16 lines

app/test- renamed from app/test-pmd/shared_rxq_fwd.c +15 -0

+#include "testpmd.h"
+
+/*
+ * Rx only sub-burst forwarding.
+ */
+static void
+forward_rx_only(uint16_t nb_rx, struct rte_mbuf **pkts_burst) {
+	rte_pktmbuf_free_bulk(pkts_burst, nb_rx); }
+
+/**
+ * Get packet source stream by source port and queue.
+ * All streams of same shared Rx queue locates on same core.
+ */
+static struct fwd_stream *
+forward_stream_get(struct fwd_stream *fs, uint16_t port) {

<snip>

Quoted 15 lines

app/test-pmd/testpmd.h line 18
Quoted 12 lines
app/test-pmd/testpmd.h index renamed from app/test-pmd/testpmd.h +0 -0
9482dab3071..ef7a6199313 100644
app/test-pmd/testpmd.h +4 -0
@@ -12,6 +12,10 @@
  #include <rte_gro.h>
  #include <rte_gso.h>
  #include <rte_os_shim.h>
+#include <rte_mbuf_dyn.h>
+#include <rte_flow.h>
+#include <rte_ethdev.h>
+
Please remove these includes and this blank line.
You only need to add the lib you need in your file like I said above.

From test, testpmd.h used these headers, otherwise compile error if not included by fwd engine.

Quoted 73 lines

Quoted 72 lines
app/test-pmd/testpmd.h +1 -0
  #include <cmdline.h>
  #include <sys/queue.h>
  #ifdef RTE_HAS_JANSSON
@@ -339,6 +343,7 @@ extern struct fwd_engine
 five_tuple_swap_fwd_engine;
 #ifdef RTE_LIBRTE_IEEE1588  extern struct fwd_engine
 ieee1588_fwd_engine;
 #endif
+extern struct fwd_engine shared_rxq_engine;
 
extern struct fwd_engine * fwd_engines[]; /**< NULL terminated array. */
extern cmdline_parse_inst_t cmd_set_raw; diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst
index faa3efb902c..74412bb82ca 100644
doc/guides/testpmd_app_ug/run_app.rst +0 -0
@@ -258,6 +258,7 @@ The command line options are:
        tm
        noisy
        5tswap
+ shared-rxq
* ``--rss-ip``
doc/guides/testpmd_app_ug/run_app.rst +0 -0
@@ -399,7 +400,9 @@ The command line options are:
 
Create queues in shared Rx queue mode if device supports. Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX,
doc/guides/testpmd_app_ug/run_app.rst +2 -1
-    implies all ports join share group 1.
+    implies all ports join share group 1. A new forwarding engine
+    "shared-rxq" should be used for shared Rx queues. This engine
does
+ Rx only and update stream statistics accordingly.
* ``--eth-link-speed``
doc/guides/testpmd_app_ug/testpmd_funcs.rst +0 -0
index 6d127d9a7bc..78d23429c42 100644
@@ -314,7 +314,7 @@ set fwd
Set the packet forwarding mode::
    testpmd> set fwd (io|mac|macswap|flowgen| \
-                     rxonly|txonly|csum|icmpecho|noisy|5tswap)
(""|retry)
+
+ rxonly|txonly|csum|icmpecho|noisy|5tswap|shared-rxq) (""|retry)
``retry`` can be specified for forwarding engines except ``rx_only``.
doc/guides/testpmd_app_ug/testpmd_funcs.rst +0 -0
@@ -357,6 +357,9 @@ The available information categories are:
 
L4 swaps the source port and destination port of transport layer (TCP and UDP).
doc/guides/testpmd_app_ug/testpmd_funcs.rst +2 -0
+* ``shared-rxq``: Receive only for shared Rx queue.
+  Resolve packet source port from mbuf and update stream
statistics
accordingly.
+
Example::
testpmd> set fwd rxonly
--
2.33.0

Li, Xiaoyun

2021-10-21 08:02 UTC #214 lore raw

Re: [dpdk-dev] [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx queue

Quoted 124 lines

-----Original Message-----
From: Xueming(Steven) Li <[email protected]>
Sent: Thursday, October 21, 2021 15:59
To: Li, Xiaoyun <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; [email protected]; NBU-Contact-Thomas Monjalon <[email protected]>; Slava Ovsiienko <[email protected]>; [email protected]; Yigit, Ferruh <[email protected]>; [email protected]; Lior Margalit <[email protected]>
Subject: Re: [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx queue
Quoted 20 lines · Xiaoyun
Quoted 20 lines
-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Thursday, October 21, 2021 13:09
To: [email protected]; Zhang, Yuying <[email protected]>; Li, Xiaoyun <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>
Subject: [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx queue
To support shared Rx queue, this patch introduces dedicate forwarding engine.
The engine groups received packets by mbuf->port into sub-group, updates stream statistics and simply frees packets.
Quoted 6 lines
Quoted 2 lines
Signed-off-by Xueming Li <[email protected]>
Acked-by Xiaoyun Li <[email protected]>
I didn't ack you on this patch. I remember I added "+1" to the comment about your includes issue.
It will confuse reviewers not to review new versions.
Yes, they there by mistake.
Quoted 35 lines
Quoted 2 lines
Acked-by Ajit Khaparde <[email protected]>
I didn't see he ack this patch as well.
Please remove these acks.
Quoted 29 lines
---
app/test-pmd/meson.build | 1 +
app/test-pmd/shared_rxq_fwd.c | 113
++++++++++++++++++++
 app/test-pmd/testpmd.c                      |   1 +
 app/test-pmd/testpmd.h                      |   5 +
 doc/guides/testpmd_app_ug/run_app.rst       |   5 +-
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +-
 6 files changed, 128 insertions(+), 2 deletions(-)  create mode
100644 app/test-
pmd/shared_rxq_fwd.c
app/test-pmd/meson.build +0 -0
index
1ad54caef2c..b5a0f7b6209 100644
app/test-pmd/meson.build +1 -0
@@ -22,6 +22,7 @@ sources = files(
          'noisy_vnf.c',
          'parameters.c',
          'rxonly.c',
+        'shared_rxq_fwd.c',
          'testpmd.c',
          'txonly.c',
          'util.c',
app/test- renamed from app/test-pmd/shared_rxq_fwd.c +0 -0
pmd/shared_rxq_fwd.c new file mode 100644 index
00000000000..c4684893674
--- /dev/null
Quoted 8 lines
app/test- line 3
Quoted 5 lines
+++ b/app/test-pmd/shared_rxq_fwd.c
app/test- renamed from app/test-pmd/shared_rxq_fwd.c +3 -0
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 NVIDIA Corporation & Affiliates  */
+
Please add "#include <rte_ethdev.h>" here.
Your shared_rxq_fwd.c only needs this include.
As explained below, testpmd relies on rte_ethdev.h.
Quoted 17 lines
Quoted 16 lines
app/test- renamed from app/test-pmd/shared_rxq_fwd.c +15 -0
+#include "testpmd.h"
+
+/*
+ * Rx only sub-burst forwarding.
+ */
+static void
+forward_rx_only(uint16_t nb_rx, struct rte_mbuf **pkts_burst) {
+	rte_pktmbuf_free_bulk(pkts_burst, nb_rx); }
+
+/**
+ * Get packet source stream by source port and queue.
+ * All streams of same shared Rx queue locates on same core.
+ */
+static struct fwd_stream *
+forward_stream_get(struct fwd_stream *fs, uint16_t port) {
<snip>
Quoted 15 lines
app/test-pmd/testpmd.h line 18
Quoted 12 lines
app/test-pmd/testpmd.h index renamed from app/test-pmd/testpmd.h +0 -0
9482dab3071..ef7a6199313 100644
app/test-pmd/testpmd.h +4 -0
@@ -12,6 +12,10 @@
  #include <rte_gro.h>
  #include <rte_gso.h>
  #include <rte_os_shim.h>
+#include <rte_mbuf_dyn.h>
+#include <rte_flow.h>
+#include <rte_ethdev.h>
+
Please remove these includes and this blank line.
You only need to add the lib you need in your file like I said above.

Quoted 2 lines

From test, testpmd.h used these headers, otherwise compile error if not included by fwd engine.

Have you tried my way? Include "#include <rte_ethdev.h>" in shared_rxq_fwd.c. Please try this and see if there's any compiling issues.

Quoted 71 lines

Quoted 71 lines
Quoted 70 lines
app/test-pmd/testpmd.h +0 -0
  #include <cmdline.h>
  #include <sys/queue.h>
  #ifdef RTE_HAS_JANSSON
@@ -339,6 +343,7 @@ extern struct fwd_engine
five_tuple_swap_fwd_engine; #ifdef RTE_LIBRTE_IEEE1588 extern struct fwd_engine ieee1588_fwd_engine; #endif +extern struct fwd_engine shared_rxq_engine;
extern struct fwd_engine * fwd_engines[]; /**< NULL terminated array. */ extern cmdline_parse_inst_t cmd_set_raw; diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst
index faa3efb902c..74412bb82ca 100644
doc/guides/testpmd_app_ug/run_app.rst +0 -0
@@ -258,6 +258,7 @@ The command line options are:
        tm
        noisy
        5tswap
+ shared-rxq
* ``--rss-ip``
doc/guides/testpmd_app_ug/run_app.rst +0 -0
@@ -399,7 +400,9 @@ The command line options are:
 
Create queues in shared Rx queue mode if device supports. Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX,
doc/guides/testpmd_app_ug/run_app.rst +2 -1
-    implies all ports join share group 1.
+    implies all ports join share group 1. A new forwarding engine
+    "shared-rxq" should be used for shared Rx queues. This engine
does
+ Rx only and update stream statistics accordingly.
* ``--eth-link-speed``
doc/guides/testpmd_app_ug/testpmd_funcs.rst +0 -0
index 6d127d9a7bc..78d23429c42 100644
@@ -314,7 +314,7 @@ set fwd
Set the packet forwarding mode::
    testpmd> set fwd (io|mac|macswap|flowgen| \
-                     rxonly|txonly|csum|icmpecho|noisy|5tswap)
(""|retry)
+
+ rxonly|txonly|csum|icmpecho|noisy|5tswap|shared-rxq) (""|retry)
``retry`` can be specified for forwarding engines except ``rx_only``.
doc/guides/testpmd_app_ug/testpmd_funcs.rst +0 -0
@@ -357,6 +357,9 @@ The available information categories are:
 
L4 swaps the source port and destination port of transport layer (TCP and UDP).
doc/guides/testpmd_app_ug/testpmd_funcs.rst +2 -0
+* ``shared-rxq``: Receive only for shared Rx queue.
+  Resolve packet source port from mbuf and update stream
statistics
accordingly.
+
Example::
testpmd> set fwd rxonly
--
2.33.0

Xueming(Steven) Li

author

2021-10-21 08:22 UTC #215 lore raw

Re: [dpdk-dev] [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx queue

Quoted 123 lines · Xiaoyun

Quoted 123 lines
-----Original Message-----
From: Xueming(Steven) Li <[email protected]>
Sent: Thursday, October 21, 2021 15:59
To: Li, Xiaoyun <[email protected]>; Zhang, Yuying <[email protected]>; [email protected]
Cc: Ananyev, Konstantin <[email protected]>; [email protected]; NBU-Contact-Thomas Monjalon <[email protected]>; Slava Ovsiienko <[email protected]>; [email protected]; Yigit, Ferruh <[email protected]>; [email protected]; Lior Margalit <[email protected]>
Subject: Re: [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx queue
Quoted 20 lines · Xiaoyun
Quoted 20 lines
-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Thursday, October 21, 2021 13:09
To: [email protected]; Zhang, Yuying <[email protected]>; Li, Xiaoyun <[email protected]>
Cc: [email protected]; Jerin Jacob <[email protected]>; Yigit, Ferruh <[email protected]>; Andrew Rybchenko <[email protected]>; Viacheslav Ovsiienko <[email protected]>; Thomas Monjalon <[email protected]>; Lior Margalit <[email protected]>; Ananyev, Konstantin <[email protected]>; Ajit Khaparde <[email protected]>
Subject: [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx queue
To support shared Rx queue, this patch introduces dedicate forwarding engine.
The engine groups received packets by mbuf->port into sub-group, updates stream statistics and simply frees packets.
Quoted 6 lines
Quoted 2 lines
Signed-off-by Xueming Li <[email protected]>
Acked-by Xiaoyun Li <[email protected]>
I didn't ack you on this patch. I remember I added "+1" to the comment about your includes issue.
It will confuse reviewers not to review new versions.
Yes, they there by mistake.
Quoted 35 lines
Quoted 2 lines
Acked-by Ajit Khaparde <[email protected]>
I didn't see he ack this patch as well.
Please remove these acks.
Quoted 29 lines
---
app/test-pmd/meson.build | 1 +
app/test-pmd/shared_rxq_fwd.c | 113
++++++++++++++++++++
 app/test-pmd/testpmd.c                      |   1 +
 app/test-pmd/testpmd.h                      |   5 +
 doc/guides/testpmd_app_ug/run_app.rst       |   5 +-
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +-
 6 files changed, 128 insertions(+), 2 deletions(-)  create mode
100644 app/test-
pmd/shared_rxq_fwd.c
app/test-pmd/meson.build +0 -0
index
1ad54caef2c..b5a0f7b6209 100644
app/test-pmd/meson.build +1 -0
@@ -22,6 +22,7 @@ sources = files(
          'noisy_vnf.c',
          'parameters.c',
          'rxonly.c',
+        'shared_rxq_fwd.c',
          'testpmd.c',
          'txonly.c',
          'util.c',
app/test- renamed from app/test-pmd/shared_rxq_fwd.c +0 -0
pmd/shared_rxq_fwd.c new file mode 100644 index
00000000000..c4684893674
--- /dev/null
Quoted 8 lines
app/test- line 3
Quoted 5 lines
+++ b/app/test-pmd/shared_rxq_fwd.c
app/test- renamed from app/test-pmd/shared_rxq_fwd.c +3 -0
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2021 NVIDIA Corporation & Affiliates  */
+
Please add "#include <rte_ethdev.h>" here.
Your shared_rxq_fwd.c only needs this include.
As explained below, testpmd relies on rte_ethdev.h.
Quoted 32 lines
Quoted 16 lines
app/test- renamed from app/test-pmd/shared_rxq_fwd.c +15 -0
+#include "testpmd.h"
+
+/*
+ * Rx only sub-burst forwarding.
+ */
+static void
+forward_rx_only(uint16_t nb_rx, struct rte_mbuf **pkts_burst) {
+	rte_pktmbuf_free_bulk(pkts_burst, nb_rx); }
+
+/**
+ * Get packet source stream by source port and queue.
+ * All streams of same shared Rx queue locates on same core.
+ */
+static struct fwd_stream *
+forward_stream_get(struct fwd_stream *fs, uint16_t port) {
<snip>
app/test-pmd/testpmd.h line 18
Quoted 12 lines
app/test-pmd/testpmd.h index renamed from app/test-pmd/testpmd.h +0 -0
9482dab3071..ef7a6199313 100644
app/test-pmd/testpmd.h +4 -0
@@ -12,6 +12,10 @@
  #include <rte_gro.h>
  #include <rte_gso.h>
  #include <rte_os_shim.h>
+#include <rte_mbuf_dyn.h>
+#include <rte_flow.h>
+#include <rte_ethdev.h>
+
Please remove these includes and this blank line.
You only need to add the lib you need in your file like I said above.

Quoted 5 lines

Quoted 2 lines
From test, testpmd.h used these headers, otherwise compile error if not included by fwd engine.
Have you tried my way? Include "#include <rte_ethdev.h>" in shared_rxq_fwd.c. Please try this and see if there's any compiling issues.

It works, seems rte_ethdev.h has everything needed, thanks!

Quoted 73 lines

Quoted 72 lines
Quoted 72 lines
Quoted 71 lines
app/test-pmd/testpmd.h +0 -0
  #include <cmdline.h>
  #include <sys/queue.h>
  #ifdef RTE_HAS_JANSSON
@@ -339,6 +343,7 @@ extern struct fwd_engine
five_tuple_swap_fwd_engine; #ifdef RTE_LIBRTE_IEEE1588 extern struct fwd_engine ieee1588_fwd_engine; #endif +extern struct fwd_engine shared_rxq_engine;
extern struct fwd_engine * fwd_engines[]; /**< NULL terminated array. */ extern cmdline_parse_inst_t cmd_set_raw; diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst
index faa3efb902c..74412bb82ca 100644
doc/guides/testpmd_app_ug/run_app.rst +0 -0
@@ -258,6 +258,7 @@ The command line options are:
        tm
        noisy
        5tswap
+ shared-rxq
* ``--rss-ip``
doc/guides/testpmd_app_ug/run_app.rst +0 -0
@@ -399,7 +400,9 @@ The command line options are:
 
Create queues in shared Rx queue mode if device supports. Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX,
doc/guides/testpmd_app_ug/run_app.rst +2 -1
-    implies all ports join share group 1.
+    implies all ports join share group 1. A new forwarding engine
+    "shared-rxq" should be used for shared Rx queues. This engine
does
+ Rx only and update stream statistics accordingly.
* ``--eth-link-speed``
doc/guides/testpmd_app_ug/testpmd_funcs.rst +0 -0
index 6d127d9a7bc..78d23429c42 100644
@@ -314,7 +314,7 @@ set fwd
Set the packet forwarding mode::
    testpmd> set fwd (io|mac|macswap|flowgen| \
-                     rxonly|txonly|csum|icmpecho|noisy|5tswap)
(""|retry)
+
+ rxonly|txonly|csum|icmpecho|noisy|5tswap|shared-rxq) (""|retry)
``retry`` can be specified for forwarding engines except ``rx_only``.
doc/guides/testpmd_app_ug/testpmd_funcs.rst +0 -0
@@ -357,6 +357,9 @@ The available information categories are:
 
L4 swaps the source port and destination port of transport layer (TCP and UDP).
doc/guides/testpmd_app_ug/testpmd_funcs.rst +2 -0
+* ``shared-rxq``: Receive only for shared Rx queue.
+  Resolve packet source port from mbuf and update stream
statistics
accordingly.
+
Example::
testpmd> set fwd rxonly
--
2.33.0

Thomas Monjalon

2021-10-21 09:28 UTC #216 lore raw

Re: [dpdk-dev] [PATCH v12 7/7] app/testpmd: add forwarding engine for shared Rx queue

In reply to Xueming Li

21/10/2021 07:08, Xueming Li:

Quoted 1 line

+ implies all ports join share group 1. A new forwarding engine

Don't say "new" as it will not be new forever.
You can say a "specific" or "specialized".

Quoted 2 lines

doc/guides/testpmd_app_ug/run_app.rst +2 -0

+    "shared-rxq" should be used for shared Rx queues. This engine does
+    Rx only and update stream statistics accordingly.

Xueming Li

author

2021-10-21 10:42 UTC #217 lore raw

v13 cover ethdev: introduce shared Rx queue

In reply to Xueming Li

This patch introduces shared Rx queue. PF and representors in same Rx domain and switch domain could share Rx queue set by specifying non-zero share group value in Rx queue configuration.

All ports that share Rx queue actually shares hardware descriptor queue and feed all Rx queues with one descriptor supply, memory is saved.

Polling any queue using same shared Rx queue receives packets from all member ports. Source port is identified by mbuf->port.

 app/test-pmd/config.c                         | 141 +++++++++++++++++-
 app/test-pmd/meson.build                      |   1 +
 app/test-pmd/parameters.c                     |  13 ++
 app/test-pmd/shared_rxq_fwd.c                 | 115 ++++++++++++++
 app/test-pmd/testpmd.c                        |  26 +++-
 app/test-pmd/testpmd.h                        |   5 +
 app/test-pmd/util.c                           |   3 +
 doc/guides/nics/features.rst                  |  13 ++
 doc/guides/nics/features/default.ini          |   1 +
 .../prog_guide/switch_representation.rst      |  11 ++
 doc/guides/rel_notes/release_21_11.rst        |   6 +
 doc/guides/testpmd_app_ug/run_app.rst         |   9 ++
 doc/guides/testpmd_app_ug/testpmd_funcs.rst   |   5 +-
 lib/ethdev/rte_ethdev.c                       |  33 ++++
 lib/ethdev/rte_ethdev.h                       |  38 +++++
 lib/ethdev/version.map                        |   1 +
 16 files changed, 415 insertions(+), 6 deletions(-)
 create mode 100644 app/test-pmd/shared_rxq_fwd.c

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-21 10:42 UTC #218 lore raw

PATCH v13 1/7 ethdev: introduce shared Rx queue

This patch introduces shared Rx queue. Ports in same Rx domain and switch domain could share Rx queue set by specifying non-zero sharing group in Rx queue configuration.

Shared Rx queue is identified by share_rxq field of Rx queue configuration. Port A RxQ X can share RxQ with Port B RxQ Y by using same shared Rx queue ID.

Shared Rx queue must be polled in same thread or core, polling a queue ID of any member port is essentially same.

Multiple share groups are supported. PMD should support mixed configuration by allowing multiple share groups and non-shared Rx queue on one port.

PMD advertise shared Rx queue capability via RTE_ETH_DEV_CAPA_RXQ_SHARE.

PMD is responsible for shared Rx queue consistency checks to avoid member port's configuration contradict each other.

Signed-off-by Xueming Li <[email protected]>

Reviewed-by Andrew Rybchenko <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

---
 doc/guides/nics/features.rst                  | 13 ++++++++++
 doc/guides/nics/features/default.ini          |  1 +
 .../prog_guide/switch_representation.rst      | 11 +++++++++
 doc/guides/rel_notes/release_21_11.rst        |  6 +++++
 lib/ethdev/rte_ethdev.c                       |  8 +++++++
 lib/ethdev/rte_ethdev.h                       | 24 +++++++++++++++++++
 6 files changed, 63 insertions(+)

doc/guides/nics/features.rst +13 -0

index 8dd421ca013..d35751d5b5a 100644
@@ -614,6 +614,19 @@ Supports inner packet L4 checksum.
   ``tx_offload_capa,tx_queue_offload_capa:DEV_TX_OFFLOAD_OUTER_UDP_CKSUM``.
 
 
+.. _nic_features_shared_rx_queue:
+
+Shared Rx queue
+---------------
+
+Supports shared Rx queue for ports in same Rx domain of a switch domain.
+
+* **[uses]     rte_eth_dev_info**: ``dev_capa:RTE_ETH_DEV_CAPA_RXQ_SHARE``.
+* **[uses]     rte_eth_dev_info，rte_eth_switch_info**: ``rx_domain``, ``domain_id``.
+* **[uses]     rte_eth_rxconf**: ``share_group``, ``share_qid``.
+* **[provides] mbuf**: ``mbuf.port``.
+
+
 .. _nic_features_packet_type_parsing:
 
 Packet type parsing

doc/guides/nics/features/default.ini +1 -0

index 09914b1ad32..39d21fcd379 100644
@@ -19,6 +19,7 @@ Free Tx mbuf on demand =
 Queue start/stop     =
 Runtime Rx queue setup =
 Runtime Tx queue setup =
+Shared Rx queue      =
 Burst mode info      =
 Power mgmt address monitor =
 MTU update           =

doc/guides/prog_guide/switch_representation.rst +11 -0

index ff6aa91c806..4f2532a91ea 100644
@@ -123,6 +123,17 @@ thought as a software "patch panel" front-end for applications.
 .. [1] `Ethernet switch device driver model (switchdev)
        <https://www.kernel.org/doc/Documentation/networking/switchdev.txt>`_
 
+- For some PMDs, memory usage of representors is huge when number of
+  representor grows, mbufs are allocated for each descriptor of Rx queue.
+  Polling large number of ports brings more CPU load, cache miss and
+  latency. Shared Rx queue can be used to share Rx queue between PF and
+  representors among same Rx domain. ``RTE_ETH_DEV_CAPA_RXQ_SHARE`` in
+  device info is used to indicate the capability. Setting non-zero share
+  group in Rx queue configuration to enable share, share_qid is used to
+  identify the shared Rx queue in group. Polling any member port can
+  receive packets of all member ports in the group, port ID is saved in
+  ``mbuf.port``.
+
 Basic SR-IOV
 ------------

doc/guides/rel_notes/release_21_11.rst +6 -0

index 74776ca0691..f4fb68e7408 100644
@@ -75,6 +75,12 @@ New Features
     operations.
   * Added multi-process support.
 
+* **Added ethdev shared Rx queue support.**
+
+  * Added new device capability flag and Rx domain field to switch info.
+  * Added share group and share queue ID to Rx queue configuration.
+  * Added testpmd support and dedicate forwarding engine.
+
 * **Added support to get all MAC addresses of a device.**
 
   Added ``rte_eth_macaddrs_get`` to allow user to retrieve all Ethernet

lib/ethdev/rte_ethdev.c +8 -0

index 1f18aa916cc..31a9cba065b 100644
@@ -2175,6 +2175,14 @@ rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id,
 		return -EINVAL;
 	}
 
+	if (local_conf.share_group > 0 &&
+	    (dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE) == 0) {
+		RTE_ETHDEV_LOG(ERR,
+			"Ethdev port_id=%d rx_queue_id=%d, enabled share_group=%hu while device doesn't support Rx queue share\n",
+			port_id, rx_queue_id, local_conf.share_group);
+		return -EINVAL;
+	}
+
 	/*
 	 * If LRO is enabled, check that the maximum aggregated packet
 	 * size is supported by the configured device.

lib/ethdev/rte_ethdev.h +24 -0

index 014270d3167..40f88cc3d64 100644
@@ -1045,6 +1045,14 @@ struct rte_eth_rxconf {
 	uint8_t rx_drop_en; /**< Drop packets if no descriptors are available. */
 	uint8_t rx_deferred_start; /**< Do not start queue with rte_eth_dev_start(). */
 	uint16_t rx_nseg; /**< Number of descriptions in rx_seg array. */
+	/**
+	 * Share group index in Rx domain and switch domain.
+	 * Non-zero value to enable Rx queue share, zero value disable share.
+	 * PMD is responsible for Rx queue consistency checks to avoid member
+	 * port's configuration contradict to each other.
+	 */
+	uint16_t share_group;
+	uint16_t share_qid; /**< Shared Rx queue ID in group. */
 	/**
 	 * Per-queue Rx offloads to be set using DEV_RX_OFFLOAD_* flags.
 	 * Only offloads set on rx_queue_offload_capa or rx_offload_capa

@@ -1445,6 +1453,16 @@ struct rte_eth_conf {
 #define RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP 0x00000001
 /** Device supports Tx queue setup after device started. */
 #define RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP 0x00000002
+/**
+ * Device supports shared Rx queue among ports within Rx domain and
+ * switch domain. Mbufs are consumed by shared Rx queue instead of
+ * each queue. Multiple groups are supported by share_group of Rx
+ * queue configuration. Shared Rx queue is identified by PMD using
+ * share_qid of Rx queue configuration. Polling any port in the group
+ * receive packets of all member ports, source port identified by
+ * mbuf->port field.
+ */
+#define RTE_ETH_DEV_CAPA_RXQ_SHARE              RTE_BIT64(2)
 /**@}*/
 
 /*

@@ -1488,6 +1506,12 @@ struct rte_eth_switch_info {
 	 * but each driver should explicitly define the mapping of switch
 	 * port identifier to that physical interconnect/switch
 	 */
+	/**
+	 * Shared Rx queue sub-domain boundary. Only ports in same Rx domain
+	 * and switch domain can share Rx queue. Valid only if device advertised
+	 * RTE_ETH_DEV_CAPA_RXQ_SHARE capability.
+	 */
+	uint16_t rx_domain;
 };
 
 /**

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-21 10:42 UTC #219 lore raw

PATCH v13 2/7 ethdev: get device capability name as string

In reply to Xueming Li

This patch adds API to return name of device capability.

Signed-off-by Xueming Li <[email protected]>

Reviewed-by Andrew Rybchenko <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

Acked-by Thomas Monjalon <[email protected]>

---
 lib/ethdev/rte_ethdev.c | 25 +++++++++++++++++++++++++
 lib/ethdev/rte_ethdev.h | 14 ++++++++++++++
 lib/ethdev/version.map  |  1 +
 3 files changed, 40 insertions(+)

lib/ethdev/rte_ethdev.c +25 -0

index 31a9cba065b..bfe5b0adbef 100644
@@ -167,6 +167,15 @@ static const struct {
 
 #undef RTE_TX_OFFLOAD_BIT2STR
 
+static const struct {
+	uint64_t offload;
+	const char *name;
+} rte_eth_dev_capa_names[] = {
+	{RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP, "RUNTIME_RX_QUEUE_SETUP"},
+	{RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP, "RUNTIME_TX_QUEUE_SETUP"},
+	{RTE_ETH_DEV_CAPA_RXQ_SHARE, "RXQ_SHARE"},
+};
+
 /**
  * The user application callback description.
  *

@@ -1236,6 +1245,22 @@ rte_eth_dev_tx_offload_name(uint64_t offload)
 	return name;
 }
 
+const char *
+rte_eth_dev_capability_name(uint64_t capability)
+{
+	const char *name = "UNKNOWN";
+	unsigned int i;
+
+	for (i = 0; i < RTE_DIM(rte_eth_dev_capa_names); ++i) {
+		if (capability == rte_eth_dev_capa_names[i].offload) {
+			name = rte_eth_dev_capa_names[i].name;
+			break;
+		}
+	}
+
+	return name;
+}
+
 static inline int
 eth_dev_check_lro_pkt_size(uint16_t port_id, uint32_t config_size,
 		   uint32_t max_rx_pkt_len, uint32_t dev_info_size)

lib/ethdev/rte_ethdev.h +14 -0

index 40f88cc3d64..9baca39e97a 100644
@@ -2109,6 +2109,20 @@ const char *rte_eth_dev_rx_offload_name(uint64_t offload);
  */
 const char *rte_eth_dev_tx_offload_name(uint64_t offload);
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get RTE_ETH_DEV_CAPA_* flag name.
+ *
+ * @param capability
+ *   Capability flag.
+ * @return
+ *   Capability name or 'UNKNOWN' if the flag cannot be recognized.
+ */
+__rte_experimental
+const char *rte_eth_dev_capability_name(uint64_t capability);
+
 /**
  * Configure an Ethernet device.
  * This function must be invoked first before any other function in the

lib/ethdev/version.map +1 -0

index d552c955c94..e1abe997290 100644
@@ -249,6 +249,7 @@ EXPERIMENTAL {
 	rte_mtr_meter_policy_validate;
 
 	# added in 21.11
+	rte_eth_dev_capability_name;
 	rte_eth_dev_conf_get;
 	rte_eth_macaddrs_get;
 	rte_eth_rx_metadata_negotiate;

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-21 10:42 UTC #220 lore raw

PATCH v13 3/7 app/testpmd: dump device capability and Rx domain info

In reply to Xueming Li

Dump device capability and Rx domain ID if shared Rx queue is supported by device.

Signed-off-by Xueming Li <[email protected]>

Acked-by Andrew Rybchenko <[email protected]>

Acked-by Xiaoyun Li <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

---
 app/test-pmd/config.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

app/test-pmd/config.c +29 -0

index 23aa334cda0..db36ca41b94 100644
@@ -644,6 +644,29 @@ device_infos_display(const char *identifier)
 	rte_devargs_reset(&da);
 }
 
+static void
+print_dev_capabilities(uint64_t capabilities)
+{
+	uint64_t single_capa;
+	int begin;
+	int end;
+	int bit;
+
+	if (capabilities == 0)
+		return;
+
+	begin = __builtin_ctzll(capabilities);
+	end = sizeof(capabilities) * CHAR_BIT - __builtin_clzll(capabilities);
+
+	single_capa = 1ULL << begin;
+	for (bit = begin; bit < end; bit++) {
+		if (capabilities & single_capa)
+			printf(" %s",
+			       rte_eth_dev_capability_name(single_capa));
+		single_capa <<= 1;
+	}
+}
+
 void
 port_infos_display(portid_t port_id)
 {

@@ -795,6 +818,9 @@ port_infos_display(portid_t port_id)
 	printf("Max segment number per MTU/TSO: %hu\n",
 		dev_info.tx_desc_lim.nb_mtu_seg_max);
 
+	printf("Device capabilities: 0x%"PRIx64"(", dev_info.dev_capa);
+	print_dev_capabilities(dev_info.dev_capa);
+	printf(" )\n");
 	/* Show switch info only if valid switch domain and port id is set */
 	if (dev_info.switch_info.domain_id !=
 		RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID) {

@@ -805,6 +831,9 @@ port_infos_display(portid_t port_id)
 			dev_info.switch_info.domain_id);
 		printf("Switch Port Id: %u\n",
 			dev_info.switch_info.port_id);
+		if ((dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE) != 0)
+			printf("Switch Rx domain: %u\n",
+			       dev_info.switch_info.rx_domain);
 	}
 }

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-21 10:42 UTC #221 lore raw

PATCH v13 4/7 app/testpmd: new parameter to enable shared Rx queue

In reply to Xueming Li

Adds "--rxq-share=X" parameter to enable shared RxQ.

Rx queue is shared if device supports, otherwise fallback to standard RxQ.

Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX, implies all ports join share group 1. Queue ID is mapped equally with shared Rx queue ID.

Signed-off-by Xueming Li <[email protected]>

Acked-by Thomas Monjalon <[email protected]>

---
 app/test-pmd/config.c                 |  7 ++++++-
 app/test-pmd/parameters.c             | 13 +++++++++++++
 app/test-pmd/testpmd.c                | 20 +++++++++++++++++---
 app/test-pmd/testpmd.h                |  2 ++
 doc/guides/testpmd_app_ug/run_app.rst |  6 ++++++
 5 files changed, 44 insertions(+), 4 deletions(-)

app/test-pmd/config.c +6 -1

index db36ca41b94..e4bbf457916 100644
@@ -2890,7 +2890,12 @@ rxtx_config_display(void)
 			printf("      RX threshold registers: pthresh=%d hthresh=%d "
 				" wthresh=%d\n",
 				pthresh_tmp, hthresh_tmp, wthresh_tmp);
-			printf("      RX Offloads=0x%"PRIx64"\n", offloads_tmp);
+			printf("      RX Offloads=0x%"PRIx64, offloads_tmp);
+			if (rx_conf->share_group > 0)
+				printf(" share_group=%u share_qid=%u",
+				       rx_conf->share_group,
+				       rx_conf->share_qid);
+			printf("\n");
 		}
 
 		/* per tx queue config only for first queue to be less verbose */

app/test-pmd/parameters.c +13 -0

index 779a721fa05..afc75f6bd21 100644
@@ -171,6 +171,7 @@ usage(char* progname)
 	printf("  --tx-ip=src,dst: IP addresses in Tx-only mode\n");
 	printf("  --tx-udp=src[,dst]: UDP ports in Tx-only mode\n");
 	printf("  --eth-link-speed: force link speed.\n");
+	printf("  --rxq-share=X: number of ports per shared Rx queue groups, defaults to UINT32_MAX (1 group)\n");
 	printf("  --disable-link-check: disable check on link status when "
 	       "starting/stopping ports.\n");
 	printf("  --disable-device-start: do not automatically start port\n");

@@ -678,6 +679,7 @@ launch_args_parse(int argc, char** argv)
 		{ "rxpkts",			1, 0, 0 },
 		{ "txpkts",			1, 0, 0 },
 		{ "txonly-multi-flow",		0, 0, 0 },
+		{ "rxq-share",			2, 0, 0 },
 		{ "eth-link-speed",		1, 0, 0 },
 		{ "disable-link-check",		0, 0, 0 },
 		{ "disable-device-start",	0, 0, 0 },

@@ -1352,6 +1354,17 @@ launch_args_parse(int argc, char** argv)
 			}
 			if (!strcmp(lgopts[opt_idx].name, "txonly-multi-flow"))
 				txonly_multi_flow = 1;
+			if (!strcmp(lgopts[opt_idx].name, "rxq-share")) {
+				if (optarg == NULL) {
+					rxq_share = UINT32_MAX;
+				} else {
+					n = atoi(optarg);
+					if (n >= 0)
+						rxq_share = (uint32_t)n;
+					else
+						rte_exit(EXIT_FAILURE, "rxq-share must be >= 0\n");
+				}
+			}
 			if (!strcmp(lgopts[opt_idx].name, "no-flush-rx"))
 				no_flush_rx = 1;
 			if (!strcmp(lgopts[opt_idx].name, "eth-link-speed")) {

app/test-pmd/testpmd.c +17 -3

index af0e79fe6d5..80337bad382 100644
@@ -502,6 +502,11 @@ uint8_t record_core_cycles;
  */
 uint8_t record_burst_stats;
 
+/*
+ * Number of ports per shared Rx queue group, 0 disable.
+ */
+uint32_t rxq_share;
+
 unsigned int num_sockets = 0;
 unsigned int socket_ids[RTE_MAX_NUMA_NODES];

@@ -3629,14 +3634,23 @@ dev_event_callback(const char *device_name, enum rte_dev_event_type type,
 }
 
 static void
-rxtx_port_config(struct rte_port *port)
+rxtx_port_config(portid_t pid)
 {
 	uint16_t qid;
 	uint64_t offloads;
+	struct rte_port *port = &ports[pid];
 
 	for (qid = 0; qid < nb_rxq; qid++) {
 		offloads = port->rx_conf[qid].offloads;
 		port->rx_conf[qid] = port->dev_info.default_rxconf;
+
+		if (rxq_share > 0 &&
+		    (port->dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)) {
+			/* Non-zero share group to enable RxQ share. */
+			port->rx_conf[qid].share_group = pid / rxq_share + 1;
+			port->rx_conf[qid].share_qid = qid; /* Equal mapping. */
+		}
+
 		if (offloads != 0)
 			port->rx_conf[qid].offloads = offloads;

@@ -3765,7 +3779,7 @@ init_port_config(void)
 			}
 		}
 
-		rxtx_port_config(port);
+		rxtx_port_config(pid);
 
 		ret = eth_macaddr_get_print_err(pid, &port->eth_addr);
 		if (ret != 0)

@@ -3977,7 +3991,7 @@ init_port_dcb_config(portid_t pid,
 
 	memcpy(&rte_port->dev_conf, &port_conf, sizeof(struct rte_eth_conf));
 
-	rxtx_port_config(rte_port);
+	rxtx_port_config(pid);
 	/* VLAN filter */
 	rte_port->dev_conf.rxmode.offloads |= DEV_RX_OFFLOAD_VLAN_FILTER;
 	for (i = 0; i < RTE_DIM(vlan_tags); i++)

app/test-pmd/testpmd.h +2 -0

index e3995d24ab5..63f9913deb6 100644
@@ -524,6 +524,8 @@ extern enum tx_pkt_split tx_pkt_split;
 
 extern uint8_t txonly_multi_flow;
 
+extern uint32_t rxq_share;
+
 extern uint16_t nb_pkt_per_burst;
 extern uint16_t nb_pkt_flowgen_clones;
 extern int nb_flows_flowgen;

doc/guides/testpmd_app_ug/run_app.rst +6 -0

index 8ff7ab85369..faa3efb902c 100644
@@ -395,6 +395,12 @@ The command line options are:
 
     Generate multiple flows in txonly mode.
 
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX,
+    implies all ports join share group 1.
+
 *   ``--eth-link-speed``
 
     Set a forced link speed to the ethernet port::

Signature · 1 line

2.33.0

Ajit Khaparde

Acked-by on v13 4/7

2021-10-21 19:46 UTC #222 lore raw

Re: [dpdk-dev] [PATCH v13 4/7] app/testpmd: new parameter to enable shared Rx queue

Quoted 6 lines · Xueming Li

Adds "--rxq-share=X" parameter to enable shared RxQ.
Rx queue is shared if device supports, otherwise fallback to standard RxQ.

Quoted 6 lines

Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX, implies all ports join share group 1. Queue ID is mapped equally with shared Rx queue ID.
Signed-off-by Xueming Li <[email protected]>
Acked-by Thomas Monjalon <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

Quoted 156 lines

---
 app/test-pmd/config.c                 |  7 ++++++-
 app/test-pmd/parameters.c             | 13 +++++++++++++
 app/test-pmd/testpmd.c                | 20 +++++++++++++++++---
 app/test-pmd/testpmd.h                |  2 ++
 doc/guides/testpmd_app_ug/run_app.rst |  6 ++++++
 5 files changed, 44 insertions(+), 4 deletions(-)

app/test-pmd/config.c +6 -1

index db36ca41b94..e4bbf457916 100644
@@ -2890,7 +2890,12 @@ rxtx_config_display(void)
                        printf("      RX threshold registers: pthresh=%d hthresh=%d "
                                " wthresh=%d\n",
                                pthresh_tmp, hthresh_tmp, wthresh_tmp);
-                       printf("      RX Offloads=0x%"PRIx64"\n", offloads_tmp);
+                       printf("      RX Offloads=0x%"PRIx64, offloads_tmp);
+                       if (rx_conf->share_group > 0)
+                               printf(" share_group=%u share_qid=%u",
+                                      rx_conf->share_group,
+                                      rx_conf->share_qid);
+                       printf("\n");
                }
 
                /* per tx queue config only for first queue to be less verbose */

app/test-pmd/parameters.c +13 -0

index 779a721fa05..afc75f6bd21 100644
@@ -171,6 +171,7 @@ usage(char* progname)
        printf("  --tx-ip=src,dst: IP addresses in Tx-only mode\n");
        printf("  --tx-udp=src[,dst]: UDP ports in Tx-only mode\n");
        printf("  --eth-link-speed: force link speed.\n");
+       printf("  --rxq-share=X: number of ports per shared Rx queue groups, defaults to UINT32_MAX (1 group)\n");
        printf("  --disable-link-check: disable check on link status when "
               "starting/stopping ports.\n");
        printf("  --disable-device-start: do not automatically start port\n");

@@ -678,6 +679,7 @@ launch_args_parse(int argc, char** argv)
                { "rxpkts",                     1, 0, 0 },
                { "txpkts",                     1, 0, 0 },
                { "txonly-multi-flow",          0, 0, 0 },
+               { "rxq-share",                  2, 0, 0 },
                { "eth-link-speed",             1, 0, 0 },
                { "disable-link-check",         0, 0, 0 },
                { "disable-device-start",       0, 0, 0 },

@@ -1352,6 +1354,17 @@ launch_args_parse(int argc, char** argv)
                        }
                        if (!strcmp(lgopts[opt_idx].name, "txonly-multi-flow"))
                                txonly_multi_flow = 1;
+                       if (!strcmp(lgopts[opt_idx].name, "rxq-share")) {
+                               if (optarg == NULL) {
+                                       rxq_share = UINT32_MAX;
+                               } else {
+                                       n = atoi(optarg);
+                                       if (n >= 0)
+                                               rxq_share = (uint32_t)n;
+                                       else
+                                               rte_exit(EXIT_FAILURE, "rxq-share must be >= 0\n");
+                               }
+                       }
                        if (!strcmp(lgopts[opt_idx].name, "no-flush-rx"))
                                no_flush_rx = 1;
                        if (!strcmp(lgopts[opt_idx].name, "eth-link-speed")) {

app/test-pmd/testpmd.c +17 -3

index af0e79fe6d5..80337bad382 100644
@@ -502,6 +502,11 @@ uint8_t record_core_cycles;
  */
 uint8_t record_burst_stats;
 
+/*
+ * Number of ports per shared Rx queue group, 0 disable.
+ */
+uint32_t rxq_share;
+
 unsigned int num_sockets = 0;
 unsigned int socket_ids[RTE_MAX_NUMA_NODES];

@@ -3629,14 +3634,23 @@ dev_event_callback(const char *device_name, enum rte_dev_event_type type,
 }
 
 static void
-rxtx_port_config(struct rte_port *port)
+rxtx_port_config(portid_t pid)
 {
        uint16_t qid;
        uint64_t offloads;
+       struct rte_port *port = &ports[pid];
 
        for (qid = 0; qid < nb_rxq; qid++) {
                offloads = port->rx_conf[qid].offloads;
                port->rx_conf[qid] = port->dev_info.default_rxconf;
+
+               if (rxq_share > 0 &&
+                   (port->dev_info.dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)) {
+                       /* Non-zero share group to enable RxQ share. */
+                       port->rx_conf[qid].share_group = pid / rxq_share + 1;
+                       port->rx_conf[qid].share_qid = qid; /* Equal mapping. */
+               }
+
                if (offloads != 0)
                        port->rx_conf[qid].offloads = offloads;

@@ -3765,7 +3779,7 @@ init_port_config(void)
                        }
                }
 
-               rxtx_port_config(port);
+               rxtx_port_config(pid);
 
                ret = eth_macaddr_get_print_err(pid, &port->eth_addr);
                if (ret != 0)

@@ -3977,7 +3991,7 @@ init_port_dcb_config(portid_t pid,
 
        memcpy(&rte_port->dev_conf, &port_conf, sizeof(struct rte_eth_conf));
 
-       rxtx_port_config(rte_port);
+       rxtx_port_config(pid);
        /* VLAN filter */
        rte_port->dev_conf.rxmode.offloads |= DEV_RX_OFFLOAD_VLAN_FILTER;
        for (i = 0; i < RTE_DIM(vlan_tags); i++)

app/test-pmd/testpmd.h +2 -0

index e3995d24ab5..63f9913deb6 100644
@@ -524,6 +524,8 @@ extern enum tx_pkt_split tx_pkt_split;
 
 extern uint8_t txonly_multi_flow;
 
+extern uint32_t rxq_share;
+
 extern uint16_t nb_pkt_per_burst;
 extern uint16_t nb_pkt_flowgen_clones;
 extern int nb_flows_flowgen;

doc/guides/testpmd_app_ug/run_app.rst +6 -1

index 8ff7ab85369..faa3efb902c 100644
@@ -395,6 +395,12 @@ The command line options are:
 
     Generate multiple flows in txonly mode.
 
+*   ``--rxq-share=[X]``
+
+    Create queues in shared Rx queue mode if device supports.
+    Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX,
+    implies all ports join share group 1.
+
 *   ``--eth-link-speed``
 
     Set a forced link speed to the ethernet port::
--

2.33.0

Xueming Li

author

2021-10-21 10:42 UTC #223 lore raw

PATCH v13 5/7 app/testpmd: dump port info for shared Rx queue

In reply to Xueming Li

In case of shared Rx queue, source port mbuf from polling result isn't the Rx port of forwarding stream. To provide original port ID, this patch dumps mbuf->port for each packet in verbose mode if shared Rx queue enabled.

Signed-off-by Xueming Li <[email protected]>

Acked-by Xiaoyun Li <[email protected]>

---
 app/test-pmd/util.c | 3 +++
 1 file changed, 3 insertions(+)

app/test-pmd/util.c +3 -0

index 26dc0c86406..f712f687287 100644
@@ -101,6 +101,9 @@ dump_pkt_burst(uint16_t port_id, uint16_t queue, struct rte_mbuf *pkts[],
 		struct rte_port *port = &ports[port_id];
 
 		mb = pkts[i];
+		if (rxq_share > 0)
+			MKDUMPSTR(print_buf, buf_size, cur_len, "port %u, ",
+				  mb->port);
 		eth_hdr = rte_pktmbuf_read(mb, 0, sizeof(_eth_hdr), &_eth_hdr);
 		eth_type = RTE_BE_TO_CPU_16(eth_hdr->ether_type);
 		packet_type = mb->packet_type;

Signature · 1 line

2.33.0

Ajit Khaparde

Acked-by on v13 5/7

2021-10-21 19:48 UTC #224 lore raw

Re: [dpdk-dev] [PATCH v13 5/7] app/testpmd: dump port info for shared Rx queue

Quoted 8 lines · Xueming Li

In case of shared Rx queue, source port mbuf from polling result isn't the Rx port of forwarding stream. To provide original port ID, this patch dumps mbuf->port for each packet in verbose mode if shared Rx queue enabled.
Signed-off-by Xueming Li <[email protected]>
Acked-by Xiaoyun Li <[email protected]>

Acked-by Ajit Khaparde <[email protected]>

Quoted 21 lines

---
 app/test-pmd/util.c | 3 +++
 1 file changed, 3 insertions(+)

app/test-pmd/util.c +3 -1

index 26dc0c86406..f712f687287 100644
@@ -101,6 +101,9 @@ dump_pkt_burst(uint16_t port_id, uint16_t queue, struct rte_mbuf *pkts[],
                struct rte_port *port = &ports[port_id];
 
                mb = pkts[i];
+               if (rxq_share > 0)
+                       MKDUMPSTR(print_buf, buf_size, cur_len, "port %u, ",
+                                 mb->port);
                eth_hdr = rte_pktmbuf_read(mb, 0, sizeof(_eth_hdr), &_eth_hdr);
                eth_type = RTE_BE_TO_CPU_16(eth_hdr->ether_type);
                packet_type = mb->packet_type;
--

2.33.0

Xueming Li

author

2021-10-21 10:42 UTC #225 lore raw

PATCH v13 6/7 app/testpmd: force shared Rx queue polled on same core

In reply to Xueming Li

Shared Rx queue must be polled on same core. This patch checks and stops forwarding if shared RxQ being scheduled on multiple
cores.

It's suggested to use same number of Rx queues and polling cores.

Signed-off-by Xueming Li <[email protected]>

Acked-by Xiaoyun Li <[email protected]>

---
 app/test-pmd/config.c  | 105 +++++++++++++++++++++++++++++++++++++++++
 app/test-pmd/testpmd.c |   5 +-
 app/test-pmd/testpmd.h |   2 +
 3 files changed, 111 insertions(+), 1 deletion(-)

app/test-pmd/config.c +105 -0

index e4bbf457916..cad78350dcc 100644
@@ -3067,6 +3067,111 @@ port_rss_hash_key_update(portid_t port_id, char rss_type[], uint8_t *hash_key,
 	}
 }
 
+/*
+ * Check whether a shared rxq scheduled on other lcores.
+ */
+static bool
+fwd_stream_on_other_lcores(uint16_t domain_id, lcoreid_t src_lc,
+			   portid_t src_port, queueid_t src_rxq,
+			   uint32_t share_group, queueid_t share_rxq)
+{
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/* Check remaining cores. */
+	for (lc_id = src_lc + 1; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)
+			    == 0 || rxq_conf->share_group == 0)
+				/* Not shared rxq. */
+				continue;
+			if (domain_id != port->dev_info.switch_info.domain_id)
+				continue;
+			if (rxq_conf->share_group != share_group)
+				continue;
+			if (rxq_conf->share_qid != share_rxq)
+				continue;
+			printf("Shared Rx queue group %u queue %hu can't be scheduled on different cores:\n",
+			       share_group, share_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       src_lc, src_port, src_rxq);
+			printf("  lcore %hhu Port %hu queue %hu\n",
+			       lc_id, fs->rx_port, fs->rx_queue);
+			printf("Please use --nb-cores=%hu to limit number of forwarding cores\n",
+			       nb_rxq);
+			return true;
+		}
+	}
+	return false;
+}
+
+/*
+ * Check shared rxq configuration.
+ *
+ * Shared group must not being scheduled on different core.
+ */
+bool
+pkt_fwd_shared_rxq_check(void)
+{
+	streamid_t sm_id;
+	streamid_t nb_fs_per_lcore;
+	lcoreid_t  nb_fc;
+	lcoreid_t  lc_id;
+	struct fwd_stream *fs;
+	uint16_t domain_id;
+	struct rte_port *port;
+	struct rte_eth_dev_info *dev_info;
+	struct rte_eth_rxconf *rxq_conf;
+
+	if (rxq_share == 0)
+		return true;
+	nb_fc = cur_fwd_config.nb_fwd_lcores;
+	/*
+	 * Check streams on each core, make sure the same switch domain +
+	 * group + queue doesn't get scheduled on other cores.
+	 */
+	for (lc_id = 0; lc_id < nb_fc; lc_id++) {
+		sm_id = fwd_lcores[lc_id]->stream_idx;
+		nb_fs_per_lcore = fwd_lcores[lc_id]->stream_nb;
+		for (; sm_id < fwd_lcores[lc_id]->stream_idx + nb_fs_per_lcore;
+		     sm_id++) {
+			fs = fwd_streams[sm_id];
+			/* Update lcore info stream being scheduled. */
+			fs->lcore = fwd_lcores[lc_id];
+			port = &ports[fs->rx_port];
+			dev_info = &port->dev_info;
+			rxq_conf = &port->rx_conf[fs->rx_queue];
+			if ((dev_info->dev_capa & RTE_ETH_DEV_CAPA_RXQ_SHARE)
+			    == 0 || rxq_conf->share_group == 0)
+				/* Not shared rxq. */
+				continue;
+			/* Check shared rxq not scheduled on remaining cores. */
+			domain_id = port->dev_info.switch_info.domain_id;
+			if (fwd_stream_on_other_lcores(domain_id, lc_id,
+						       fs->rx_port,
+						       fs->rx_queue,
+						       rxq_conf->share_group,
+						       rxq_conf->share_qid))
+				return false;
+		}
+	}
+	return true;
+}
+
 /*
  * Setup forwarding configuration for each logical core.
  */

app/test-pmd/testpmd.c +4 -1

index 80337bad382..d76d298a4b9 100644
@@ -2309,6 +2309,10 @@ start_packet_forwarding(int with_tx_first)
 
 	fwd_config_setup();
 
+	pkt_fwd_config_display(&cur_fwd_config);
+	if (!pkt_fwd_shared_rxq_check())
+		return;
+
 	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
 	if (port_fwd_begin != NULL) {
 		for (i = 0; i < cur_fwd_config.nb_fwd_ports; i++) {

@@ -2338,7 +2342,6 @@ start_packet_forwarding(int with_tx_first)
 	if(!no_flush_rx)
 		flush_fwd_rx_queues();
 
-	pkt_fwd_config_display(&cur_fwd_config);
 	rxtx_config_display();
 
 	fwd_stats_reset();

app/test-pmd/testpmd.h +2 -0

index 63f9913deb6..9482dab3071 100644
@@ -147,6 +147,7 @@ struct fwd_stream {
 	uint64_t     core_cycles; /**< used for RX and TX processing */
 	struct pkt_burst_stats rx_burst_stats;
 	struct pkt_burst_stats tx_burst_stats;
+	struct fwd_lcore *lcore; /**< Lcore being scheduled. */
 };
 
 /**

@@ -842,6 +843,7 @@ void port_summary_header_display(void);
 void rx_queue_infos_display(portid_t port_idi, uint16_t queue_id);
 void tx_queue_infos_display(portid_t port_idi, uint16_t queue_id);
 void fwd_lcores_config_display(void);
+bool pkt_fwd_shared_rxq_check(void);
 void pkt_fwd_config_display(struct fwd_config *cfg);
 void rxtx_config_display(void);
 void fwd_config_setup(void);

Signature · 1 line

2.33.0

Xueming Li

author

2021-10-21 10:43 UTC #226 lore raw

PATCH v13 7/7 app/testpmd: add forwarding engine for shared Rx queue

In reply to Xueming Li

To support shared Rx queue, this patch introduces dedicate forwarding engine. The engine groups received packets by mbuf->port into sub-group, updates stream statistics and simply frees packets.

Signed-off-by Xueming Li <[email protected]>

---
 app/test-pmd/meson.build                    |   1 +
 app/test-pmd/shared_rxq_fwd.c               | 115 ++++++++++++++++++++
 app/test-pmd/testpmd.c                      |   1 +
 app/test-pmd/testpmd.h                      |   1 +
 doc/guides/testpmd_app_ug/run_app.rst       |   5 +-
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +-
 6 files changed, 126 insertions(+), 2 deletions(-)
 create mode 100644 app/test-pmd/shared_rxq_fwd.c

app/test-pmd/meson.build +1 -0

index 1ad54caef2c..b5a0f7b6209 100644
@@ -22,6 +22,7 @@ sources = files(
         'noisy_vnf.c',
         'parameters.c',
         'rxonly.c',
+        'shared_rxq_fwd.c',
         'testpmd.c',
         'txonly.c',
         'util.c',

app/test-pmd/shared_rxq_fwd.c new file +115 -0

115 lines not on this page · raw message

app/test-pmd/testpmd.c +1 -0

index d76d298a4b9..6d5bbc82404 100644
@@ -188,6 +188,7 @@ struct fwd_engine * fwd_engines[] = {
 #ifdef RTE_LIBRTE_IEEE1588
 	&ieee1588_fwd_engine,
 #endif
+	&shared_rxq_engine,
 	NULL,
 };

app/test-pmd/testpmd.h +1 -0

index 9482dab3071..bf3669134aa 100644
@@ -339,6 +339,7 @@ extern struct fwd_engine five_tuple_swap_fwd_engine;
 #ifdef RTE_LIBRTE_IEEE1588
 extern struct fwd_engine ieee1588_fwd_engine;
 #endif
+extern struct fwd_engine shared_rxq_engine;
 
 extern struct fwd_engine * fwd_engines[]; /**< NULL terminated array. */
 extern cmdline_parse_inst_t cmd_set_raw;

doc/guides/testpmd_app_ug/run_app.rst +4 -1

index faa3efb902c..d23e0b6a7a2 100644
@@ -258,6 +258,7 @@ The command line options are:
        tm
        noisy
        5tswap
+       shared-rxq
 
 *   ``--rss-ip``

@@ -399,7 +400,9 @@ The command line options are:
 
     Create queues in shared Rx queue mode if device supports.
     Shared Rx queues are grouped per X ports. X defaults to UINT32_MAX,
-    implies all ports join share group 1.
+    implies all ports join share group 1. Forwarding engine "shared-rxq"
+    should be used for shared Rx queues. This engine does Rx only and
+    update stream statistics accordingly.
 
 *   ``--eth-link-speed``

doc/guides/testpmd_app_ug/testpmd_funcs.rst +4 -1

index 6d127d9a7bc..78d23429c42 100644
@@ -314,7 +314,7 @@ set fwd
 Set the packet forwarding mode::
 
    testpmd> set fwd (io|mac|macswap|flowgen| \
-                     rxonly|txonly|csum|icmpecho|noisy|5tswap) (""|retry)
+                     rxonly|txonly|csum|icmpecho|noisy|5tswap|shared-rxq) (""|retry)
 
 ``retry`` can be specified for forwarding engines except ``rx_only``.

@@ -357,6 +357,9 @@ The available information categories are:
 
   L4 swaps the source port and destination port of transport layer (TCP and UDP).
 
+* ``shared-rxq``: Receive only for shared Rx queue.
+  Resolve packet source port from mbuf and update stream statistics accordingly.
+
 Example::
 
    testpmd> set fwd rxonly

Signature · 1 line

2.33.0

Ferruh Yigit

2021-10-21 23:41 UTC #227 lore raw

In reply to Xueming Li

Quoted 84 lines · Xueming Li

In current DPDK framework, all Rx queues is pre-loaded with mbufs for incoming packets. When number of representors scale out in a switch domain, the memory consumption became significant. Further more, polling all ports leads to high cache miss, high latency and low throughputs.
This patch introduces shared Rx queue. PF and representors in same Rx domain and switch domain could share Rx queue set by specifying non-zero share group value in Rx queue configuration.
All ports that share Rx queue actually shares hardware descriptor queue and feed all Rx queues with one descriptor supply, memory is saved.
Polling any queue using same shared Rx queue receives packets from all member ports. Source port is identified by mbuf->port.
Multiple groups is supported by group ID. Port queue number in a shared group should be identical. Queue index is 1:1 mapped in shared group. An example of two share groups:
Group1, 4 shared Rx queues per member port: PF, repr0, repr1 Group2, 2 shared Rx queues per member port: repr2, repr3, ... repr127 Poll first port for each group:
   core	port	queue
   0	0	0
   1	0	1
   2	0	2
   3	0	3
   4	2	0
   5	2	1
Shared Rx queue must be polled on single thread or core. If both PF0 and representor0 joined same share group, can't poll pf0rxq0 on core1 and rep0rxq0 on core2. Actually, polling one port within share group is sufficient since polling any port in group will return packets for any port in group.
There was some discussion to aggregate member ports in same group into a dummy port, several ways to achieve it. Since it optional, need to collect more feedback and requirement from user, make better decision later.
v1:
- initial version
v2:
- add testpmd patches
v3:
- change common forwarding api to macro for performance, thanks Jerin.
- save global variable accessed in forwarding to flowstream to minimize cache miss
- combined patches for each forwarding engine
- support multiple groups in testpmd "--share-rxq" parameter
- new api to aggregate shared rxq group
v4:
- spelling fixes
- remove shared-rxq support for all forwarding engines
- add dedicate shared-rxq forwarding engine
v5:
- fix grammars
- remove aggregate api and leave it for later discussion
- add release notes
- add deployment example
v6:
- replace RxQ offload flag with device offload capability flag
- add Rx domain
- RxQ is shared when share group > 0
- update testpmd accordingly
v7:
- fix testpmd share group id allocation
- change rx_domain to 16bits
v8:
- add new patch for testpmd to show device Rx domain ID and capability
- new share_qid in RxQ configuration
v9:
- fix some spelling
v10:
- add device capability name api
v11:
- remove macro from device capability name list
v12:
- rephrase
- in forwarding core check, add global flag and RxQ enabled check
v13:
- update imports of new forwarding engine
- rephrase

Quoted 9 lines

Xueming Li (7):
ethdev: introduce shared Rx queue
ethdev: get device capability name as string app/testpmd: dump device capability and Rx domain info app/testpmd: new parameter to enable shared Rx queue app/testpmd: dump port info for shared Rx queue app/testpmd: force shared Rx queue polled on same core app/testpmd: add forwarding engine for shared Rx queue

This patch is changing some common ethdev structs for a use case I am not sure how common, I would like to see more reviews from more vendors but we didn't get, at this stage I will proceed based on Andres's review.

Since only nvidia will be able to test this feature in this release, can you please be sure nvidia test report contains this feature? To be sure the feature is tested at least by a vendor.

Series applied to dpdk-next-net/main, thanks.

Xueming(Steven) Li

author

2021-10-22 06:31 UTC #228 lore raw

Quoted 99 lines · Ferruh Yigit

Quoted 84 lines · Xueming Li
In current DPDK framework, all Rx queues is pre-loaded with mbufs for incoming packets. When number of representors scale out in a switch domain, the memory consumption became significant. Further more, polling all ports leads to high cache miss, high latency and low throughputs.
This patch introduces shared Rx queue. PF and representors in same Rx domain and switch domain could share Rx queue set by specifying non-zero share group value in Rx queue configuration.
All ports that share Rx queue actually shares hardware descriptor queue and feed all Rx queues with one descriptor supply, memory is saved.
Polling any queue using same shared Rx queue receives packets from all member ports. Source port is identified by mbuf->port.
Multiple groups is supported by group ID. Port queue number in a shared group should be identical. Queue index is 1:1 mapped in shared group. An example of two share groups:
Group1, 4 shared Rx queues per member port: PF, repr0, repr1 Group2, 2 shared Rx queues per member port: repr2, repr3, ... repr127 Poll first port for each group:
   core	port	queue
   0	0	0
   1	0	1
   2	0	2
   3	0	3
   4	2	0
   5	2	1
Shared Rx queue must be polled on single thread or core. If both PF0 and representor0 joined same share group, can't poll pf0rxq0 on core1 and rep0rxq0 on core2. Actually, polling one port within share group is sufficient since polling any port in group will return packets for any port in group.
There was some discussion to aggregate member ports in same group into a dummy port, several ways to achieve it. Since it optional, need to collect more feedback and requirement from user, make better decision later.
v1:
- initial version
v2:
- add testpmd patches
v3:
- change common forwarding api to macro for performance, thanks Jerin.
- save global variable accessed in forwarding to flowstream to minimize cache miss
- combined patches for each forwarding engine
- support multiple groups in testpmd "--share-rxq" parameter
- new api to aggregate shared rxq group
v4:
- spelling fixes
- remove shared-rxq support for all forwarding engines
- add dedicate shared-rxq forwarding engine
v5:
- fix grammars
- remove aggregate api and leave it for later discussion
- add release notes
- add deployment example
v6:
- replace RxQ offload flag with device offload capability flag
- add Rx domain
- RxQ is shared when share group > 0
- update testpmd accordingly
v7:
- fix testpmd share group id allocation
- change rx_domain to 16bits
v8:
- add new patch for testpmd to show device Rx domain ID and capability
- new share_qid in RxQ configuration
v9:
- fix some spelling
v10:
- add device capability name api
v11:
- remove macro from device capability name list
v12:
- rephrase
- in forwarding core check, add global flag and RxQ enabled check
v13:
- update imports of new forwarding engine
- rephrase
Quoted 9 lines
Xueming Li (7):
ethdev: introduce shared Rx queue
ethdev: get device capability name as string app/testpmd: dump device capability and Rx domain info app/testpmd: new parameter to enable shared Rx queue app/testpmd: dump port info for shared Rx queue app/testpmd: force shared Rx queue polled on same core app/testpmd: add forwarding engine for shared Rx queue
This patch is changing some common ethdev structs for a use case I am not sure how common, I would like to see more reviews from more vendors but we didn't get, at this stage I will proceed based on Andres's review.

Quoted 6 lines

Since only nvidia will be able to test this feature in this release, can you please be sure nvidia test report contains this feature? To be sure the feature is tested at least by a vendor.
Series applied to dpdk-next-net/main, thanks.

Hi Ferruh,

Thanks very much for your help!

+Raslan, Ali
Let's make sure the test report contains this feature.

Best Regards,
Xueming Li

Tom Barbette

2021-11-04 15:52 UTC #229 lore raw

In reply to Xueming Li

Le 21-10-21 à 12:41, Xueming Li a écrit :

Quoted 110 lines

In current DPDK framework, all Rx queues is pre-loaded with mbufs for incoming packets. When number of representors scale out in a switch domain, the memory consumption became significant. Further more, polling all ports leads to high cache miss, high latency and low throughputs.
This patch introduces shared Rx queue. PF and representors in same Rx domain and switch domain could share Rx queue set by specifying non-zero share group value in Rx queue configuration.
All ports that share Rx queue actually shares hardware descriptor queue and feed all Rx queues with one descriptor supply, memory is saved.
Polling any queue using same shared Rx queue receives packets from all member ports. Source port is identified by mbuf->port.
Multiple groups is supported by group ID. Port queue number in a shared group should be identical. Queue index is 1:1 mapped in shared group. An example of two share groups:
Group1, 4 shared Rx queues per member port: PF, repr0, repr1 Group2, 2 shared Rx queues per member port: repr2, repr3, ... repr127 Poll first port for each group:
   core	port	queue
   0	0	0
   1	0	1
   2	0	2
   3	0	3
   4	2	0
   5	2	1
Shared Rx queue must be polled on single thread or core. If both PF0 and representor0 joined same share group, can't poll pf0rxq0 on core1 and rep0rxq0 on core2. Actually, polling one port within share group is sufficient since polling any port in group will return packets for any port in group.
There was some discussion to aggregate member ports in same group into a dummy port, several ways to achieve it. Since it optional, need to collect more feedback and requirement from user, make better decision later.
v1:
- initial version
v2:
- add testpmd patches
v3:
- change common forwarding api to macro for performance, thanks Jerin.
- save global variable accessed in forwarding to flowstream to minimize cache miss
- combined patches for each forwarding engine
- support multiple groups in testpmd "--share-rxq" parameter
- new api to aggregate shared rxq group
v4:
- spelling fixes
- remove shared-rxq support for all forwarding engines
- add dedicate shared-rxq forwarding engine
v5:
- fix grammars
- remove aggregate api and leave it for later discussion
- add release notes
- add deployment example
v6:
- replace RxQ offload flag with device offload capability flag
- add Rx domain
- RxQ is shared when share group > 0
- update testpmd accordingly
v7:
- fix testpmd share group id allocation
- change rx_domain to 16bits
v8:
- add new patch for testpmd to show device Rx domain ID and capability
- new share_qid in RxQ configuration
v9:
- fix some spelling
v10:
- add device capability name api
v11:
- remove macro from device capability name list
v12:
- rephrase
- in forwarding core check, add global flag and RxQ enabled check
v13:
- update imports of new forwarding engine
- rephrase
Xueming Li (7):
ethdev: introduce shared Rx queue
ethdev: get device capability name as string app/testpmd: dump device capability and Rx domain info app/testpmd: new parameter to enable shared Rx queue app/testpmd: dump port info for shared Rx queue app/testpmd: force shared Rx queue polled on same core app/testpmd: add forwarding engine for shared Rx queue
  app/test-pmd/config.c                         | 141 +++++++++++++++++-
  app/test-pmd/meson.build                      |   1 +
  app/test-pmd/parameters.c                     |  13 ++
  app/test-pmd/shared_rxq_fwd.c                 | 115 ++++++++++++++
  app/test-pmd/testpmd.c                        |  26 +++-
  app/test-pmd/testpmd.h                        |   5 +
  app/test-pmd/util.c                           |   3 +
  doc/guides/nics/features.rst                  |  13 ++
  doc/guides/nics/features/default.ini          |   1 +
  .../prog_guide/switch_representation.rst      |  11 ++
  doc/guides/rel_notes/release_21_11.rst        |   6 +
  doc/guides/testpmd_app_ug/run_app.rst         |   9 ++
  doc/guides/testpmd_app_ug/testpmd_funcs.rst   |   5 +-
  lib/ethdev/rte_ethdev.c                       |  33 ++++
  lib/ethdev/rte_ethdev.h                       |  38 +++++
  lib/ethdev/version.map                        |   1 +
  16 files changed, 415 insertions(+), 6 deletions(-)

Quoted 1 line

create mode 100644 app/test-pmd/shared_rxq_fwd.c

Hi all,

Sorry to jump in this late but I think this solves only a consequence of another "problem", the fact the mbuf descriptor is coupled with the buffer. And you might want to consider another approach that does not require API change.

The problem (partially solved by this patch) is that you'll "touch" many descriptors (the rte_mbuf itself) if you have many queues, or even a few queues but with quite large rings. Those descriptors, will all be likely out of cache when you access them. However, as we demonstrated with mlx5 (see https://packetmill.io/) you can build a descriptor from scratch out of the NIC hw ring that points to the underlying buffer in an indirect way. This descriptor can be taken out of the thread-local buffer pool. You'll actualy keep as much mbufs descriptors in-flight as your burst size. Which probably even defeats what this patch can do, as you can actually use only 32 descriptors per thread for any number of queues of any size.

What that solution does not solve is the need to poll many different queues. I think that is orthogonal, with the NICs getting smarter we're going to have many rules sending traffic to per-application, per-priority queues anyway. Maybe even per-microflows. To solve this we would need a kind of queue bitmask set in hw to indicate which queue to poll instead of trying all of them. Maybe this can be done through a FW update? It's a feature we'll want in the future in any cases.

The shared RX queue is surely an easy fix for the polling itself, but one problem of the shared RX queue is that it will lead to scattered batches. We'll get batches of packets from all ports that will surely take different code path for anything above forwarding, breaking the benefit of batching (this can also lead up to 50% of performance penalty due to interleaved burst, see https://people.kth.se/~dejanko/documents/publications/ordermatters-nsdi22.pdf).

Cheers,

Tom

Xueming Li

author

2021-11-03 07:59 UTC #230 lore raw

v3 cover net/mlx5: support shared Rx queue

In reply to Xueming Li

Implemetation of Shared Rx queue.

Depends-on: series-20232 ("Flow entites behavior on port restart")

v1:
- initial version
v2:
- rebased on latest dependent series
- fully tested
v3:
- support share_qid of RxQ configuration
v4:
- internally reviewed
- removed MPRQ support
- fixed multi-segment support
- fixed configure not applied after port restart

Viacheslav Ovsiienko (1):
net/mlx5: add shared Rx queue port datapath support

Xueming Li (13):
common/mlx5: introduce user index field in completion net/mlx5: fix field reference for PPC common/mlx5: adds basic receive memory pool support common/mlx5: support receive memory pool net/mlx5: fix Rx queue memory allocation return value net/mlx5: clean Rx queue code
net/mlx5: split Rx queue into shareable and private net/mlx5: move Rx queue reference count net/mlx5: move Rx queue hairpin info to private data net/mlx5: remove port info from shareable Rx queue net/mlx5: move Rx queue DevX resource net/mlx5: remove Rx queue data list from device net/mlx5: support shared Rx queue

 doc/guides/nics/features/mlx5.ini        |   1 +
 doc/guides/nics/mlx5.rst                 |   6 +
 drivers/common/mlx5/mlx5_common_devx.c   | 295 +++++++++--
 drivers/common/mlx5/mlx5_common_devx.h   |  19 +-
 drivers/common/mlx5/mlx5_devx_cmds.c     |  52 ++
 drivers/common/mlx5/mlx5_devx_cmds.h     |  16 +
 drivers/common/mlx5/mlx5_prm.h           |  93 +++-
 drivers/common/mlx5/version.map          |   1 +
 drivers/net/mlx5/linux/mlx5_os.c         |   2 +
 drivers/net/mlx5/linux/mlx5_verbs.c      | 169 +++---
 drivers/net/mlx5/mlx5.c                  |  10 +-
 drivers/net/mlx5/mlx5.h                  |  17 +-
 drivers/net/mlx5/mlx5_devx.c             | 270 +++++-----
 drivers/net/mlx5/mlx5_ethdev.c           |  21 +-
 drivers/net/mlx5/mlx5_flow.c             |  47 +-
 drivers/net/mlx5/mlx5_rss.c              |   6 +-
 drivers/net/mlx5/mlx5_rx.c               |  31 +-
 drivers/net/mlx5/mlx5_rx.h               |  45 +-
 drivers/net/mlx5/mlx5_rxq.c              | 631 +++++++++++++++++------
 drivers/net/mlx5/mlx5_rxtx.c             |   6 +-
 drivers/net/mlx5/mlx5_rxtx_vec.c         |   8 +-
 drivers/net/mlx5/mlx5_rxtx_vec_altivec.h |  14 +-
 drivers/net/mlx5/mlx5_rxtx_vec_neon.h    |  12 +-
 drivers/net/mlx5/mlx5_rxtx_vec_sse.h     |   8 +-
 drivers/net/mlx5/mlx5_stats.c            |   9 +-
 drivers/net/mlx5/mlx5_trigger.c          | 155 +++---
 drivers/net/mlx5/mlx5_vlan.c             |  16 +-
 drivers/regex/mlx5/mlx5_regex_fastpath.c |   2 +-
 28 files changed, 1378 insertions(+), 584 deletions(-)

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 07:59 UTC #231 lore raw

PATCH v3 1/14 common/mlx5: introduce user index field in completion

On ConnectX devices the completion entry provides the dedicated 24-bit field, that is filled up with some static value assigned at the Receiving Queue creation moment. This patch declares this field. This is a preparation step for supporting shared RQs and the field is supposed to provide actual port index while handling the shared receiving queue(s).

Signed-off-by Xueming Li <[email protected]>

---
 drivers/common/mlx5/mlx5_prm.h           | 8 +++++++-
 drivers/regex/mlx5/mlx5_regex_fastpath.c | 2 +-
 2 files changed, 8 insertions(+), 2 deletions(-)

drivers/common/mlx5/mlx5_prm.h +7 -1

index eab80eaead9..53931ebf1cc 100644
@@ -393,7 +393,13 @@ struct mlx5_cqe {
 	uint16_t hdr_type_etc;
 	uint16_t vlan_info;
 	uint8_t lro_num_seg;
-	uint8_t rsvd3[3];
+	union {
+		uint8_t user_index_bytes[3];
+		struct {
+			uint8_t user_index_hi;
+			uint16_t user_index_low;
+		} __rte_packed;
+	};
 	uint32_t flow_table_metadata;
 	uint8_t rsvd4[4];
 	uint32_t byte_cnt;

drivers/regex/mlx5/mlx5_regex_fastpath.c +1 -1

index adb5343a46b..6836203ecf2 100644
@@ -559,7 +559,7 @@ mlx5_regexdev_dequeue(struct rte_regexdev *dev, uint16_t qp_id,
 		uint16_t wq_counter
 			= (rte_be_to_cpu_16(cqe->wqe_counter) + 1) &
 			  MLX5_REGEX_MAX_WQE_INDEX;
-		size_t hw_qpid = cqe->rsvd3[2];
+		size_t hw_qpid = cqe->user_index_bytes[2];
 		struct mlx5_regex_hw_qp *qp_obj = &queue->qps[hw_qpid];
 
 		/* UMR mode WQE counter move as WQE set(4 WQEBBS).*/

Signature · 1 line

2.33.0

Slava Ovsiienko

Acked-by on v3 1/14

2021-11-04 09:14 UTC #232 lore raw

Re: [dpdk-dev] [PATCH v3 01/14] common/mlx5: introduce user index field in completion

Quoted 10 lines

-----Original Message-----
From: Xueming(Steven) Li <[email protected]>
Sent: Wednesday, November 3, 2021 9:58
To: [email protected]
Cc: Xueming(Steven) Li <[email protected]>; Lior Margalit <[email protected]>; Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>; Ori Kam <[email protected]>
Subject: [PATCH v3 01/14] common/mlx5: introduce user index field in completion

Quoted 8 lines

On ConnectX devices the completion entry provides the dedicated 24-bit field, that is filled up with some static value assigned at the Receiving Queue creation moment. This patch declares this field. This is a preparation step for supporting shared RQs and the field is supposed to provide actual port index while handling the shared receiving queue(s).
Signed-off-by Xueming Li <[email protected]>

For the entire series:

Acked-by Viacheslav Ovsiienko <[email protected]>

Xueming Li

author

2021-11-03 07:59 UTC #233 lore raw

PATCH v3 2/14 net/mlx5: fix field reference for PPC

In reply to Xueming Li

This patch fixes stale field reference.

Fixes a18ac6113331 ("net/mlx5: add metadata support to Rx datapath")

Cc [email protected]

Signed-off-by Xueming Li <[email protected]>

---
 drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +4 -4

index bcf487c34e9..1d00c1c43d1 100644
@@ -974,10 +974,10 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 			(vector unsigned short)cqe_tmp1, cqe_sel_mask1);
 		cqe_tmp2 = (vector unsigned char)(vector unsigned long){
 			*(__rte_aligned(8) unsigned long *)
-			&cq[pos + p3].rsvd3[9], 0LL};
+			&cq[pos + p3].rsvd4[2], 0LL};
 		cqe_tmp1 = (vector unsigned char)(vector unsigned long){
 			*(__rte_aligned(8) unsigned long *)
-			&cq[pos + p2].rsvd3[9], 0LL};
+			&cq[pos + p2].rsvd4[2], 0LL};
 		cqes[3] = (vector unsigned char)
 			vec_sel((vector unsigned short)cqes[3],
 			(vector unsigned short)cqe_tmp2,

@@ -1037,10 +1037,10 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 			(vector unsigned short)cqe_tmp1, cqe_sel_mask1);
 		cqe_tmp2 = (vector unsigned char)(vector unsigned long){
 			*(__rte_aligned(8) unsigned long *)
-			&cq[pos + p1].rsvd3[9], 0LL};
+			&cq[pos + p1].rsvd4[2], 0LL};
 		cqe_tmp1 = (vector unsigned char)(vector unsigned long){
 			*(__rte_aligned(8) unsigned long *)
-			&cq[pos].rsvd3[9], 0LL};
+			&cq[pos].rsvd4[2], 0LL};
 		cqes[1] = (vector unsigned char)
 			vec_sel((vector unsigned short)cqes[1],
 			(vector unsigned short)cqe_tmp2, cqe_sel_mask2);

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 07:59 UTC #234 lore raw

PATCH v3 3/14 common/mlx5: adds basic receive memory pool support

In reply to Xueming Li

The hardware Receive Memory Pool (RMP) object holds the destination for incoming packets/messages that are routed to the RMP through RQs. RMP enables sharing of memory across multiple Receive Queues. Multiple Receive Queues can be attached to the same RMP and consume memory from that shared poll. When using RMPs, completions are reported to the CQ pointed to by the RQ, and this Completion Queue can be shared as well.

This patch adds DevX supports of PRM RMP object.

Signed-off-by Xueming Li <[email protected]>

---
 drivers/common/mlx5/mlx5_devx_cmds.c | 52 +++++++++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h | 16 ++++++
 drivers/common/mlx5/mlx5_prm.h       | 85 +++++++++++++++++++++++++++-
 drivers/common/mlx5/version.map      |  1 +
 4 files changed, 153 insertions(+), 1 deletion(-)

drivers/common/mlx5/mlx5_devx_cmds.c +52 -0

index fb7c8e986f8..119641df470 100644
@@ -766,6 +766,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 			MLX5_GET(cmd_hca_cap, hcattr, flow_counter_bulk_alloc);
 	attr->flow_counters_dump = MLX5_GET(cmd_hca_cap, hcattr,
 					    flow_counters_dump);
+	attr->log_max_rmp = MLX5_GET(cmd_hca_cap, hcattr, log_max_rmp);
+	attr->mem_rq_rmp = MLX5_GET(cmd_hca_cap, hcattr, mem_rq_rmp);
 	attr->log_max_rqt_size = MLX5_GET(cmd_hca_cap, hcattr,
 					  log_max_rqt_size);
 	attr->eswitch_manager = MLX5_GET(cmd_hca_cap, hcattr, eswitch_manager);

@@ -1277,6 +1279,56 @@ mlx5_devx_cmd_modify_rq(struct mlx5_devx_obj *rq,
 }
 
 /**
+ * Create RMP using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param [in] rmp_attr
+ *   Pointer to create RMP attributes structure.
+ * @param [in] socket
+ *   CPU socket ID for allocations.
+ *
+ * @return
+ *   The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_rmp(void *ctx,
+			 struct mlx5_devx_create_rmp_attr *rmp_attr,
+			 int socket)
+{
+	uint32_t in[MLX5_ST_SZ_DW(create_rmp_in)] = {0};
+	uint32_t out[MLX5_ST_SZ_DW(create_rmp_out)] = {0};
+	void *rmp_ctx, *wq_ctx;
+	struct mlx5_devx_wq_attr *wq_attr;
+	struct mlx5_devx_obj *rmp = NULL;
+
+	rmp = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*rmp), 0, socket);
+	if (!rmp) {
+		DRV_LOG(ERR, "Failed to allocate RMP data");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	MLX5_SET(create_rmp_in, in, opcode, MLX5_CMD_OP_CREATE_RMP);
+	rmp_ctx = MLX5_ADDR_OF(create_rmp_in, in, ctx);
+	MLX5_SET(rmpc, rmp_ctx, state, rmp_attr->state);
+	MLX5_SET(rmpc, rmp_ctx, basic_cyclic_rcv_wqe,
+		 rmp_attr->basic_cyclic_rcv_wqe);
+	wq_ctx = MLX5_ADDR_OF(rmpc, rmp_ctx, wq);
+	wq_attr = &rmp_attr->wq_attr;
+	devx_cmd_fill_wq_data(wq_ctx, wq_attr);
+	rmp->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in), out,
+					      sizeof(out));
+	if (!rmp->obj) {
+		DRV_LOG(ERR, "Failed to create RMP using DevX");
+		rte_errno = errno;
+		mlx5_free(rmp);
+		return NULL;
+	}
+	rmp->id = MLX5_GET(create_rmp_out, out, rmpn);
+	return rmp;
+}
+
+/*
  * Create TIR using DevX API.
  *
  * @param[in] ctx

drivers/common/mlx5/mlx5_devx_cmds.h +16 -0

index 80b5dca1eb4..5759c4c9473 100644
@@ -93,6 +93,8 @@ struct mlx5_hca_flow_attr {
 struct mlx5_hca_attr {
 	uint32_t eswitch_manager:1;
 	uint32_t flow_counters_dump:1;
+	uint32_t mem_rq_rmp:1;
+	uint32_t log_max_rmp:5;
 	uint32_t log_max_rqt_size:5;
 	uint32_t parse_graph_flex_node:1;
 	uint8_t flow_counter_bulk_alloc_bitmap;

@@ -259,6 +261,17 @@ struct mlx5_devx_modify_rq_attr {
 	uint32_t lwm:16; /* Contained WQ lwm. */
 };
 
+/* Create RMP attributes structure, used by create RMP operation. */
+struct mlx5_devx_create_rmp_attr {
+	uint32_t rsvd0:8;
+	uint32_t state:4;
+	uint32_t rsvd1:20;
+	uint32_t basic_cyclic_rcv_wqe:1;
+	uint32_t rsvd4:31;
+	uint32_t rsvd8[10];
+	struct mlx5_devx_wq_attr wq_attr;
+};
+
 struct mlx5_rx_hash_field_select {
 	uint32_t l3_prot_type:1;
 	uint32_t l4_prot_type:1;

@@ -536,6 +549,9 @@ __rte_internal
 int mlx5_devx_cmd_modify_rq(struct mlx5_devx_obj *rq,
 			    struct mlx5_devx_modify_rq_attr *rq_attr);
 __rte_internal
+struct mlx5_devx_obj *mlx5_devx_cmd_create_rmp(void *ctx,
+			struct mlx5_devx_create_rmp_attr *rq_attr, int socket);
+__rte_internal
 struct mlx5_devx_obj *mlx5_devx_cmd_create_tir(void *ctx,
 					   struct mlx5_devx_tir_attr *tir_attr);
 __rte_internal

drivers/common/mlx5/mlx5_prm.h +84 -1

103 lines not on this page · raw message

drivers/common/mlx5/version.map +1 -0

index 0ea8325f9ac..7265ff8c56f 100644
@@ -30,6 +30,7 @@ INTERNAL {
 	mlx5_devx_cmd_create_geneve_tlv_option;
 	mlx5_devx_cmd_create_import_kek_obj;
 	mlx5_devx_cmd_create_qp;
+	mlx5_devx_cmd_create_rmp;
 	mlx5_devx_cmd_create_rq;
 	mlx5_devx_cmd_create_rqt;
 	mlx5_devx_cmd_create_sq;

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 07:59 UTC #235 lore raw

PATCH v3 4/14 common/mlx5: support receive memory pool

In reply to Xueming Li

This patch enables RMP based RQ, RMP is created when mlx5_devx_rq.rmp is set.

Signed-off-by Xueming Li <[email protected]>

---
 drivers/common/mlx5/mlx5_common_devx.c | 295 +++++++++++++++++++++----
 drivers/common/mlx5/mlx5_common_devx.h |  19 +-
 drivers/net/mlx5/mlx5_devx.c           |   4 +-
 3 files changed, 271 insertions(+), 47 deletions(-)

drivers/common/mlx5/mlx5_common_devx.c +253 ~~-42~~

index 825f84b1833..85b5282061a 100644
@@ -271,6 +271,39 @@ mlx5_devx_sq_create(void *ctx, struct mlx5_devx_sq *sq_obj, uint16_t log_wqbb_n,
 	return -rte_errno;
 }
 
+/**
+ * Destroy DevX Receive Queue resources.
+ *
+ * @param[in] rq_res
+ *   DevX RQ resource to destroy.
+ */
+static void
+mlx5_devx_wq_res_destroy(struct mlx5_devx_wq_res *rq_res)
+{
+	if (rq_res->umem_obj)
+		claim_zero(mlx5_os_umem_dereg(rq_res->umem_obj));
+	if (rq_res->umem_buf)
+		mlx5_free((void *)(uintptr_t)rq_res->umem_buf);
+	memset(rq_res, 0, sizeof(*rq_res));
+}
+
+/**
+ * Destroy DevX Receive Memory Pool.
+ *
+ * @param[in] rmp
+ *   DevX RMP to destroy.
+ */
+static void
+mlx5_devx_rmp_destroy(struct mlx5_devx_rmp *rmp)
+{
+	MLX5_ASSERT(rmp->ref_cnt == 0);
+	if (rmp->rmp) {
+		claim_zero(mlx5_devx_cmd_destroy(rmp->rmp));
+		rmp->rmp = NULL;
+	}
+	mlx5_devx_wq_res_destroy(&rmp->wq);
+}
+
 /**
  * Destroy DevX Queue Pair.
  *

@@ -389,55 +422,48 @@ mlx5_devx_qp_create(void *ctx, struct mlx5_devx_qp *qp_obj, uint16_t log_wqbb_n,
 void
 mlx5_devx_rq_destroy(struct mlx5_devx_rq *rq)
 {
-	if (rq->rq)
+	if (rq->rq) {
 		claim_zero(mlx5_devx_cmd_destroy(rq->rq));
-	if (rq->umem_obj)
-		claim_zero(mlx5_os_umem_dereg(rq->umem_obj));
-	if (rq->umem_buf)
-		mlx5_free((void *)(uintptr_t)rq->umem_buf);
+		rq->rq = NULL;
+		if (rq->rmp)
+			rq->rmp->ref_cnt--;
+	}
+	if (rq->rmp == NULL) {
+		mlx5_devx_wq_res_destroy(&rq->wq);
+	} else {
+		if (rq->rmp->ref_cnt == 0)
+			mlx5_devx_rmp_destroy(rq->rmp);
+	}
 }
 
 /**
- * Create Receive Queue using DevX API.
- *
- * Get a pointer to partially initialized attributes structure, and updates the
- * following fields:
- *   wq_umem_valid
- *   wq_umem_id
- *   wq_umem_offset
- *   dbr_umem_valid
- *   dbr_umem_id
- *   dbr_addr
- *   log_wq_pg_sz
- * All other fields are updated by caller.
+ * Create WQ resources using DevX API.
  *
  * @param[in] ctx
  *   Context returned from mlx5 open_device() glue function.
- * @param[in/out] rq_obj
- *   Pointer to RQ to create.
  * @param[in] wqe_size
  *   Size of WQE structure.
  * @param[in] log_wqbb_n
  *   Log of number of WQBBs in queue.
- * @param[in] attr
- *   Pointer to RQ attributes structure.
  * @param[in] socket
  *   Socket to use for allocation.
+ * @param[out] wq_attr
+ *   Pointer to WQ attributes structure.
+ * @param[out] wq_res
+ *   Pointer to WQ resource to create.
  *
  * @return
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
-int
-mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj, uint32_t wqe_size,
-		    uint16_t log_wqbb_n,
-		    struct mlx5_devx_create_rq_attr *attr, int socket)
+static int
+mlx5_devx_wq_init(void *ctx, uint32_t wqe_size, uint16_t log_wqbb_n, int socket,
+		  struct mlx5_devx_wq_attr *wq_attr,
+		  struct mlx5_devx_wq_res *wq_res)
 {
-	struct mlx5_devx_obj *rq = NULL;
 	struct mlx5dv_devx_umem *umem_obj = NULL;
 	void *umem_buf = NULL;
 	size_t alignment = MLX5_WQE_BUF_ALIGNMENT;
 	uint32_t umem_size, umem_dbrec;
-	uint16_t rq_size = 1 << log_wqbb_n;
 	int ret;
 
 	if (alignment == (size_t)-1) {

@@ -446,7 +472,7 @@ mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj, uint32_t wqe_size,
 		return -rte_errno;
 	}
 	/* Allocate memory buffer for WQEs and doorbell record. */
-	umem_size = wqe_size * rq_size;
+	umem_size = wqe_size * (1 << log_wqbb_n);
 	umem_dbrec = RTE_ALIGN(umem_size, MLX5_DBR_SIZE);
 	umem_size += MLX5_DBR_SIZE;
 	umem_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, umem_size,

@@ -464,14 +490,60 @@ mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj, uint32_t wqe_size,
 		rte_errno = errno;
 		goto error;
 	}
+	/* Fill WQ attributes for RQ/RMP object creation. */
+	wq_attr->wq_umem_valid = 1;
+	wq_attr->wq_umem_id = mlx5_os_get_umem_id(umem_obj);
+	wq_attr->wq_umem_offset = 0;
+	wq_attr->dbr_umem_valid = 1;
+	wq_attr->dbr_umem_id = wq_attr->wq_umem_id;
+	wq_attr->dbr_addr = umem_dbrec;
+	wq_attr->log_wq_pg_sz = MLX5_LOG_PAGE_SIZE;
 	/* Fill attributes for RQ object creation. */
-	attr->wq_attr.wq_umem_valid = 1;
-	attr->wq_attr.wq_umem_id = mlx5_os_get_umem_id(umem_obj);
-	attr->wq_attr.wq_umem_offset = 0;
-	attr->wq_attr.dbr_umem_valid = 1;
-	attr->wq_attr.dbr_umem_id = attr->wq_attr.wq_umem_id;
-	attr->wq_attr.dbr_addr = umem_dbrec;
-	attr->wq_attr.log_wq_pg_sz = MLX5_LOG_PAGE_SIZE;
+	wq_res->umem_buf = umem_buf;
+	wq_res->umem_obj = umem_obj;
+	wq_res->db_rec = RTE_PTR_ADD(umem_buf, umem_dbrec);
+	return 0;
+error:
+	ret = rte_errno;
+	if (umem_obj)
+		claim_zero(mlx5_os_umem_dereg(umem_obj));
+	if (umem_buf)
+		mlx5_free((void *)(uintptr_t)umem_buf);
+	rte_errno = ret;
+	return -rte_errno;
+}
+
+/**
+ * Create standalone Receive Queue using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param[in/out] rq_obj
+ *   Pointer to RQ to create.
+ * @param[in] wqe_size
+ *   Size of WQE structure.
+ * @param[in] log_wqbb_n
+ *   Log of number of WQBBs in queue.
+ * @param[in] attr
+ *   Pointer to RQ attributes structure.
+ * @param[in] socket
+ *   Socket to use for allocation.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+mlx5_devx_rq_std_create(void *ctx, struct mlx5_devx_rq *rq_obj,
+			uint32_t wqe_size, uint16_t log_wqbb_n,
+			struct mlx5_devx_create_rq_attr *attr, int socket)
+{
+	struct mlx5_devx_obj *rq;
+	int ret;
+
+	ret = mlx5_devx_wq_init(ctx, wqe_size, log_wqbb_n, socket,
+				&attr->wq_attr, &rq_obj->wq);
+	if (ret != 0)
+		return ret;
 	/* Create receive queue object with DevX. */
 	rq = mlx5_devx_cmd_create_rq(ctx, attr, socket);
 	if (!rq) {

@@ -479,21 +551,160 @@ mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj, uint32_t wqe_size,
 		rte_errno = ENOMEM;
 		goto error;
 	}
-	rq_obj->umem_buf = umem_buf;
-	rq_obj->umem_obj = umem_obj;
 	rq_obj->rq = rq;
-	rq_obj->db_rec = RTE_PTR_ADD(rq_obj->umem_buf, umem_dbrec);
 	return 0;
 error:
 	ret = rte_errno;
-	if (umem_obj)
-		claim_zero(mlx5_os_umem_dereg(umem_obj));
-	if (umem_buf)
-		mlx5_free((void *)(uintptr_t)umem_buf);
+	mlx5_devx_wq_res_destroy(&rq_obj->wq);
+	rte_errno = ret;
+	return -rte_errno;
+}
+
+/**
+ * Create Receive Memory Pool using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param[in/out] rq_obj
+ *   Pointer to RQ to create.
+ * @param[in] wqe_size
+ *   Size of WQE structure.
+ * @param[in] log_wqbb_n
+ *   Log of number of WQBBs in queue.
+ * @param[in] attr
+ *   Pointer to RQ attributes structure.
+ * @param[in] socket
+ *   Socket to use for allocation.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+mlx5_devx_rmp_create(void *ctx, struct mlx5_devx_rmp *rmp_obj,
+		     uint32_t wqe_size, uint16_t log_wqbb_n,
+		     struct mlx5_devx_wq_attr *wq_attr, int socket)
+{
+	struct mlx5_devx_create_rmp_attr rmp_attr = { 0 };
+	int ret;
+
+	if (rmp_obj->rmp != NULL)
+		return 0;
+	rmp_attr.wq_attr = *wq_attr;
+	ret = mlx5_devx_wq_init(ctx, wqe_size, log_wqbb_n, socket,
+				&rmp_attr.wq_attr, &rmp_obj->wq);
+	if (ret != 0)
+		return ret;
+	rmp_attr.state = MLX5_RMPC_STATE_RDY;
+	rmp_attr.basic_cyclic_rcv_wqe =
+		wq_attr->wq_type != MLX5_WQ_TYPE_CYCLIC_STRIDING_RQ;
+	/* Create receive memory pool object with DevX. */
+	rmp_obj->rmp = mlx5_devx_cmd_create_rmp(ctx, &rmp_attr, socket);
+	if (rmp_obj->rmp == NULL) {
+		DRV_LOG(ERR, "Can't create DevX RMP object.");
+		rte_errno = ENOMEM;
+		goto error;
+	}
+	return 0;
+error:
+	ret = rte_errno;
+	mlx5_devx_wq_res_destroy(&rmp_obj->wq);
+	rte_errno = ret;
+	return -rte_errno;
+}
+
+/**
+ * Create Shared Receive Queue based on RMP using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param[in/out] rq_obj
+ *   Pointer to RQ to create.
+ * @param[in] wqe_size
+ *   Size of WQE structure.
+ * @param[in] log_wqbb_n
+ *   Log of number of WQBBs in queue.
+ * @param[in] attr
+ *   Pointer to RQ attributes structure.
+ * @param[in] socket
+ *   Socket to use for allocation.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+mlx5_devx_rq_shared_create(void *ctx, struct mlx5_devx_rq *rq_obj,
+			   uint32_t wqe_size, uint16_t log_wqbb_n,
+			   struct mlx5_devx_create_rq_attr *attr, int socket)
+{
+	struct mlx5_devx_obj *rq;
+	int ret;
+
+	ret = mlx5_devx_rmp_create(ctx, rq_obj->rmp, wqe_size, log_wqbb_n,
+				   &attr->wq_attr, socket);
+	if (ret != 0)
+		return ret;
+	attr->mem_rq_type = MLX5_RQC_MEM_RQ_TYPE_MEMORY_RQ_RMP;
+	attr->rmpn = rq_obj->rmp->rmp->id;
+	attr->flush_in_error_en = 0;
+	memset(&attr->wq_attr, 0, sizeof(attr->wq_attr));
+	/* Create receive queue object with DevX. */
+	rq = mlx5_devx_cmd_create_rq(ctx, attr, socket);
+	if (!rq) {
+		DRV_LOG(ERR, "Can't create DevX RMP RQ object.");
+		rte_errno = ENOMEM;
+		goto error;
+	}
+	rq_obj->rq = rq;
+	rq_obj->rmp->ref_cnt++;
+	return 0;
+error:
+	ret = rte_errno;
+	mlx5_devx_rq_destroy(rq_obj);
 	rte_errno = ret;
 	return -rte_errno;
 }
 
+/**
+ * Create Receive Queue using DevX API. Shared RQ is created only if rmp set.
+ *
+ * Get a pointer to partially initialized attributes structure, and updates the
+ * following fields:
+ *   wq_umem_valid
+ *   wq_umem_id
+ *   wq_umem_offset
+ *   dbr_umem_valid
+ *   dbr_umem_id
+ *   dbr_addr
+ *   log_wq_pg_sz
+ * All other fields are updated by caller.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param[in/out] rq_obj
+ *   Pointer to RQ to create.
+ * @param[in] wqe_size
+ *   Size of WQE structure.
+ * @param[in] log_wqbb_n
+ *   Log of number of WQBBs in queue.
+ * @param[in] attr
+ *   Pointer to RQ attributes structure.
+ * @param[in] socket
+ *   Socket to use for allocation.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj,
+		    uint32_t wqe_size, uint16_t log_wqbb_n,
+		    struct mlx5_devx_create_rq_attr *attr, int socket)
+{
+	if (rq_obj->rmp == NULL)
+		return mlx5_devx_rq_std_create(ctx, rq_obj, wqe_size,
+					       log_wqbb_n, attr, socket);
+	return mlx5_devx_rq_shared_create(ctx, rq_obj, wqe_size,
+					  log_wqbb_n, attr, socket);
+}
 
 /**
  * Change QP state to RTS.

drivers/common/mlx5/mlx5_common_devx.h +16 -3

index f699405f69b..7ceac040f8b 100644
@@ -45,14 +45,27 @@ struct mlx5_devx_qp {
 	volatile uint32_t *db_rec; /* The QP doorbell record. */
 };
 
-/* DevX Receive Queue structure. */
-struct mlx5_devx_rq {
-	struct mlx5_devx_obj *rq; /* The RQ DevX object. */
+/* DevX Receive Queue resource structure. */
+struct mlx5_devx_wq_res {
 	void *umem_obj; /* The RQ umem object. */
 	volatile void *umem_buf;
 	volatile uint32_t *db_rec; /* The RQ doorbell record. */
 };
 
+/* DevX Receive Memory Pool structure. */
+struct mlx5_devx_rmp {
+	struct mlx5_devx_obj *rmp; /* The RMP DevX object. */
+	uint32_t ref_cnt; /* Reference count. */
+	struct mlx5_devx_wq_res wq;
+};
+
+/* DevX Receive Queue structure. */
+struct mlx5_devx_rq {
+	struct mlx5_devx_obj *rq; /* The RQ DevX object. */
+	struct mlx5_devx_rmp *rmp; /* Shared RQ RMP object. */
+	struct mlx5_devx_wq_res wq; /* WQ resource of standalone RQ. */
+};
+
 /* mlx5_common_devx.c */
 
 __rte_internal

drivers/net/mlx5/mlx5_devx.c +2 -2

index 424f77be790..443252df05d 100644
@@ -515,8 +515,8 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	ret = mlx5_devx_modify_rq(tmpl, MLX5_RXQ_MOD_RST2RDY);
 	if (ret)
 		goto error;
-	rxq_data->wqes = (void *)(uintptr_t)tmpl->rq_obj.umem_buf;
-	rxq_data->rq_db = (uint32_t *)(uintptr_t)tmpl->rq_obj.db_rec;
+	rxq_data->wqes = (void *)(uintptr_t)tmpl->rq_obj.wq.umem_buf;
+	rxq_data->rq_db = (uint32_t *)(uintptr_t)tmpl->rq_obj.wq.db_rec;
 	rxq_data->cq_arm_sn = 0;
 	rxq_data->cq_ci = 0;
 	mlx5_rxq_initialize(rxq_data);

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 07:59 UTC #236 lore raw

PATCH v3 5/14 net/mlx5: fix Rx queue memory allocation return value

In reply to Xueming Li

If error happened during Rx queue mbuf allocation, boolean value returned. From description, return value should be error number.

This patch returns negative error number.

Fixes 0f20acbf5eda ("net/mlx5: implement vectorized MPRQ burst")

Cc [email protected]

Signed-off-by Xueming Li <[email protected]>

---
 drivers/net/mlx5/mlx5_rxq.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

drivers/net/mlx5/mlx5_rxq.c +5 -3

index 9220bb2c15c..4567b43c1b6 100644
@@ -129,7 +129,7 @@ rxq_alloc_elts_mprq(struct mlx5_rxq_ctrl *rxq_ctrl)
  *   Pointer to RX queue structure.
  *
  * @return
- *   0 on success, errno value on failure.
+ *   0 on success, negative errno value on failure.
  */
 static int
 rxq_alloc_elts_sprq(struct mlx5_rxq_ctrl *rxq_ctrl)

@@ -220,7 +220,7 @@ rxq_alloc_elts_sprq(struct mlx5_rxq_ctrl *rxq_ctrl)
  *   Pointer to RX queue structure.
  *
  * @return
- *   0 on success, errno value on failure.
+ *   0 on success, negative errno value on failure.
  */
 int
 rxq_alloc_elts(struct mlx5_rxq_ctrl *rxq_ctrl)

@@ -233,7 +233,9 @@ rxq_alloc_elts(struct mlx5_rxq_ctrl *rxq_ctrl)
 	 */
 	if (mlx5_rxq_mprq_enabled(&rxq_ctrl->rxq))
 		ret = rxq_alloc_elts_mprq(rxq_ctrl);
-	return (ret || rxq_alloc_elts_sprq(rxq_ctrl));
+	if (ret == 0)
+		ret = rxq_alloc_elts_sprq(rxq_ctrl);
+	return ret;
 }
 
 /**

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 07:59 UTC #237 lore raw

PATCH v3 6/14 net/mlx5: clean Rx queue code

In reply to Xueming Li

This patch removes unused Rx queue code.

Signed-off-by Xueming Li <[email protected]>

---
 drivers/net/mlx5/mlx5_rxq.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

drivers/net/mlx5/mlx5_rxq.c +2 -6

index 4567b43c1b6..b2e4389ad60 100644
@@ -674,9 +674,7 @@ mlx5_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc,
 		    struct rte_mempool *mp)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq = (*priv->rxqs)[idx];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_rxq_ctrl *rxq_ctrl;
 	struct rte_eth_rxseg_split *rx_seg =
 				(struct rte_eth_rxseg_split *)conf->rx_seg;
 	struct rte_eth_rxseg_split rx_single = {.mp = mp};

@@ -743,9 +741,7 @@ mlx5_rx_hairpin_queue_setup(struct rte_eth_dev *dev, uint16_t idx,
 			    const struct rte_eth_hairpin_conf *hairpin_conf)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq = (*priv->rxqs)[idx];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_rxq_ctrl *rxq_ctrl;
 	int res;
 
 	res = mlx5_rx_queue_pre_setup(dev, idx, &desc);

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 08:00 UTC #238 lore raw

PATCH v3 7/14 net/mlx5: split Rx queue into shareable and private

In reply to Xueming Li

To prepare shared Rx queue, splits RxQ data into shareable and private. Struct mlx5_rxq_priv is per queue data.
Struct mlx5_rxq_ctrl is shared queue resources and data.

Signed-off-by Xueming Li <[email protected]>

---
 drivers/net/mlx5/mlx5.c        |  4 +++
 drivers/net/mlx5/mlx5.h        |  5 ++-
 drivers/net/mlx5/mlx5_ethdev.c | 10 ++++++
 drivers/net/mlx5/mlx5_rx.h     | 17 +++++++--
 drivers/net/mlx5/mlx5_rxq.c    | 66 ++++++++++++++++++++++++++++------
 5 files changed, 88 insertions(+), 14 deletions(-)

drivers/net/mlx5/mlx5.c +4 -0

index 4ba850af263..d0fae518025 100644
@@ -1699,6 +1699,10 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 		mlx5_free(dev->intr_handle);
 		dev->intr_handle = NULL;
 	}
+	if (priv->rxq_privs != NULL) {
+		mlx5_free(priv->rxq_privs);
+		priv->rxq_privs = NULL;
+	}
 	if (priv->txqs != NULL) {
 		/* XXX race condition if mlx5_tx_burst() is still running. */
 		rte_delay_us_sleep(1000);

drivers/net/mlx5/mlx5.h +4 -1

index 39c001aa1bf..3e008241ca8 100644
@@ -1317,6 +1317,8 @@ enum mlx5_txq_modify_type {
 	MLX5_TXQ_MOD_ERR2RDY, /* modify state from error to ready. */
 };
 
+struct mlx5_rxq_priv;
+
 /* HW objects operations structure. */
 struct mlx5_obj_ops {
 	int (*rxq_obj_modify_vlan_strip)(struct mlx5_rxq_obj *rxq_obj, int on);

@@ -1380,7 +1382,8 @@ struct mlx5_priv {
 	/* RX/TX queues. */
 	unsigned int rxqs_n; /* RX queues array size. */
 	unsigned int txqs_n; /* TX queues array size. */
-	struct mlx5_rxq_data *(*rxqs)[]; /* RX queues. */
+	struct mlx5_rxq_priv *(*rxq_privs)[]; /* RX queue non-shared data. */
+	struct mlx5_rxq_data *(*rxqs)[]; /* (Shared) RX queues. */
 	struct mlx5_txq_data *(*txqs)[]; /* TX queues. */
 	struct rte_mempool *mprq_mp; /* Mempool for Multi-Packet RQ. */
 	struct rte_eth_rss_conf rss_conf; /* RSS configuration. */

drivers/net/mlx5/mlx5_ethdev.c +10 -0

index 81fa8845bb5..cde505955df 100644
@@ -104,6 +104,16 @@ mlx5_dev_configure(struct rte_eth_dev *dev)
 	       MLX5_RSS_HASH_KEY_LEN);
 	priv->rss_conf.rss_key_len = MLX5_RSS_HASH_KEY_LEN;
 	priv->rss_conf.rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+	priv->rxq_privs = mlx5_realloc(priv->rxq_privs,
+				       MLX5_MEM_RTE | MLX5_MEM_ZERO,
+				       sizeof(void *) * rxqs_n, 0,
+				       SOCKET_ID_ANY);
+	if (priv->rxq_privs == NULL) {
+		DRV_LOG(ERR, "port %u cannot allocate rxq private data",
+			dev->data->port_id);
+		rte_errno = ENOMEM;
+		return -rte_errno;
+	}
 	priv->rxqs = (void *)dev->data->rx_queues;
 	priv->txqs = (void *)dev->data->tx_queues;
 	if (txqs_n != priv->txqs_n) {

drivers/net/mlx5/mlx5_rx.h +15 -2

index 69b1263339e..fa24f5cdf3a 100644
@@ -150,10 +150,14 @@ struct mlx5_rxq_ctrl {
 	struct mlx5_rxq_data rxq; /* Data path structure. */
 	LIST_ENTRY(mlx5_rxq_ctrl) next; /* Pointer to the next element. */
 	uint32_t refcnt; /* Reference counter. */
+	LIST_HEAD(priv, mlx5_rxq_priv) owners; /* Owner rxq list. */
 	struct mlx5_rxq_obj *obj; /* Verbs/DevX elements. */
+	struct mlx5_dev_ctx_shared *sh; /* Shared context. */
 	struct mlx5_priv *priv; /* Back pointer to private data. */
 	enum mlx5_rxq_type type; /* Rxq type. */
 	unsigned int socket; /* CPU socket ID for allocations. */
+	uint32_t share_group; /* Group ID of shared RXQ. */
+	uint16_t share_qid; /* Shared RxQ ID in group. */
 	unsigned int irq:1; /* Whether IRQ is enabled. */
 	uint32_t flow_mark_n; /* Number of Mark/Flag flows using this Queue. */
 	uint32_t flow_tunnels_n[MLX5_FLOW_TUNNEL]; /* Tunnels counters. */

@@ -163,6 +167,14 @@ struct mlx5_rxq_ctrl {
 	uint32_t hairpin_status; /* Hairpin binding status. */
 };
 
+/* RX queue private data. */
+struct mlx5_rxq_priv {
+	uint16_t idx; /* Queue index. */
+	struct mlx5_rxq_ctrl *ctrl; /* Shared Rx Queue. */
+	LIST_ENTRY(mlx5_rxq_priv) owner_entry; /* Entry in shared rxq_ctrl. */
+	struct mlx5_priv *priv; /* Back pointer to private data. */
+};
+
 /* mlx5_rxq.c */
 
 extern uint8_t rss_hash_default_key[];

@@ -186,13 +198,14 @@ void mlx5_rx_intr_vec_disable(struct rte_eth_dev *dev);
 int mlx5_rx_intr_enable(struct rte_eth_dev *dev, uint16_t rx_queue_id);
 int mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id);
 int mlx5_rxq_obj_verify(struct rte_eth_dev *dev);
-struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx,
+struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev,
+				   struct mlx5_rxq_priv *rxq,
 				   uint16_t desc, unsigned int socket,
 				   const struct rte_eth_rxconf *conf,
 				   const struct rte_eth_rxseg_split *rx_seg,
 				   uint16_t n_seg);
 struct mlx5_rxq_ctrl *mlx5_rxq_hairpin_new
-	(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc,
+	(struct rte_eth_dev *dev, struct mlx5_rxq_priv *rxq, uint16_t desc,
 	 const struct rte_eth_hairpin_conf *hairpin_conf);
 struct mlx5_rxq_ctrl *mlx5_rxq_get(struct rte_eth_dev *dev, uint16_t idx);
 int mlx5_rxq_release(struct rte_eth_dev *dev, uint16_t idx);

drivers/net/mlx5/mlx5_rxq.c +55 ~~-11~~

163 lines not on this page · raw message

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 08:00 UTC #239 lore raw

PATCH v3 8/14 net/mlx5: move Rx queue reference count

In reply to Xueming Li

Rx queue reference count is counter of RQ, used to count reference to RQ object. To prepare for shared Rx queue, this patch moves it from rxq_ctrl to Rx queue private data.

Signed-off-by Xueming Li <[email protected]>

---
 drivers/net/mlx5/mlx5_rx.h      |   8 +-
 drivers/net/mlx5/mlx5_rxq.c     | 169 +++++++++++++++++++++-----------
 drivers/net/mlx5/mlx5_trigger.c |  57 +++++------
 3 files changed, 142 insertions(+), 92 deletions(-)

drivers/net/mlx5/mlx5_rx.h +6 -2

index fa24f5cdf3a..eccfbf1108d 100644
@@ -149,7 +149,6 @@ enum mlx5_rxq_type {
 struct mlx5_rxq_ctrl {
 	struct mlx5_rxq_data rxq; /* Data path structure. */
 	LIST_ENTRY(mlx5_rxq_ctrl) next; /* Pointer to the next element. */
-	uint32_t refcnt; /* Reference counter. */
 	LIST_HEAD(priv, mlx5_rxq_priv) owners; /* Owner rxq list. */
 	struct mlx5_rxq_obj *obj; /* Verbs/DevX elements. */
 	struct mlx5_dev_ctx_shared *sh; /* Shared context. */

@@ -170,6 +169,7 @@ struct mlx5_rxq_ctrl {
 /* RX queue private data. */
 struct mlx5_rxq_priv {
 	uint16_t idx; /* Queue index. */
+	uint32_t refcnt; /* Reference counter. */
 	struct mlx5_rxq_ctrl *ctrl; /* Shared Rx Queue. */
 	LIST_ENTRY(mlx5_rxq_priv) owner_entry; /* Entry in shared rxq_ctrl. */
 	struct mlx5_priv *priv; /* Back pointer to private data. */

@@ -207,7 +207,11 @@ struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev,
 struct mlx5_rxq_ctrl *mlx5_rxq_hairpin_new
 	(struct rte_eth_dev *dev, struct mlx5_rxq_priv *rxq, uint16_t desc,
 	 const struct rte_eth_hairpin_conf *hairpin_conf);
-struct mlx5_rxq_ctrl *mlx5_rxq_get(struct rte_eth_dev *dev, uint16_t idx);
+struct mlx5_rxq_priv *mlx5_rxq_ref(struct rte_eth_dev *dev, uint16_t idx);
+uint32_t mlx5_rxq_deref(struct rte_eth_dev *dev, uint16_t idx);
+struct mlx5_rxq_priv *mlx5_rxq_get(struct rte_eth_dev *dev, uint16_t idx);
+struct mlx5_rxq_ctrl *mlx5_rxq_ctrl_get(struct rte_eth_dev *dev, uint16_t idx);
+struct mlx5_rxq_data *mlx5_rxq_data_get(struct rte_eth_dev *dev, uint16_t idx);
 int mlx5_rxq_release(struct rte_eth_dev *dev, uint16_t idx);
 int mlx5_rxq_verify(struct rte_eth_dev *dev);
 int rxq_alloc_elts(struct mlx5_rxq_ctrl *rxq_ctrl);

drivers/net/mlx5/mlx5_rxq.c +113 ~~-56~~

307 lines not on this page · raw message

drivers/net/mlx5/mlx5_trigger.c +23 ~~-34~~

227 lines not on this page · raw message

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 08:00 UTC #240 lore raw

PATCH v3 9/14 net/mlx5: move Rx queue hairpin info to private data

In reply to Xueming Li

Hairpin info of Rx queue can't be shared, moves to private queue data.

Signed-off-by Xueming Li <[email protected]>

---
 drivers/net/mlx5/mlx5_rx.h      |  4 ++--
 drivers/net/mlx5/mlx5_rxq.c     | 13 +++++--------
 drivers/net/mlx5/mlx5_trigger.c | 24 ++++++++++++------------
 3 files changed, 19 insertions(+), 22 deletions(-)

drivers/net/mlx5/mlx5_rx.h +2 -2

index eccfbf1108d..b21918223b8 100644
@@ -162,8 +162,6 @@ struct mlx5_rxq_ctrl {
 	uint32_t flow_tunnels_n[MLX5_FLOW_TUNNEL]; /* Tunnels counters. */
 	uint32_t wqn; /* WQ number. */
 	uint16_t dump_file_n; /* Number of dump files. */
-	struct rte_eth_hairpin_conf hairpin_conf; /* Hairpin configuration. */
-	uint32_t hairpin_status; /* Hairpin binding status. */
 };
 
 /* RX queue private data. */

@@ -173,6 +171,8 @@ struct mlx5_rxq_priv {
 	struct mlx5_rxq_ctrl *ctrl; /* Shared Rx Queue. */
 	LIST_ENTRY(mlx5_rxq_priv) owner_entry; /* Entry in shared rxq_ctrl. */
 	struct mlx5_priv *priv; /* Back pointer to private data. */
+	struct rte_eth_hairpin_conf hairpin_conf; /* Hairpin configuration. */
+	uint32_t hairpin_status; /* Hairpin binding status. */
 };
 
 /* mlx5_rxq.c */

drivers/net/mlx5/mlx5_rxq.c +5 -8

index 8071ddbd61c..7b637fda643 100644
@@ -1695,8 +1695,8 @@ mlx5_rxq_hairpin_new(struct rte_eth_dev *dev, struct mlx5_rxq_priv *rxq,
 	tmpl->rxq.elts_n = log2above(desc);
 	tmpl->rxq.elts = NULL;
 	tmpl->rxq.mr_ctrl.cache_bh = (struct mlx5_mr_btree) { 0 };
-	tmpl->hairpin_conf = *hairpin_conf;
 	tmpl->rxq.idx = idx;
+	rxq->hairpin_conf = *hairpin_conf;
 	mlx5_rxq_ref(dev, idx);
 	LIST_INSERT_HEAD(&priv->rxqsctrl, tmpl, next);
 	return tmpl;

@@ -1913,14 +1913,11 @@ const struct rte_eth_hairpin_conf *
 mlx5_rxq_get_hairpin_conf(struct rte_eth_dev *dev, uint16_t idx)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_ctrl *rxq_ctrl = NULL;
+	struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, idx);
 
-	if (idx < priv->rxqs_n && (*priv->rxqs)[idx]) {
-		rxq_ctrl = container_of((*priv->rxqs)[idx],
-					struct mlx5_rxq_ctrl,
-					rxq);
-		if (rxq_ctrl->type == MLX5_RXQ_TYPE_HAIRPIN)
-			return &rxq_ctrl->hairpin_conf;
+	if (idx < priv->rxqs_n && rxq != NULL) {
+		if (rxq->ctrl->type == MLX5_RXQ_TYPE_HAIRPIN)
+			return &rxq->hairpin_conf;
 	}
 	return NULL;
 }

drivers/net/mlx5/mlx5_trigger.c +12 ~~-12~~

index e5d74d275f8..a124f74fcda 100644
@@ -324,7 +324,7 @@ mlx5_hairpin_auto_bind(struct rte_eth_dev *dev)
 		}
 		rxq_ctrl = rxq->ctrl;
 		if (rxq_ctrl->type != MLX5_RXQ_TYPE_HAIRPIN ||
-		    rxq_ctrl->hairpin_conf.peers[0].queue != i) {
+		    rxq->hairpin_conf.peers[0].queue != i) {
 			rte_errno = ENOMEM;
 			DRV_LOG(ERR, "port %u Tx queue %d can't be binded to "
 				"Rx queue %d", dev->data->port_id,

@@ -354,7 +354,7 @@ mlx5_hairpin_auto_bind(struct rte_eth_dev *dev)
 		if (ret)
 			goto error;
 		/* Qs with auto-bind will be destroyed directly. */
-		rxq_ctrl->hairpin_status = 1;
+		rxq->hairpin_status = 1;
 		txq_ctrl->hairpin_status = 1;
 		mlx5_txq_release(dev, i);
 	}

@@ -457,9 +457,9 @@ mlx5_hairpin_queue_peer_update(struct rte_eth_dev *dev, uint16_t peer_queue,
 		}
 		peer_info->qp_id = rxq_ctrl->obj->rq->id;
 		peer_info->vhca_id = priv->config.hca_attr.vhca_id;
-		peer_info->peer_q = rxq_ctrl->hairpin_conf.peers[0].queue;
-		peer_info->tx_explicit = rxq_ctrl->hairpin_conf.tx_explicit;
-		peer_info->manual_bind = rxq_ctrl->hairpin_conf.manual_bind;
+		peer_info->peer_q = rxq->hairpin_conf.peers[0].queue;
+		peer_info->tx_explicit = rxq->hairpin_conf.tx_explicit;
+		peer_info->manual_bind = rxq->hairpin_conf.manual_bind;
 	}
 	return 0;
 }

@@ -581,20 +581,20 @@ mlx5_hairpin_queue_peer_bind(struct rte_eth_dev *dev, uint16_t cur_queue,
 				dev->data->port_id, cur_queue);
 			return -rte_errno;
 		}
-		if (rxq_ctrl->hairpin_status != 0) {
+		if (rxq->hairpin_status != 0) {
 			DRV_LOG(DEBUG, "port %u Rx queue %d is already bound",
 				dev->data->port_id, cur_queue);
 			return 0;
 		}
 		if (peer_info->tx_explicit !=
-		    rxq_ctrl->hairpin_conf.tx_explicit) {
+		    rxq->hairpin_conf.tx_explicit) {
 			rte_errno = EINVAL;
 			DRV_LOG(ERR, "port %u Rx queue %d and peer Tx rule mode"
 				" mismatch", dev->data->port_id, cur_queue);
 			return -rte_errno;
 		}
 		if (peer_info->manual_bind !=
-		    rxq_ctrl->hairpin_conf.manual_bind) {
+		    rxq->hairpin_conf.manual_bind) {
 			rte_errno = EINVAL;
 			DRV_LOG(ERR, "port %u Rx queue %d and peer binding mode"
 				" mismatch", dev->data->port_id, cur_queue);

@@ -606,7 +606,7 @@ mlx5_hairpin_queue_peer_bind(struct rte_eth_dev *dev, uint16_t cur_queue,
 		rq_attr.hairpin_peer_vhca = peer_info->vhca_id;
 		ret = mlx5_devx_cmd_modify_rq(rxq_ctrl->obj->rq, &rq_attr);
 		if (ret == 0)
-			rxq_ctrl->hairpin_status = 1;
+			rxq->hairpin_status = 1;
 	}
 	return ret;
 }

@@ -688,7 +688,7 @@ mlx5_hairpin_queue_peer_unbind(struct rte_eth_dev *dev, uint16_t cur_queue,
 				dev->data->port_id, cur_queue);
 			return -rte_errno;
 		}
-		if (rxq_ctrl->hairpin_status == 0) {
+		if (rxq->hairpin_status == 0) {
 			DRV_LOG(DEBUG, "port %u Rx queue %d is already unbound",
 				dev->data->port_id, cur_queue);
 			return 0;

@@ -703,7 +703,7 @@ mlx5_hairpin_queue_peer_unbind(struct rte_eth_dev *dev, uint16_t cur_queue,
 		rq_attr.rq_state = MLX5_SQC_STATE_RST;
 		ret = mlx5_devx_cmd_modify_rq(rxq_ctrl->obj->rq, &rq_attr);
 		if (ret == 0)
-			rxq_ctrl->hairpin_status = 0;
+			rxq->hairpin_status = 0;
 	}
 	return ret;
 }

@@ -1041,7 +1041,7 @@ mlx5_hairpin_get_peer_ports(struct rte_eth_dev *dev, uint16_t *peer_ports,
 			rxq_ctrl = rxq->ctrl;
 			if (rxq_ctrl->type != MLX5_RXQ_TYPE_HAIRPIN)
 				continue;
-			pp = rxq_ctrl->hairpin_conf.peers[0].port;
+			pp = rxq->hairpin_conf.peers[0].port;
 			if (pp >= RTE_MAX_ETHPORTS) {
 				rte_errno = ERANGE;
 				DRV_LOG(ERR, "port %hu queue %u peer port "

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 08:01 UTC #241 lore raw

PATCH v3 10/14 net/mlx5: remove port info from shareable Rx queue

In reply to Xueming Li

To prepare for shared Rx queue, removes port info from shareable Rx queue control.

Signed-off-by Xueming Li <[email protected]>

---
 drivers/net/mlx5/mlx5_devx.c     |  2 +-
 drivers/net/mlx5/mlx5_rx.c       | 15 +++--------
 drivers/net/mlx5/mlx5_rx.h       |  7 ++++--
 drivers/net/mlx5/mlx5_rxq.c      | 43 ++++++++++++++++++++++----------
 drivers/net/mlx5/mlx5_rxtx_vec.c |  2 +-
 drivers/net/mlx5/mlx5_trigger.c  | 13 +++++-----
 6 files changed, 47 insertions(+), 35 deletions(-)

drivers/net/mlx5/mlx5_devx.c +1 -1

index 443252df05d..8b3651f5034 100644
@@ -918,7 +918,7 @@ mlx5_rxq_devx_obj_drop_create(struct rte_eth_dev *dev)
 	}
 	rxq->rxq_ctrl = rxq_ctrl;
 	rxq_ctrl->type = MLX5_RXQ_TYPE_STANDARD;
-	rxq_ctrl->priv = priv;
+	rxq_ctrl->sh = priv->sh;
 	rxq_ctrl->obj = rxq;
 	rxq_data = &rxq_ctrl->rxq;
 	/* Create CQ using DevX API. */

drivers/net/mlx5/mlx5_rx.c +3 ~~-12~~

index 258a6453144..d41905a2a04 100644
@@ -118,15 +118,7 @@ int
 mlx5_rx_descriptor_status(void *rx_queue, uint16_t offset)
 {
 	struct mlx5_rxq_data *rxq = rx_queue;
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-			container_of(rxq, struct mlx5_rxq_ctrl, rxq);
-	struct rte_eth_dev *dev = ETH_DEV(rxq_ctrl->priv);
 
-	if (dev->rx_pkt_burst == NULL ||
-	    dev->rx_pkt_burst == removed_rx_burst) {
-		rte_errno = ENOTSUP;
-		return -rte_errno;
-	}
 	if (offset >= (1 << rxq->cqe_n)) {
 		rte_errno = EINVAL;
 		return -rte_errno;

@@ -438,10 +430,10 @@ mlx5_rx_err_handle(struct mlx5_rxq_data *rxq, uint8_t vec)
 		sm.is_wq = 1;
 		sm.queue_id = rxq->idx;
 		sm.state = IBV_WQS_RESET;
-		if (mlx5_queue_state_modify(ETH_DEV(rxq_ctrl->priv), &sm))
+		if (mlx5_queue_state_modify(RXQ_DEV(rxq_ctrl), &sm))
 			return -1;
 		if (rxq_ctrl->dump_file_n <
-		    rxq_ctrl->priv->config.max_dump_files_num) {
+		    RXQ_PORT(rxq_ctrl)->config.max_dump_files_num) {
 			MKSTR(err_str, "Unexpected CQE error syndrome "
 			      "0x%02x CQN = %u RQN = %u wqe_counter = %u"
 			      " rq_ci = %u cq_ci = %u", u.err_cqe->syndrome,

@@ -478,8 +470,7 @@ mlx5_rx_err_handle(struct mlx5_rxq_data *rxq, uint8_t vec)
 			sm.is_wq = 1;
 			sm.queue_id = rxq->idx;
 			sm.state = IBV_WQS_RDY;
-			if (mlx5_queue_state_modify(ETH_DEV(rxq_ctrl->priv),
-						    &sm))
+			if (mlx5_queue_state_modify(RXQ_DEV(rxq_ctrl), &sm))
 				return -1;
 			if (vec) {
 				const uint32_t elts_n =

drivers/net/mlx5/mlx5_rx.h +5 -2

index b21918223b8..c04c0c73349 100644
@@ -22,6 +22,10 @@
 /* Support tunnel matching. */
 #define MLX5_FLOW_TUNNEL 10
 
+#define RXQ_PORT(rxq_ctrl) LIST_FIRST(&(rxq_ctrl)->owners)->priv
+#define RXQ_DEV(rxq_ctrl) ETH_DEV(RXQ_PORT(rxq_ctrl))
+#define RXQ_PORT_ID(rxq_ctrl) PORT_ID(RXQ_PORT(rxq_ctrl))
+
 /* First entry must be NULL for comparison. */
 #define mlx5_mr_btree_len(bt) ((bt)->len - 1)

@@ -152,7 +156,6 @@ struct mlx5_rxq_ctrl {
 	LIST_HEAD(priv, mlx5_rxq_priv) owners; /* Owner rxq list. */
 	struct mlx5_rxq_obj *obj; /* Verbs/DevX elements. */
 	struct mlx5_dev_ctx_shared *sh; /* Shared context. */
-	struct mlx5_priv *priv; /* Back pointer to private data. */
 	enum mlx5_rxq_type type; /* Rxq type. */
 	unsigned int socket; /* CPU socket ID for allocations. */
 	uint32_t share_group; /* Group ID of shared RXQ. */

@@ -318,7 +321,7 @@ mlx5_rx_addr2mr(struct mlx5_rxq_data *rxq, uintptr_t addr)
 	 */
 	rxq_ctrl = container_of(rxq, struct mlx5_rxq_ctrl, rxq);
 	mp = mlx5_rxq_mprq_enabled(rxq) ? rxq->mprq_mp : rxq->mp;
-	return mlx5_mr_mempool2mr_bh(&rxq_ctrl->priv->sh->cdev->mr_scache,
+	return mlx5_mr_mempool2mr_bh(&rxq_ctrl->sh->cdev->mr_scache,
 				     mr_ctrl, mp, addr);
 }

drivers/net/mlx5/mlx5_rxq.c +30 ~~-13~~

79 lines not on this page · raw message

drivers/net/mlx5/mlx5_rxtx_vec.c +1 -1

index ecd273e00a8..511681841ca 100644
@@ -550,7 +550,7 @@ mlx5_rxq_check_vec_support(struct mlx5_rxq_data *rxq)
 	struct mlx5_rxq_ctrl *ctrl =
 		container_of(rxq, struct mlx5_rxq_ctrl, rxq);
 
-	if (!ctrl->priv->config.rx_vec_en || rxq->sges_n != 0)
+	if (!RXQ_PORT(ctrl)->config.rx_vec_en || rxq->sges_n != 0)
 		return -ENOTSUP;
 	if (rxq->lro)
 		return -ENOTSUP;

drivers/net/mlx5/mlx5_trigger.c +7 -6

index a124f74fcda..caafdf27e8f 100644
@@ -131,9 +131,11 @@ mlx5_rxq_mempool_register_cb(struct rte_mempool *mp, void *opaque,
  *   0 on success, (-1) on failure and rte_errno is set.
  */
 static int
-mlx5_rxq_mempool_register(struct mlx5_rxq_ctrl *rxq_ctrl)
+mlx5_rxq_mempool_register(struct rte_eth_dev *dev,
+			  struct mlx5_rxq_ctrl *rxq_ctrl)
 {
-	struct mlx5_priv *priv = rxq_ctrl->priv;
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_dev_ctx_shared *sh = rxq_ctrl->sh;
 	struct rte_mempool *mp;
 	uint32_t s;
 	int ret = 0;

@@ -148,9 +150,8 @@ mlx5_rxq_mempool_register(struct mlx5_rxq_ctrl *rxq_ctrl)
 	}
 	for (s = 0; s < rxq_ctrl->rxq.rxseg_n; s++) {
 		mp = rxq_ctrl->rxq.rxseg[s].mp;
-		ret = mlx5_mr_mempool_register(&priv->sh->cdev->mr_scache,
-					       priv->sh->cdev->pd, mp,
-					       &priv->mp_id);
+		ret = mlx5_mr_mempool_register(&sh->cdev->mr_scache,
+					       sh->cdev->pd, mp, &priv->mp_id);
 		if (ret < 0 && rte_errno != EEXIST)
 			return ret;
 		rte_mempool_mem_iter(mp, mlx5_rxq_mempool_register_cb,

@@ -213,7 +214,7 @@ mlx5_rxq_start(struct rte_eth_dev *dev)
 			 * the implicit registration is enabled or not,
 			 * Rx mempool destruction is tracked to free MRs.
 			 */
-			if (mlx5_rxq_mempool_register(rxq_ctrl) < 0)
+			if (mlx5_rxq_mempool_register(dev, rxq_ctrl) < 0)
 				goto error;
 			ret = rxq_alloc_elts(rxq_ctrl);
 			if (ret)

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 08:01 UTC #242 lore raw

PATCH v3 11/14 net/mlx5: move Rx queue DevX resource

In reply to Xueming Li

To support shared RX queue, moves DevX RQ which is per queue resource to Rx queue private data.

Signed-off-by Xueming Li <[email protected]>

---
 drivers/net/mlx5/linux/mlx5_verbs.c | 154 +++++++++++--------
 drivers/net/mlx5/mlx5.h             |  11 +-
 drivers/net/mlx5/mlx5_devx.c        | 227 +++++++++++++---------------
 drivers/net/mlx5/mlx5_rx.h          |   1 +
 drivers/net/mlx5/mlx5_rxq.c         |  44 +++---
 drivers/net/mlx5/mlx5_rxtx.c        |   6 +-
 drivers/net/mlx5/mlx5_trigger.c     |   2 +-
 drivers/net/mlx5/mlx5_vlan.c        |  16 +-
 8 files changed, 240 insertions(+), 221 deletions(-)

drivers/net/mlx5/linux/mlx5_verbs.c +90 ~~-64~~

index 4779b37aa65..5d4ae3ea752 100644
@@ -29,13 +29,13 @@
 /**
  * Modify Rx WQ vlan stripping offload
  *
- * @param rxq_obj
- *   Rx queue object.
+ * @param rxq
+ *   Rx queue.
  *
  * @return 0 on success, non-0 otherwise
  */
 static int
-mlx5_rxq_obj_modify_wq_vlan_strip(struct mlx5_rxq_obj *rxq_obj, int on)
+mlx5_rxq_obj_modify_wq_vlan_strip(struct mlx5_rxq_priv *rxq, int on)
 {
 	uint16_t vlan_offloads =
 		(on ? IBV_WQ_FLAGS_CVLAN_STRIPPING : 0) |

@@ -47,14 +47,14 @@ mlx5_rxq_obj_modify_wq_vlan_strip(struct mlx5_rxq_obj *rxq_obj, int on)
 		.flags = vlan_offloads,
 	};
 
-	return mlx5_glue->modify_wq(rxq_obj->wq, &mod);
+	return mlx5_glue->modify_wq(rxq->ctrl->obj->wq, &mod);
 }
 
 /**
  * Modifies the attributes for the specified WQ.
  *
- * @param rxq_obj
- *   Verbs Rx queue object.
+ * @param rxq
+ *   Verbs Rx queue.
  * @param type
  *   Type of change queue state.
  *

@@ -62,14 +62,14 @@ mlx5_rxq_obj_modify_wq_vlan_strip(struct mlx5_rxq_obj *rxq_obj, int on)
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-mlx5_ibv_modify_wq(struct mlx5_rxq_obj *rxq_obj, uint8_t type)
+mlx5_ibv_modify_wq(struct mlx5_rxq_priv *rxq, uint8_t type)
 {
 	struct ibv_wq_attr mod = {
 		.attr_mask = IBV_WQ_ATTR_STATE,
 		.wq_state = (enum ibv_wq_state)type,
 	};
 
-	return mlx5_glue->modify_wq(rxq_obj->wq, &mod);
+	return mlx5_glue->modify_wq(rxq->ctrl->obj->wq, &mod);
 }
 
 /**

@@ -139,21 +139,18 @@ mlx5_ibv_modify_qp(struct mlx5_txq_obj *obj, enum mlx5_txq_modify_type type,
 /**
  * Create a CQ Verbs object.
  *
- * @param dev
- *   Pointer to Ethernet device.
- * @param idx
- *   Queue index in DPDK Rx queue array.
+ * @param rxq
+ *   Pointer to Rx queue.
  *
  * @return
  *   The Verbs CQ object initialized, NULL otherwise and rte_errno is set.
  */
 static struct ibv_cq *
-mlx5_rxq_ibv_cq_create(struct rte_eth_dev *dev, uint16_t idx)
+mlx5_rxq_ibv_cq_create(struct mlx5_rxq_priv *rxq)
 {
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[idx];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_priv *priv = rxq->priv;
+	struct mlx5_rxq_ctrl *rxq_ctrl = rxq->ctrl;
+	struct mlx5_rxq_data *rxq_data = &rxq_ctrl->rxq;
 	struct mlx5_rxq_obj *rxq_obj = rxq_ctrl->obj;
 	unsigned int cqe_n = mlx5_rxq_cqe_num(rxq_data);
 	struct {

@@ -199,7 +196,7 @@ mlx5_rxq_ibv_cq_create(struct rte_eth_dev *dev, uint16_t idx)
 		DRV_LOG(DEBUG,
 			"Port %u Rx CQE compression is disabled for HW"
 			" timestamp.",
-			dev->data->port_id);
+			priv->dev_data->port_id);
 	}
 #ifdef HAVE_IBV_MLX5_MOD_CQE_128B_PAD
 	if (RTE_CACHE_LINE_SIZE == 128) {

@@ -216,21 +213,18 @@ mlx5_rxq_ibv_cq_create(struct rte_eth_dev *dev, uint16_t idx)
 /**
  * Create a WQ Verbs object.
  *
- * @param dev
- *   Pointer to Ethernet device.
- * @param idx
- *   Queue index in DPDK Rx queue array.
+ * @param rxq
+ *   Pointer to Rx queue.
  *
  * @return
  *   The Verbs WQ object initialized, NULL otherwise and rte_errno is set.
  */
 static struct ibv_wq *
-mlx5_rxq_ibv_wq_create(struct rte_eth_dev *dev, uint16_t idx)
+mlx5_rxq_ibv_wq_create(struct mlx5_rxq_priv *rxq)
 {
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[idx];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_priv *priv = rxq->priv;
+	struct mlx5_rxq_ctrl *rxq_ctrl = rxq->ctrl;
+	struct mlx5_rxq_data *rxq_data = &rxq_ctrl->rxq;
 	struct mlx5_rxq_obj *rxq_obj = rxq_ctrl->obj;
 	unsigned int wqe_n = 1 << rxq_data->elts_n;
 	struct {

@@ -297,7 +291,7 @@ mlx5_rxq_ibv_wq_create(struct rte_eth_dev *dev, uint16_t idx)
 			DRV_LOG(ERR,
 				"Port %u Rx queue %u requested %u*%u but got"
 				" %u*%u WRs*SGEs.",
-				dev->data->port_id, idx,
+				priv->dev_data->port_id, rxq->idx,
 				wqe_n >> rxq_data->sges_n,
 				(1 << rxq_data->sges_n),
 				wq_attr.ibv.max_wr, wq_attr.ibv.max_sge);

@@ -312,21 +306,20 @@ mlx5_rxq_ibv_wq_create(struct rte_eth_dev *dev, uint16_t idx)
 /**
  * Create the Rx queue Verbs object.
  *
- * @param dev
- *   Pointer to Ethernet device.
- * @param idx
- *   Queue index in DPDK Rx queue array.
+ * @param rxq
+ *   Pointer to Rx queue.
  *
  * @return
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
+mlx5_rxq_ibv_obj_new(struct mlx5_rxq_priv *rxq)
 {
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[idx];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
+	uint16_t idx = rxq->idx;
+	struct mlx5_priv *priv = rxq->priv;
+	uint16_t port_id = priv->dev_data->port_id;
+	struct mlx5_rxq_ctrl *rxq_ctrl = rxq->ctrl;
+	struct mlx5_rxq_data *rxq_data = &rxq_ctrl->rxq;
 	struct mlx5_rxq_obj *tmpl = rxq_ctrl->obj;
 	struct mlx5dv_cq cq_info;
 	struct mlx5dv_rwq rwq;

@@ -341,17 +334,17 @@ mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 			mlx5_glue->create_comp_channel(priv->sh->cdev->ctx);
 		if (!tmpl->ibv_channel) {
 			DRV_LOG(ERR, "Port %u: comp channel creation failure.",
-				dev->data->port_id);
+				port_id);
 			rte_errno = ENOMEM;
 			goto error;
 		}
 		tmpl->fd = ((struct ibv_comp_channel *)(tmpl->ibv_channel))->fd;
 	}
 	/* Create CQ using Verbs API. */
-	tmpl->ibv_cq = mlx5_rxq_ibv_cq_create(dev, idx);
+	tmpl->ibv_cq = mlx5_rxq_ibv_cq_create(rxq);
 	if (!tmpl->ibv_cq) {
 		DRV_LOG(ERR, "Port %u Rx queue %u CQ creation failure.",
-			dev->data->port_id, idx);
+			port_id, idx);
 		rte_errno = ENOMEM;
 		goto error;
 	}

@@ -366,7 +359,7 @@ mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 		DRV_LOG(ERR,
 			"Port %u wrong MLX5_CQE_SIZE environment "
 			"variable value: it should be set to %u.",
-			dev->data->port_id, RTE_CACHE_LINE_SIZE);
+			port_id, RTE_CACHE_LINE_SIZE);
 		rte_errno = EINVAL;
 		goto error;
 	}

@@ -377,19 +370,19 @@ mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	rxq_data->cq_uar = cq_info.cq_uar;
 	rxq_data->cqn = cq_info.cqn;
 	/* Create WQ (RQ) using Verbs API. */
-	tmpl->wq = mlx5_rxq_ibv_wq_create(dev, idx);
+	tmpl->wq = mlx5_rxq_ibv_wq_create(rxq);
 	if (!tmpl->wq) {
 		DRV_LOG(ERR, "Port %u Rx queue %u WQ creation failure.",
-			dev->data->port_id, idx);
+			port_id, idx);
 		rte_errno = ENOMEM;
 		goto error;
 	}
 	/* Change queue state to ready. */
-	ret = mlx5_ibv_modify_wq(tmpl, IBV_WQS_RDY);
+	ret = mlx5_ibv_modify_wq(rxq, IBV_WQS_RDY);
 	if (ret) {
 		DRV_LOG(ERR,
 			"Port %u Rx queue %u WQ state to IBV_WQS_RDY failed.",
-			dev->data->port_id, idx);
+			port_id, idx);
 		rte_errno = ret;
 		goto error;
 	}

@@ -405,7 +398,7 @@ mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	rxq_data->cq_arm_sn = 0;
 	mlx5_rxq_initialize(rxq_data);
 	rxq_data->cq_ci = 0;
-	dev->data->rx_queue_state[idx] = RTE_ETH_QUEUE_STATE_STARTED;
+	priv->dev_data->rx_queue_state[idx] = RTE_ETH_QUEUE_STATE_STARTED;
 	rxq_ctrl->wqn = ((struct ibv_wq *)(tmpl->wq))->wq_num;
 	return 0;
 error:

@@ -423,12 +416,14 @@ mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 /**
  * Release an Rx verbs queue object.
  *
- * @param rxq_obj
- *   Verbs Rx queue object.
+ * @param rxq
+ *   Pointer to Rx queue.
  */
 static void
-mlx5_rxq_ibv_obj_release(struct mlx5_rxq_obj *rxq_obj)
+mlx5_rxq_ibv_obj_release(struct mlx5_rxq_priv *rxq)
 {
+	struct mlx5_rxq_obj *rxq_obj = rxq->ctrl->obj;
+
 	MLX5_ASSERT(rxq_obj);
 	MLX5_ASSERT(rxq_obj->wq);
 	MLX5_ASSERT(rxq_obj->ibv_cq);

@@ -652,12 +647,24 @@ static void
 mlx5_rxq_ibv_obj_drop_release(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_obj *rxq = priv->drop_queue.rxq;
+	struct mlx5_rxq_priv *rxq = priv->drop_queue.rxq;
+	struct mlx5_rxq_obj *rxq_obj;
 
-	if (rxq->wq)
-		claim_zero(mlx5_glue->destroy_wq(rxq->wq));
-	if (rxq->ibv_cq)
-		claim_zero(mlx5_glue->destroy_cq(rxq->ibv_cq));
+	if (rxq == NULL)
+		return;
+	if (rxq->ctrl == NULL)
+		goto free_priv;
+	rxq_obj = rxq->ctrl->obj;
+	if (rxq_obj == NULL)
+		goto free_ctrl;
+	if (rxq_obj->wq)
+		claim_zero(mlx5_glue->destroy_wq(rxq_obj->wq));
+	if (rxq_obj->ibv_cq)
+		claim_zero(mlx5_glue->destroy_cq(rxq_obj->ibv_cq));
+	mlx5_free(rxq_obj);
+free_ctrl:
+	mlx5_free(rxq->ctrl);
+free_priv:
 	mlx5_free(rxq);
 	priv->drop_queue.rxq = NULL;
 }

@@ -676,39 +683,58 @@ mlx5_rxq_ibv_obj_drop_create(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 	struct ibv_context *ctx = priv->sh->cdev->ctx;
-	struct mlx5_rxq_obj *rxq = priv->drop_queue.rxq;
+	struct mlx5_rxq_priv *rxq = priv->drop_queue.rxq;
+	struct mlx5_rxq_ctrl *rxq_ctrl = NULL;
+	struct mlx5_rxq_obj *rxq_obj = NULL;
 
-	if (rxq)
+	if (rxq != NULL)
 		return 0;
 	rxq = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*rxq), 0, SOCKET_ID_ANY);
-	if (!rxq) {
+	if (rxq == NULL) {
 		DRV_LOG(DEBUG, "Port %u cannot allocate drop Rx queue memory.",
 		      dev->data->port_id);
 		rte_errno = ENOMEM;
 		return -rte_errno;
 	}
 	priv->drop_queue.rxq = rxq;
-	rxq->ibv_cq = mlx5_glue->create_cq(ctx, 1, NULL, NULL, 0);
-	if (!rxq->ibv_cq) {
+	rxq_ctrl = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*rxq_ctrl), 0,
+			       SOCKET_ID_ANY);
+	if (rxq_ctrl == NULL) {
+		DRV_LOG(DEBUG, "Port %u cannot allocate drop Rx queue control memory.",
+		      dev->data->port_id);
+		rte_errno = ENOMEM;
+		goto error;
+	}
+	rxq->ctrl = rxq_ctrl;
+	rxq_obj = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*rxq_obj), 0,
+			      SOCKET_ID_ANY);
+	if (rxq_obj == NULL) {
+		DRV_LOG(DEBUG, "Port %u cannot allocate drop Rx queue memory.",
+		      dev->data->port_id);
+		rte_errno = ENOMEM;
+		goto error;
+	}
+	rxq_ctrl->obj = rxq_obj;
+	rxq_obj->ibv_cq = mlx5_glue->create_cq(ctx, 1, NULL, NULL, 0);
+	if (!rxq_obj->ibv_cq) {
 		DRV_LOG(DEBUG, "Port %u cannot allocate CQ for drop queue.",
 		      dev->data->port_id);
 		rte_errno = errno;
 		goto error;
 	}
-	rxq->wq = mlx5_glue->create_wq(ctx, &(struct ibv_wq_init_attr){
+	rxq_obj->wq = mlx5_glue->create_wq(ctx, &(struct ibv_wq_init_attr){
 						    .wq_type = IBV_WQT_RQ,
 						    .max_wr = 1,
 						    .max_sge = 1,
 						    .pd = priv->sh->cdev->pd,
-						    .cq = rxq->ibv_cq,
+						    .cq = rxq_obj->ibv_cq,
 					      });
-	if (!rxq->wq) {
+	if (!rxq_obj->wq) {
 		DRV_LOG(DEBUG, "Port %u cannot allocate WQ for drop queue.",
 		      dev->data->port_id);
 		rte_errno = errno;
 		goto error;
 	}
-	priv->drop_queue.rxq = rxq;
 	return 0;
 error:
 	mlx5_rxq_ibv_obj_drop_release(dev);

@@ -737,7 +763,7 @@ mlx5_ibv_drop_action_create(struct rte_eth_dev *dev)
 	ret = mlx5_rxq_ibv_obj_drop_create(dev);
 	if (ret < 0)
 		goto error;
-	rxq = priv->drop_queue.rxq;
+	rxq = priv->drop_queue.rxq->ctrl->obj;
 	ind_tbl = mlx5_glue->create_rwq_ind_table
 				(priv->sh->cdev->ctx,
 				 &(struct ibv_rwq_ind_table_init_attr){

drivers/net/mlx5/mlx5.h +5 -6

index 3e008241ca8..bc1b6b96cda 100644
@@ -294,7 +294,7 @@ struct mlx5_vf_vlan {
 /* Flow drop context necessary due to Verbs API. */
 struct mlx5_drop {
 	struct mlx5_hrxq *hrxq; /* Hash Rx queue queue. */
-	struct mlx5_rxq_obj *rxq; /* Rx queue object. */
+	struct mlx5_rxq_priv *rxq; /* Rx queue. */
 };
 
 /* Loopback dummy queue resources required due to Verbs API. */

@@ -1239,7 +1239,6 @@ struct mlx5_rxq_obj {
 		};
 		struct mlx5_devx_obj *rq; /* DevX RQ object for hairpin. */
 		struct {
-			struct mlx5_devx_rq rq_obj; /* DevX RQ object. */
 			struct mlx5_devx_cq cq_obj; /* DevX CQ object. */
 			void *devx_channel;
 		};

@@ -1321,11 +1320,11 @@ struct mlx5_rxq_priv;
 
 /* HW objects operations structure. */
 struct mlx5_obj_ops {
-	int (*rxq_obj_modify_vlan_strip)(struct mlx5_rxq_obj *rxq_obj, int on);
-	int (*rxq_obj_new)(struct rte_eth_dev *dev, uint16_t idx);
+	int (*rxq_obj_modify_vlan_strip)(struct mlx5_rxq_priv *rxq, int on);
+	int (*rxq_obj_new)(struct mlx5_rxq_priv *rxq);
 	int (*rxq_event_get)(struct mlx5_rxq_obj *rxq_obj);
-	int (*rxq_obj_modify)(struct mlx5_rxq_obj *rxq_obj, uint8_t type);
-	void (*rxq_obj_release)(struct mlx5_rxq_obj *rxq_obj);
+	int (*rxq_obj_modify)(struct mlx5_rxq_priv *rxq, uint8_t type);
+	void (*rxq_obj_release)(struct mlx5_rxq_priv *rxq);
 	int (*ind_table_new)(struct rte_eth_dev *dev, const unsigned int log_n,
 			     struct mlx5_ind_table_obj *ind_tbl);
 	int (*ind_table_modify)(struct rte_eth_dev *dev,

drivers/net/mlx5/mlx5_devx.c +109 ~~-118~~

467 lines not on this page · raw message

drivers/net/mlx5/mlx5_rx.h +1 -0

index c04c0c73349..337dcca59fb 100644
@@ -174,6 +174,7 @@ struct mlx5_rxq_priv {
 	struct mlx5_rxq_ctrl *ctrl; /* Shared Rx Queue. */
 	LIST_ENTRY(mlx5_rxq_priv) owner_entry; /* Entry in shared rxq_ctrl. */
 	struct mlx5_priv *priv; /* Back pointer to private data. */
+	struct mlx5_devx_rq devx_rq;
 	struct rte_eth_hairpin_conf hairpin_conf; /* Hairpin configuration. */
 	uint32_t hairpin_status; /* Hairpin binding status. */
 };

drivers/net/mlx5/mlx5_rxq.c +24 ~~-20~~

93 lines not on this page · raw message

drivers/net/mlx5/mlx5_rxtx.c +2 -4

index 0bcdff1b116..54d410b513b 100644
@@ -373,11 +373,9 @@ mlx5_queue_state_modify_primary(struct rte_eth_dev *dev,
 	struct mlx5_priv *priv = dev->data->dev_private;
 
 	if (sm->is_wq) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[sm->queue_id];
-		struct mlx5_rxq_ctrl *rxq_ctrl =
-			container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+		struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, sm->queue_id);
 
-		ret = priv->obj_ops.rxq_obj_modify(rxq_ctrl->obj, sm->state);
+		ret = priv->obj_ops.rxq_obj_modify(rxq, sm->state);
 		if (ret) {
 			DRV_LOG(ERR, "Cannot change Rx WQ state to %u  - %s",
 					sm->state, strerror(errno));

drivers/net/mlx5/mlx5_trigger.c +1 -1

index caafdf27e8f..2cf62a9780d 100644
@@ -231,7 +231,7 @@ mlx5_rxq_start(struct rte_eth_dev *dev)
 			rte_errno = ENOMEM;
 			goto error;
 		}
-		ret = priv->obj_ops.rxq_obj_new(dev, i);
+		ret = priv->obj_ops.rxq_obj_new(rxq);
 		if (ret) {
 			mlx5_free(rxq_ctrl->obj);
 			rxq_ctrl->obj = NULL;

drivers/net/mlx5/mlx5_vlan.c +8 -8

index 07792fc5d94..ea841bb32fb 100644
@@ -91,11 +91,11 @@ void
 mlx5_vlan_strip_queue_set(struct rte_eth_dev *dev, uint16_t queue, int on)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq = (*priv->rxqs)[queue];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, queue);
+	struct mlx5_rxq_data *rxq_data = &rxq->ctrl->rxq;
 	int ret = 0;
 
+	MLX5_ASSERT(rxq != NULL && rxq->ctrl != NULL);
 	/* Validate hw support */
 	if (!priv->config.hw_vlan_strip) {
 		DRV_LOG(ERR, "port %u VLAN stripping is not supported",

@@ -109,20 +109,20 @@ mlx5_vlan_strip_queue_set(struct rte_eth_dev *dev, uint16_t queue, int on)
 		return;
 	}
 	DRV_LOG(DEBUG, "port %u set VLAN stripping offloads %d for port %uqueue %d",
-		dev->data->port_id, on, rxq->port_id, queue);
-	if (!rxq_ctrl->obj) {
+		dev->data->port_id, on, rxq_data->port_id, queue);
+	if (rxq->ctrl->obj == NULL) {
 		/* Update related bits in RX queue. */
-		rxq->vlan_strip = !!on;
+		rxq_data->vlan_strip = !!on;
 		return;
 	}
-	ret = priv->obj_ops.rxq_obj_modify_vlan_strip(rxq_ctrl->obj, on);
+	ret = priv->obj_ops.rxq_obj_modify_vlan_strip(rxq, on);
 	if (ret) {
 		DRV_LOG(ERR, "Port %u failed to modify object stripping mode:"
 			" %s", dev->data->port_id, strerror(rte_errno));
 		return;
 	}
 	/* Update related bits in RX queue. */
-	rxq->vlan_strip = !!on;
+	rxq_data->vlan_strip = !!on;
 }
 
 /**

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 08:01 UTC #243 lore raw

PATCH v3 12/14 net/mlx5: remove Rx queue data list from device

In reply to Xueming Li

Rx queue data list(priv->rxqs) can be replaced by Rx queue list(priv->rxq_privs), removes it and replaces with universal wrapper API.

Signed-off-by Xueming Li <[email protected]>

---
 drivers/net/mlx5/linux/mlx5_verbs.c |  7 ++---
 drivers/net/mlx5/mlx5.c             | 10 +-----
 drivers/net/mlx5/mlx5.h             |  1 -
 drivers/net/mlx5/mlx5_devx.c        | 12 +++++---
 drivers/net/mlx5/mlx5_ethdev.c      |  6 +---
 drivers/net/mlx5/mlx5_flow.c        | 47 +++++++++++++++--------------
 drivers/net/mlx5/mlx5_rss.c         |  6 ++--
 drivers/net/mlx5/mlx5_rx.c          | 15 +++++----
 drivers/net/mlx5/mlx5_rx.h          |  9 +++---
 drivers/net/mlx5/mlx5_rxq.c         | 43 ++++++++++++--------------
 drivers/net/mlx5/mlx5_rxtx_vec.c    |  6 ++--
 drivers/net/mlx5/mlx5_stats.c       |  9 +++---
 drivers/net/mlx5/mlx5_trigger.c     |  2 +-
 13 files changed, 79 insertions(+), 94 deletions(-)

drivers/net/mlx5/linux/mlx5_verbs.c +3 -4

index 5d4ae3ea752..f78916c868f 100644
@@ -486,11 +486,10 @@ mlx5_ibv_ind_table_new(struct rte_eth_dev *dev, const unsigned int log_n,
 
 	MLX5_ASSERT(ind_tbl);
 	for (i = 0; i != ind_tbl->queues_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[ind_tbl->queues[i]];
-		struct mlx5_rxq_ctrl *rxq_ctrl =
-				container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+		struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev,
+							 ind_tbl->queues[i]);
 
-		wq[i] = rxq_ctrl->obj->wq;
+		wq[i] = rxq->ctrl->obj->wq;
 	}
 	MLX5_ASSERT(i > 0);
 	/* Finalise indirection table. */

drivers/net/mlx5/mlx5.c +1 -9

index d0fae518025..b81c959d642 100644
@@ -1686,20 +1686,12 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	mlx5_mp_os_req_stop_rxtx(dev);
 	/* Free the eCPRI flex parser resource. */
 	mlx5_flex_parser_ecpri_release(dev);
-	if (priv->rxqs != NULL) {
+	if (priv->rxq_privs != NULL) {
 		/* XXX race condition if mlx5_rx_burst() is still running. */
 		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->rxqs_n); ++i)
 			mlx5_rxq_release(dev, i);
 		priv->rxqs_n = 0;
-		priv->rxqs = NULL;
-	}
-	if (priv->representor) {
-		/* Each representor has a dedicated interrupts handler */
-		mlx5_free(dev->intr_handle);
-		dev->intr_handle = NULL;
-	}
-	if (priv->rxq_privs != NULL) {
 		mlx5_free(priv->rxq_privs);
 		priv->rxq_privs = NULL;
 	}

drivers/net/mlx5/mlx5.h +0 -1

index bc1b6b96cda..75c58b93f91 100644
@@ -1382,7 +1382,6 @@ struct mlx5_priv {
 	unsigned int rxqs_n; /* RX queues array size. */
 	unsigned int txqs_n; /* TX queues array size. */
 	struct mlx5_rxq_priv *(*rxq_privs)[]; /* RX queue non-shared data. */
-	struct mlx5_rxq_data *(*rxqs)[]; /* (Shared) RX queues. */
 	struct mlx5_txq_data *(*txqs)[]; /* TX queues. */
 	struct rte_mempool *mprq_mp; /* Mempool for Multi-Packet RQ. */
 	struct rte_eth_rss_conf rss_conf; /* RSS configuration. */

drivers/net/mlx5/mlx5_devx.c +7 -5

index b90a5d82458..668d47025e8 100644
@@ -684,15 +684,17 @@ mlx5_devx_tir_attr_set(struct rte_eth_dev *dev, const uint8_t *rss_key,
 
 	/* NULL queues designate drop queue. */
 	if (ind_tbl->queues != NULL) {
-		struct mlx5_rxq_data *rxq_data =
-					(*priv->rxqs)[ind_tbl->queues[0]];
 		struct mlx5_rxq_ctrl *rxq_ctrl =
-			container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
-		rxq_obj_type = rxq_ctrl->type;
+				mlx5_rxq_ctrl_get(dev, ind_tbl->queues[0]);
+		rxq_obj_type = rxq_ctrl != NULL ? rxq_ctrl->type :
+						  MLX5_RXQ_TYPE_STANDARD;
 
 		/* Enable TIR LRO only if all the queues were configured for. */
 		for (i = 0; i < ind_tbl->queues_n; ++i) {
-			if (!(*priv->rxqs)[ind_tbl->queues[i]]->lro) {
+			struct mlx5_rxq_data *rxq_i =
+				mlx5_rxq_data_get(dev, ind_tbl->queues[i]);
+
+			if (rxq_i != NULL && !rxq_i->lro) {
 				lro = false;
 				break;
 			}

drivers/net/mlx5/mlx5_ethdev.c +1 -5

index cde505955df..bb38d5d2ade 100644
@@ -114,7 +114,6 @@ mlx5_dev_configure(struct rte_eth_dev *dev)
 		rte_errno = ENOMEM;
 		return -rte_errno;
 	}
-	priv->rxqs = (void *)dev->data->rx_queues;
 	priv->txqs = (void *)dev->data->tx_queues;
 	if (txqs_n != priv->txqs_n) {
 		DRV_LOG(INFO, "port %u Tx queues number update: %u -> %u",

@@ -171,11 +170,8 @@ mlx5_dev_configure_rss_reta(struct rte_eth_dev *dev)
 		return -rte_errno;
 	}
 	for (i = 0, j = 0; i < rxqs_n; i++) {
-		struct mlx5_rxq_data *rxq_data;
-		struct mlx5_rxq_ctrl *rxq_ctrl;
+		struct mlx5_rxq_ctrl *rxq_ctrl = mlx5_rxq_ctrl_get(dev, i);
 
-		rxq_data = (*priv->rxqs)[i];
-		rxq_ctrl = container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
 		if (rxq_ctrl && rxq_ctrl->type == MLX5_RXQ_TYPE_STANDARD)
 			rss_queue_arr[j++] = i;
 	}

drivers/net/mlx5/mlx5_flow.c +24 ~~-23~~

109 lines not on this page · raw message

drivers/net/mlx5/mlx5_rss.c +4 -2

index a04e22398db..75af05b7b02 100644
@@ -65,9 +65,11 @@ mlx5_rss_hash_update(struct rte_eth_dev *dev,
 	priv->rss_conf.rss_hf = rss_conf->rss_hf;
 	/* Enable the RSS hash in all Rx queues. */
 	for (i = 0, idx = 0; idx != priv->rxqs_n; ++i) {
-		if (!(*priv->rxqs)[i])
+		struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, i);
+
+		if (rxq == NULL || rxq->ctrl == NULL)
 			continue;
-		(*priv->rxqs)[i]->rss_hash = !!rss_conf->rss_hf &&
+		rxq->ctrl->rxq.rss_hash = !!rss_conf->rss_hf &&
 			!!(dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS);
 		++idx;
 	}

drivers/net/mlx5/mlx5_rx.c +7 -8

index d41905a2a04..1ffa1b95b88 100644
@@ -148,10 +148,8 @@ void
 mlx5_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 		  struct rte_eth_rxq_info *qinfo)
 {
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq = (*priv->rxqs)[rx_queue_id];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_rxq_ctrl *rxq_ctrl = mlx5_rxq_ctrl_get(dev, rx_queue_id);
+	struct mlx5_rxq_data *rxq = mlx5_rxq_data_get(dev, rx_queue_id);
 
 	if (!rxq)
 		return;

@@ -162,7 +160,10 @@ mlx5_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	qinfo->conf.rx_thresh.wthresh = 0;
 	qinfo->conf.rx_free_thresh = rxq->rq_repl_thresh;
 	qinfo->conf.rx_drop_en = 1;
-	qinfo->conf.rx_deferred_start = rxq_ctrl ? 0 : 1;
+	if (rxq_ctrl == NULL || rxq_ctrl->obj == NULL)
+		qinfo->conf.rx_deferred_start = 0;
+	else
+		qinfo->conf.rx_deferred_start = 1;
 	qinfo->conf.offloads = dev->data->dev_conf.rxmode.offloads;
 	qinfo->scattered_rx = dev->data->scattered_rx;
 	qinfo->nb_desc = mlx5_rxq_mprq_enabled(rxq) ?

@@ -191,10 +192,8 @@ mlx5_rx_burst_mode_get(struct rte_eth_dev *dev,
 		       struct rte_eth_burst_mode *mode)
 {
 	eth_rx_burst_t pkt_burst = dev->rx_pkt_burst;
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq;
+	struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, rx_queue_id);
 
-	rxq = (*priv->rxqs)[rx_queue_id];
 	if (!rxq) {
 		rte_errno = EINVAL;
 		return -rte_errno;

drivers/net/mlx5/mlx5_rx.h +4 -5

index 337dcca59fb..413e36f6d8d 100644
@@ -603,14 +603,13 @@ mlx5_mprq_enabled(struct rte_eth_dev *dev)
 		return 0;
 	/* All the configured queues should be enabled. */
 	for (i = 0; i < priv->rxqs_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
-		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
-			(rxq, struct mlx5_rxq_ctrl, rxq);
+		struct mlx5_rxq_ctrl *rxq_ctrl = mlx5_rxq_ctrl_get(dev, i);
 
-		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
+		if (rxq_ctrl == NULL ||
+		    rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
 			continue;
 		n_ibv++;
-		if (mlx5_rxq_mprq_enabled(rxq))
+		if (mlx5_rxq_mprq_enabled(&rxq_ctrl->rxq))
 			++n;
 	}
 	/* Multi-Packet RQ can't be partially configured. */

drivers/net/mlx5/mlx5_rxq.c +19 ~~-24~~

107 lines not on this page · raw message

drivers/net/mlx5/mlx5_rxtx_vec.c +3 -3

index 511681841ca..6212ce8247d 100644
@@ -578,11 +578,11 @@ mlx5_check_vec_rx_support(struct rte_eth_dev *dev)
 		return -ENOTSUP;
 	/* All the configured queues should support. */
 	for (i = 0; i < priv->rxqs_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
+		struct mlx5_rxq_data *rxq_data = mlx5_rxq_data_get(dev, i);
 
-		if (!rxq)
+		if (!rxq_data)
 			continue;
-		if (mlx5_rxq_check_vec_support(rxq) < 0)
+		if (mlx5_rxq_check_vec_support(rxq_data) < 0)
 			break;
 	}
 	if (i != priv->rxqs_n)

drivers/net/mlx5/mlx5_stats.c +5 -4

index ae2f5668a74..732775954ad 100644
@@ -107,7 +107,7 @@ mlx5_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
 	memset(&tmp, 0, sizeof(tmp));
 	/* Add software counters. */
 	for (i = 0; (i != priv->rxqs_n); ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
+		struct mlx5_rxq_data *rxq = mlx5_rxq_data_get(dev, i);
 
 		if (rxq == NULL)
 			continue;

@@ -181,10 +181,11 @@ mlx5_stats_reset(struct rte_eth_dev *dev)
 	unsigned int i;
 
 	for (i = 0; (i != priv->rxqs_n); ++i) {
-		if ((*priv->rxqs)[i] == NULL)
+		struct mlx5_rxq_data *rxq_data = mlx5_rxq_data_get(dev, i);
+
+		if (rxq_data == NULL)
 			continue;
-		memset(&(*priv->rxqs)[i]->stats, 0,
-		       sizeof(struct mlx5_rxq_stats));
+		memset(&rxq_data->stats, 0, sizeof(struct mlx5_rxq_stats));
 	}
 	for (i = 0; (i != priv->txqs_n); ++i) {
 		if ((*priv->txqs)[i] == NULL)

drivers/net/mlx5/mlx5_trigger.c +1 -1

index 2cf62a9780d..72475e4b5b5 100644
@@ -227,7 +227,7 @@ mlx5_rxq_start(struct rte_eth_dev *dev)
 		if (!rxq_ctrl->obj) {
 			DRV_LOG(ERR,
 				"Port %u Rx queue %u can't allocate resources.",
-				dev->data->port_id, (*priv->rxqs)[i]->idx);
+				dev->data->port_id, i);
 			rte_errno = ENOMEM;
 			goto error;
 		}

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 08:01 UTC #244 lore raw

PATCH v3 13/14 net/mlx5: support shared Rx queue

In reply to Xueming Li

This patch introduces shared RxQ. All shared Rx queues with same group and queue ID share the same rxq_ctrl. Rxq_ctrl and rxq_data are shared, all queues from different member port share same WQ and CQ, essentially one Rx WQ, mbufs are filled into this singleton WQ.

Shared rxq_data is set into device Rx queues of all member ports as RxQ object, used for receiving packets. Polling queue of any member ports returns packets of any member, mbuf->port is used to identify source port.

Signed-off-by Xueming Li <[email protected]>

---
 doc/guides/nics/features/mlx5.ini   |   1 +
 doc/guides/nics/mlx5.rst            |   6 +
 drivers/net/mlx5/linux/mlx5_os.c    |   2 +
 drivers/net/mlx5/linux/mlx5_verbs.c |   8 +-
 drivers/net/mlx5/mlx5.h             |   2 +
 drivers/net/mlx5/mlx5_devx.c        |  46 +++--
 drivers/net/mlx5/mlx5_ethdev.c      |   5 +
 drivers/net/mlx5/mlx5_rx.h          |   3 +
 drivers/net/mlx5/mlx5_rxq.c         | 274 ++++++++++++++++++++++++----
 drivers/net/mlx5/mlx5_trigger.c     |  61 ++++---
 10 files changed, 330 insertions(+), 78 deletions(-)

doc/guides/nics/features/mlx5.ini +1 -0

index 403f58cd7e2..7cbd11bb160 100644
@@ -11,6 +11,7 @@ Removal event        = Y
 Rx interrupt         = Y
 Fast mbuf free       = Y
 Queue start/stop     = Y
+Shared Rx queue      = Y
 Burst mode info      = Y
 Power mgmt address monitor = Y
 MTU update           = Y

doc/guides/nics/mlx5.rst +6 -0

index bb92520dff4..824971d89ae 100644
@@ -113,6 +113,7 @@ Features
 - Connection tracking.
 - Sub-Function representors.
 - Sub-Function.
+- Shared Rx queue.
 
 
 Limitations

@@ -465,6 +466,11 @@ Limitations
   - In order to achieve best insertion rate, application should manage the flows per lcore.
   - Better to disable memory reclaim by setting ``reclaim_mem_mode`` to 0 to accelerate the flow object allocation and release with cache.
 
+ Shared Rx queue:
+
+  - Counters of received packets and bytes number of devices in same share group are same.
+  - Counters of received packets and bytes number of queues in same group and queue ID are same.
+
 - HW hashed bonding
 
   - TXQ affinity subjects to HW hash once enabled.

drivers/net/mlx5/linux/mlx5_os.c +2 -0

index dd4fc0c7165..48acae65133 100644
@@ -410,6 +410,7 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv)
 			mlx5_glue->dr_create_flow_action_default_miss();
 	if (!sh->default_miss_action)
 		DRV_LOG(WARNING, "Default miss action is not supported.");
+	LIST_INIT(&sh->shared_rxqs);
 	return 0;
 error:
 	/* Rollback the created objects. */

@@ -484,6 +485,7 @@ mlx5_os_free_shared_dr(struct mlx5_priv *priv)
 	MLX5_ASSERT(sh && sh->refcnt);
 	if (sh->refcnt > 1)
 		return;
+	MLX5_ASSERT(LIST_EMPTY(&sh->shared_rxqs));
 #ifdef HAVE_MLX5DV_DR
 	if (sh->rx_domain) {
 		mlx5_glue->dr_destroy_domain(sh->rx_domain);

drivers/net/mlx5/linux/mlx5_verbs.c +5 -3

index f78916c868f..9d299542614 100644
@@ -424,14 +424,16 @@ mlx5_rxq_ibv_obj_release(struct mlx5_rxq_priv *rxq)
 {
 	struct mlx5_rxq_obj *rxq_obj = rxq->ctrl->obj;
 
-	MLX5_ASSERT(rxq_obj);
-	MLX5_ASSERT(rxq_obj->wq);
-	MLX5_ASSERT(rxq_obj->ibv_cq);
+	if (rxq_obj == NULL || rxq_obj->wq == NULL)
+		return;
 	claim_zero(mlx5_glue->destroy_wq(rxq_obj->wq));
+	rxq_obj->wq = NULL;
+	MLX5_ASSERT(rxq_obj->ibv_cq);
 	claim_zero(mlx5_glue->destroy_cq(rxq_obj->ibv_cq));
 	if (rxq_obj->ibv_channel)
 		claim_zero(mlx5_glue->destroy_comp_channel
 							(rxq_obj->ibv_channel));
+	rxq->ctrl->started = false;
 }
 
 /**

drivers/net/mlx5/mlx5.h +2 -0

index 75c58b93f91..3950f0dabb0 100644
@@ -1172,6 +1172,7 @@ struct mlx5_dev_ctx_shared {
 	struct mlx5_flex_parser_profiles fp[MLX5_FLEX_PARSER_MAX];
 	/* Flex parser profiles information. */
 	void *devx_rx_uar; /* DevX UAR for Rx. */
+	LIST_HEAD(shared_rxqs, mlx5_rxq_ctrl) shared_rxqs; /* Shared RXQs. */
 	struct mlx5_aso_age_mng *aso_age_mng;
 	/* Management data for aging mechanism using ASO Flow Hit. */
 	struct mlx5_geneve_tlv_option_resource *geneve_tlv_option_resource;

@@ -1239,6 +1240,7 @@ struct mlx5_rxq_obj {
 		};
 		struct mlx5_devx_obj *rq; /* DevX RQ object for hairpin. */
 		struct {
+			struct mlx5_devx_rmp devx_rmp; /* RMP for shared RQ. */
 			struct mlx5_devx_cq cq_obj; /* DevX CQ object. */
 			void *devx_channel;
 		};

drivers/net/mlx5/mlx5_devx.c +34 ~~-12~~

107 lines not on this page · raw message

drivers/net/mlx5/mlx5_ethdev.c +5 -0

index bb38d5d2ade..dc647d5580c 100644
@@ -26,6 +26,7 @@
 #include "mlx5_rx.h"
 #include "mlx5_tx.h"
 #include "mlx5_autoconf.h"
+#include "mlx5_devx.h"
 
 /**
  * Get the interface index from device name.

@@ -336,9 +337,13 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info)
 	info->flow_type_rss_offloads = ~MLX5_RSS_HF_MASK;
 	mlx5_set_default_params(dev, info);
 	mlx5_set_txlimit_params(dev, info);
+	if (priv->config.hca_attr.mem_rq_rmp &&
+	    priv->obj_ops.rxq_obj_new == devx_obj_ops.rxq_obj_new)
+		info->dev_capa |= RTE_ETH_DEV_CAPA_RXQ_SHARE;
 	info->switch_info.name = dev->data->name;
 	info->switch_info.domain_id = priv->domain_id;
 	info->switch_info.port_id = priv->representor_id;
+	info->switch_info.rx_domain = 0; /* No sub Rx domains. */
 	if (priv->representor) {
 		uint16_t port_id;

drivers/net/mlx5/mlx5_rx.h +3 -0

index 413e36f6d8d..eda6eca8dea 100644
@@ -96,6 +96,7 @@ struct mlx5_rxq_data {
 	unsigned int lro:1; /* Enable LRO. */
 	unsigned int dynf_meta:1; /* Dynamic metadata is configured. */
 	unsigned int mcqe_format:3; /* CQE compression format. */
+	unsigned int shared:1; /* Shared RXQ. */
 	volatile uint32_t *rq_db;
 	volatile uint32_t *cq_db;
 	uint16_t port_id;

@@ -158,8 +159,10 @@ struct mlx5_rxq_ctrl {
 	struct mlx5_dev_ctx_shared *sh; /* Shared context. */
 	enum mlx5_rxq_type type; /* Rxq type. */
 	unsigned int socket; /* CPU socket ID for allocations. */
+	LIST_ENTRY(mlx5_rxq_ctrl) share_entry; /* Entry in shared RXQ list. */
 	uint32_t share_group; /* Group ID of shared RXQ. */
 	uint16_t share_qid; /* Shared RxQ ID in group. */
+	unsigned int started:1; /* Whether (shared) RXQ has been started. */
 	unsigned int irq:1; /* Whether IRQ is enabled. */
 	uint32_t flow_mark_n; /* Number of Mark/Flag flows using this Queue. */
 	uint32_t flow_tunnels_n[MLX5_FLOW_TUNNEL]; /* Tunnels counters. */

drivers/net/mlx5/mlx5_rxq.c +235 ~~-39~~

374 lines not on this page · raw message

drivers/net/mlx5/mlx5_trigger.c +37 ~~-24~~

80 lines not on this page · raw message

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-03 08:01 UTC #245 lore raw

PATCH v3 14/14 net/mlx5: add shared Rx queue port datapath support

In reply to Xueming Li

From: Viacheslav Ovsiienko <[email protected]>

When receive packet, mlx5 PMD saves mbuf port number from RxQ data.

To support shared RxQ, save port number into RQ context as user index. Received packet resolve port number from CQE user index which derived from RQ context.

Legacy Verbs API doesn't support RQ user index setting, still read from RxQ port number.

Signed-off-by Xueming Li <[email protected]>

Signed-off-by Viacheslav Ovsiienko <[email protected]>

---
 drivers/net/mlx5/mlx5_devx.c             |  1 +
 drivers/net/mlx5/mlx5_rx.c               |  1 +
 drivers/net/mlx5/mlx5_rxq.c              |  3 ++-
 drivers/net/mlx5/mlx5_rxtx_vec_altivec.h |  6 ++++++
 drivers/net/mlx5/mlx5_rxtx_vec_neon.h    | 12 +++++++++++-
 drivers/net/mlx5/mlx5_rxtx_vec_sse.h     |  8 +++++++-
 6 files changed, 28 insertions(+), 3 deletions(-)

drivers/net/mlx5/mlx5_devx.c +1 -0

index d3d189ab7f2..a9f9f4af700 100644
@@ -277,6 +277,7 @@ mlx5_rxq_create_devx_rq_resources(struct mlx5_rxq_priv *rxq)
 						MLX5_WQ_END_PAD_MODE_NONE;
 	rq_attr.wq_attr.pd = cdev->pdn;
 	rq_attr.counter_set_id = priv->counter_set_id;
+	rq_attr.user_index = rte_cpu_to_be_16(priv->dev_data->port_id);
 	if (rxq_data->shared) /* Create RMP based RQ. */
 		rxq->devx_rq.rmp = &rxq_ctrl->obj->devx_rmp;
 	/* Create RQ using DevX API. */

drivers/net/mlx5/mlx5_rx.c +1 -0

index 1ffa1b95b88..4d85f64accd 100644
@@ -709,6 +709,7 @@ rxq_cq_to_mbuf(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt,
 {
 	/* Update packet information. */
 	pkt->packet_type = rxq_cq_to_pkt_type(rxq, cqe, mcqe);
+	pkt->port = unlikely(rxq->shared) ? cqe->user_index_low : rxq->port_id;
 
 	if (rxq->rss_hash) {
 		uint32_t rss_hash_res = 0;

drivers/net/mlx5/mlx5_rxq.c +2 -1

index 0f1f4660bc7..6c715c0803e 100644
@@ -186,7 +186,8 @@ rxq_alloc_elts_sprq(struct mlx5_rxq_ctrl *rxq_ctrl)
 		mbuf_init->data_off = RTE_PKTMBUF_HEADROOM;
 		rte_mbuf_refcnt_set(mbuf_init, 1);
 		mbuf_init->nb_segs = 1;
-		mbuf_init->port = rxq->port_id;
+		/* For shared queues port is provided in CQE */
+		mbuf_init->port = rxq->shared ? 0 : rxq->port_id;
 		if (priv->flags & RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF)
 			mbuf_init->ol_flags = RTE_MBUF_F_EXTERNAL;
 		/*

drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +6 -0

index 1d00c1c43d1..423e229508c 100644
@@ -1189,6 +1189,12 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 
 		/* D.5 fill in mbuf - rearm_data and packet_type. */
 		rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]);
+		if (unlikely(rxq->shared)) {
+			pkts[pos]->port = cq[pos].user_index_low;
+			pkts[pos + p1]->port = cq[pos + p1].user_index_low;
+			pkts[pos + p2]->port = cq[pos + p2].user_index_low;
+			pkts[pos + p3]->port = cq[pos + p3].user_index_low;
+		}
 		if (rxq->hw_timestamp) {
 			int offset = rxq->timestamp_offset;
 			if (rxq->rt_timestamp) {

drivers/net/mlx5/mlx5_rxtx_vec_neon.h +11 -1

index aa36df29a09..b1d16baa619 100644
@@ -787,7 +787,17 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 		/* C.4 fill in mbuf - rearm_data and packet_type. */
 		rxq_cq_to_ptype_oflags_v(rxq, ptype_info, flow_tag,
 					 opcode, &elts[pos]);
-		if (rxq->hw_timestamp) {
+		if (unlikely(rxq->shared)) {
+			elts[pos]->port = container_of(p0, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 1]->port = container_of(p1, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 2]->port = container_of(p2, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 3]->port = container_of(p3, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+		}
+		if (unlikely(rxq->hw_timestamp)) {
 			int offset = rxq->timestamp_offset;
 			if (rxq->rt_timestamp) {
 				struct mlx5_dev_ctx_shared *sh = rxq->sh;

drivers/net/mlx5/mlx5_rxtx_vec_sse.h +7 -1

index b0fc29d7b9e..f3d838389e2 100644
@@ -736,7 +736,13 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 		*err |= _mm_cvtsi128_si64(opcode);
 		/* D.5 fill in mbuf - rearm_data and packet_type. */
 		rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]);
-		if (rxq->hw_timestamp) {
+		if (unlikely(rxq->shared)) {
+			pkts[pos]->port = cq[pos].user_index_low;
+			pkts[pos + p1]->port = cq[pos + p1].user_index_low;
+			pkts[pos + p2]->port = cq[pos + p2].user_index_low;
+			pkts[pos + p3]->port = cq[pos + p3].user_index_low;
+		}
+		if (unlikely(rxq->hw_timestamp)) {
 			int offset = rxq->timestamp_offset;
 			if (rxq->rt_timestamp) {
 				struct mlx5_dev_ctx_shared *sh = rxq->sh;

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:34 UTC #246 lore raw

v4 cover net/mlx5: support shared Rx queue

In reply to Xueming Li

Implemetation of Shared Rx queue.

v1:
- initial version
v2:
- rebased on latest dependent series
- fully tested
- support share_qid of RxQ configuration
v3:
- internally reviewed
- removed MPRQ support
- fixed multi-segment support
- fixed configure not applied after port restart
v4:
- rebase with latest code

Viacheslav Ovsiienko (1):
net/mlx5: add shared Rx queue port datapath support

 doc/guides/nics/features/mlx5.ini        |   1 +
 doc/guides/nics/mlx5.rst                 |   6 +
 drivers/common/mlx5/mlx5_common_devx.c   | 295 +++++++++--
 drivers/common/mlx5/mlx5_common_devx.h   |  19 +-
 drivers/common/mlx5/mlx5_devx_cmds.c     |  52 ++
 drivers/common/mlx5/mlx5_devx_cmds.h     |  16 +
 drivers/common/mlx5/mlx5_prm.h           |  93 +++-
 drivers/common/mlx5/version.map          |   1 +
 drivers/net/mlx5/linux/mlx5_os.c         |   2 +
 drivers/net/mlx5/linux/mlx5_verbs.c      | 169 +++---
 drivers/net/mlx5/mlx5.c                  |  10 +-
 drivers/net/mlx5/mlx5.h                  |  17 +-
 drivers/net/mlx5/mlx5_devx.c             | 270 +++++-----
 drivers/net/mlx5/mlx5_ethdev.c           |  21 +-
 drivers/net/mlx5/mlx5_flow.c             |  47 +-
 drivers/net/mlx5/mlx5_rss.c              |   6 +-
 drivers/net/mlx5/mlx5_rx.c               |  31 +-
 drivers/net/mlx5/mlx5_rx.h               |  45 +-
 drivers/net/mlx5/mlx5_rxq.c              | 630 +++++++++++++++++------
 drivers/net/mlx5/mlx5_rxtx.c             |   6 +-
 drivers/net/mlx5/mlx5_rxtx_vec.c         |   8 +-
 drivers/net/mlx5/mlx5_rxtx_vec_altivec.h |  14 +-
 drivers/net/mlx5/mlx5_rxtx_vec_neon.h    |  12 +-
 drivers/net/mlx5/mlx5_rxtx_vec_sse.h     |   8 +-
 drivers/net/mlx5/mlx5_stats.c            |   9 +-
 drivers/net/mlx5/mlx5_trigger.c          | 155 +++---
 drivers/net/mlx5/mlx5_vlan.c             |  16 +-
 drivers/regex/mlx5/mlx5_regex_fastpath.c |   2 +-
 28 files changed, 1377 insertions(+), 584 deletions(-)

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:34 UTC #247 lore raw

PATCH v4 1/14 common/mlx5: introduce user index field in completion

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/common/mlx5/mlx5_prm.h           | 8 +++++++-
 drivers/regex/mlx5/mlx5_regex_fastpath.c | 2 +-
 2 files changed, 8 insertions(+), 2 deletions(-)

drivers/common/mlx5/mlx5_prm.h +7 -1

index 8014ec2f925..c85634c774c 100644
@@ -393,7 +393,13 @@ struct mlx5_cqe {
 	uint16_t hdr_type_etc;
 	uint16_t vlan_info;
 	uint8_t lro_num_seg;
-	uint8_t rsvd3[3];
+	union {
+		uint8_t user_index_bytes[3];
+		struct {
+			uint8_t user_index_hi;
+			uint16_t user_index_low;
+		} __rte_packed;
+	};
 	uint32_t flow_table_metadata;
 	uint8_t rsvd4[4];
 	uint32_t byte_cnt;

drivers/regex/mlx5/mlx5_regex_fastpath.c +1 -1

index adb5343a46b..6836203ecf2 100644
@@ -559,7 +559,7 @@ mlx5_regexdev_dequeue(struct rte_regexdev *dev, uint16_t qp_id,
 		uint16_t wq_counter
 			= (rte_be_to_cpu_16(cqe->wqe_counter) + 1) &
 			  MLX5_REGEX_MAX_WQE_INDEX;
-		size_t hw_qpid = cqe->rsvd3[2];
+		size_t hw_qpid = cqe->user_index_bytes[2];
 		struct mlx5_regex_hw_qp *qp_obj = &queue->qps[hw_qpid];
 
 		/* UMR mode WQE counter move as WQE set(4 WQEBBS).*/

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:34 UTC #248 lore raw

PATCH v4 2/14 net/mlx5: fix field reference for PPC

In reply to Xueming Li

This patch fixes stale field reference.

Fixes a18ac6113331 ("net/mlx5: add metadata support to Rx datapath")

Cc [email protected]

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +4 -4

index bcf487c34e9..1d00c1c43d1 100644
@@ -974,10 +974,10 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 			(vector unsigned short)cqe_tmp1, cqe_sel_mask1);
 		cqe_tmp2 = (vector unsigned char)(vector unsigned long){
 			*(__rte_aligned(8) unsigned long *)
-			&cq[pos + p3].rsvd3[9], 0LL};
+			&cq[pos + p3].rsvd4[2], 0LL};
 		cqe_tmp1 = (vector unsigned char)(vector unsigned long){
 			*(__rte_aligned(8) unsigned long *)
-			&cq[pos + p2].rsvd3[9], 0LL};
+			&cq[pos + p2].rsvd4[2], 0LL};
 		cqes[3] = (vector unsigned char)
 			vec_sel((vector unsigned short)cqes[3],
 			(vector unsigned short)cqe_tmp2,

@@ -1037,10 +1037,10 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 			(vector unsigned short)cqe_tmp1, cqe_sel_mask1);
 		cqe_tmp2 = (vector unsigned char)(vector unsigned long){
 			*(__rte_aligned(8) unsigned long *)
-			&cq[pos + p1].rsvd3[9], 0LL};
+			&cq[pos + p1].rsvd4[2], 0LL};
 		cqe_tmp1 = (vector unsigned char)(vector unsigned long){
 			*(__rte_aligned(8) unsigned long *)
-			&cq[pos].rsvd3[9], 0LL};
+			&cq[pos].rsvd4[2], 0LL};
 		cqes[1] = (vector unsigned char)
 			vec_sel((vector unsigned short)cqes[1],
 			(vector unsigned short)cqe_tmp2, cqe_sel_mask2);

Signature · 1 line

2.33.0

Raslan Darawsheh

2021-11-04 17:07 UTC #249 lore raw

Re: [dpdk-dev] [PATCH v4 02/14] net/mlx5: fix field reference for PPC

Hi,

Quoted 10 lines

-----Original Message-----
From: dev <[email protected]> On Behalf Of Xueming Li
Sent: Thursday, November 4, 2021 2:33 PM
To: [email protected]
Cc: Xueming(Steven) Li <[email protected]>; Lior Margalit <[email protected]>; Slava Ovsiienko <[email protected]>; [email protected]; David Christensen <[email protected]>; Matan Azrad <[email protected]>; Yongseok Koh <[email protected]>
Subject: [dpdk-dev] [PATCH v4 02/14] net/mlx5: fix field reference for PPC

Quoted 6 lines

This patch fixes stale field reference.
Fixes a18ac6113331 ("net/mlx5: add metadata support to Rx datapath")
Cc [email protected]
Cc [email protected]

This should be the first patch in the series since the first patch is removing the rsvd3 from the structure. I'll change the order during integration,

Kindest regards,
Raslan Darawsheh

David Christensen

Reviewed-by on v4 2/14

2021-11-04 17:49 UTC #250 lore raw

Re: [dpdk-dev] [PATCH v4 02/14] net/mlx5: fix field reference for PPC

In reply to Xueming Li

Quoted 33 lines · Xueming Li

This patch fixes stale field reference.

Fixes a18ac6113331 ("net/mlx5: add metadata support to Rx datapath")

Cc [email protected]

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
  drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 8 ++++----
  1 file changed, 4 insertions(+), 4 deletions(-)

drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +2 -2

index bcf487c34e9..1d00c1c43d1 100644
@@ -974,10 +974,10 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
  			(vector unsigned short)cqe_tmp1, cqe_sel_mask1);
  		cqe_tmp2 = (vector unsigned char)(vector unsigned long){
  			*(__rte_aligned(8) unsigned long *)
-			&cq[pos + p3].rsvd3[9], 0LL};
+			&cq[pos + p3].rsvd4[2], 0LL};
  		cqe_tmp1 = (vector unsigned char)(vector unsigned long){
  			*(__rte_aligned(8) unsigned long *)
-			&cq[pos + p2].rsvd3[9], 0LL};
+			&cq[pos + p2].rsvd4[2], 0LL};
  		cqes[3] = (vector unsigned char)
  			vec_sel((vector unsigned short)cqes[3],
  			(vector unsigned short)cqe_tmp2,

@@ -1037,10 +1037,10 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
  			(vector unsigned short)cqe_tmp1, cqe_sel_mask1);
  		cqe_tmp2 = (vector unsigned char)(vector unsigned long){

drivers/net/mlx5/mlx5_rxtx_vec_altivec.h rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, · line 1046

Quoted 10 lines

drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +2 -2

  			*(__rte_aligned(8) unsigned long *)
-			&cq[pos + p1].rsvd3[9], 0LL};
+			&cq[pos + p1].rsvd4[2], 0LL};
  		cqe_tmp1 = (vector unsigned char)(vector unsigned long){
  			*(__rte_aligned(8) unsigned long *)
-			&cq[pos].rsvd3[9], 0LL};
+			&cq[pos].rsvd4[2], 0LL};
  		cqes[1] = (vector unsigned char)
  			vec_sel((vector unsigned short)cqes[1],
  			(vector unsigned short)cqe_tmp2, cqe_sel_mask2);

Reviewed-by David Christensen <[email protected]>

Xueming Li

author

2021-11-04 12:34 UTC #251 lore raw

PATCH v4 3/14 common/mlx5: adds basic receive memory pool support

In reply to Xueming Li

This patch adds DevX supports of PRM RMP object.

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/common/mlx5/mlx5_devx_cmds.c | 52 +++++++++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h | 16 ++++++
 drivers/common/mlx5/mlx5_prm.h       | 85 +++++++++++++++++++++++++++-
 drivers/common/mlx5/version.map      |  1 +
 4 files changed, 153 insertions(+), 1 deletion(-)

drivers/common/mlx5/mlx5_devx_cmds.c +52 -0

index 12c114a91b6..4ab3070da0c 100644
@@ -836,6 +836,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 			MLX5_GET(cmd_hca_cap, hcattr, flow_counter_bulk_alloc);
 	attr->flow_counters_dump = MLX5_GET(cmd_hca_cap, hcattr,
 					    flow_counters_dump);
+	attr->log_max_rmp = MLX5_GET(cmd_hca_cap, hcattr, log_max_rmp);
+	attr->mem_rq_rmp = MLX5_GET(cmd_hca_cap, hcattr, mem_rq_rmp);
 	attr->log_max_rqt_size = MLX5_GET(cmd_hca_cap, hcattr,
 					  log_max_rqt_size);
 	attr->eswitch_manager = MLX5_GET(cmd_hca_cap, hcattr, eswitch_manager);

@@ -1312,6 +1314,56 @@ mlx5_devx_cmd_modify_rq(struct mlx5_devx_obj *rq,
 }
 
 /**
+ * Create RMP using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param [in] rmp_attr
+ *   Pointer to create RMP attributes structure.
+ * @param [in] socket
+ *   CPU socket ID for allocations.
+ *
+ * @return
+ *   The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_rmp(void *ctx,
+			 struct mlx5_devx_create_rmp_attr *rmp_attr,
+			 int socket)
+{
+	uint32_t in[MLX5_ST_SZ_DW(create_rmp_in)] = {0};
+	uint32_t out[MLX5_ST_SZ_DW(create_rmp_out)] = {0};
+	void *rmp_ctx, *wq_ctx;
+	struct mlx5_devx_wq_attr *wq_attr;
+	struct mlx5_devx_obj *rmp = NULL;
+
+	rmp = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*rmp), 0, socket);
+	if (!rmp) {
+		DRV_LOG(ERR, "Failed to allocate RMP data");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	MLX5_SET(create_rmp_in, in, opcode, MLX5_CMD_OP_CREATE_RMP);
+	rmp_ctx = MLX5_ADDR_OF(create_rmp_in, in, ctx);
+	MLX5_SET(rmpc, rmp_ctx, state, rmp_attr->state);
+	MLX5_SET(rmpc, rmp_ctx, basic_cyclic_rcv_wqe,
+		 rmp_attr->basic_cyclic_rcv_wqe);
+	wq_ctx = MLX5_ADDR_OF(rmpc, rmp_ctx, wq);
+	wq_attr = &rmp_attr->wq_attr;
+	devx_cmd_fill_wq_data(wq_ctx, wq_attr);
+	rmp->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in), out,
+					      sizeof(out));
+	if (!rmp->obj) {
+		DRV_LOG(ERR, "Failed to create RMP using DevX");
+		rte_errno = errno;
+		mlx5_free(rmp);
+		return NULL;
+	}
+	rmp->id = MLX5_GET(create_rmp_out, out, rmpn);
+	return rmp;
+}
+
+/*
  * Create TIR using DevX API.
  *
  * @param[in] ctx

drivers/common/mlx5/mlx5_devx_cmds.h +16 -0

index 2326f1e9686..86ee4f7b78b 100644
@@ -152,6 +152,8 @@ mlx5_hca_parse_graph_node_base_hdr_len_mask
 struct mlx5_hca_attr {
 	uint32_t eswitch_manager:1;
 	uint32_t flow_counters_dump:1;
+	uint32_t mem_rq_rmp:1;
+	uint32_t log_max_rmp:5;
 	uint32_t log_max_rqt_size:5;
 	uint32_t parse_graph_flex_node:1;
 	uint8_t flow_counter_bulk_alloc_bitmap;

@@ -319,6 +321,17 @@ struct mlx5_devx_modify_rq_attr {
 	uint32_t lwm:16; /* Contained WQ lwm. */
 };
 
+/* Create RMP attributes structure, used by create RMP operation. */
+struct mlx5_devx_create_rmp_attr {
+	uint32_t rsvd0:8;
+	uint32_t state:4;
+	uint32_t rsvd1:20;
+	uint32_t basic_cyclic_rcv_wqe:1;
+	uint32_t rsvd4:31;
+	uint32_t rsvd8[10];
+	struct mlx5_devx_wq_attr wq_attr;
+};
+
 struct mlx5_rx_hash_field_select {
 	uint32_t l3_prot_type:1;
 	uint32_t l4_prot_type:1;

@@ -596,6 +609,9 @@ __rte_internal
 int mlx5_devx_cmd_modify_rq(struct mlx5_devx_obj *rq,
 			    struct mlx5_devx_modify_rq_attr *rq_attr);
 __rte_internal
+struct mlx5_devx_obj *mlx5_devx_cmd_create_rmp(void *ctx,
+			struct mlx5_devx_create_rmp_attr *rq_attr, int socket);
+__rte_internal
 struct mlx5_devx_obj *mlx5_devx_cmd_create_tir(void *ctx,
 					   struct mlx5_devx_tir_attr *tir_attr);
 __rte_internal

drivers/common/mlx5/mlx5_prm.h +84 -1

103 lines not on this page · raw message

drivers/common/mlx5/version.map +1 -0

index 0ea8325f9ac..7265ff8c56f 100644
@@ -30,6 +30,7 @@ INTERNAL {
 	mlx5_devx_cmd_create_geneve_tlv_option;
 	mlx5_devx_cmd_create_import_kek_obj;
 	mlx5_devx_cmd_create_qp;
+	mlx5_devx_cmd_create_rmp;
 	mlx5_devx_cmd_create_rq;
 	mlx5_devx_cmd_create_rqt;
 	mlx5_devx_cmd_create_sq;

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:34 UTC #252 lore raw

PATCH v4 4/14 common/mlx5: support receive memory pool

In reply to Xueming Li

This patch enables RMP based RQ, RMP is created when mlx5_devx_rq.rmp is set.

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/common/mlx5/mlx5_common_devx.c | 295 +++++++++++++++++++++----
 drivers/common/mlx5/mlx5_common_devx.h |  19 +-
 drivers/net/mlx5/mlx5_devx.c           |   4 +-
 3 files changed, 271 insertions(+), 47 deletions(-)

drivers/common/mlx5/mlx5_common_devx.c +253 ~~-42~~

index 825f84b1833..85b5282061a 100644
@@ -271,6 +271,39 @@ mlx5_devx_sq_create(void *ctx, struct mlx5_devx_sq *sq_obj, uint16_t log_wqbb_n,
 	return -rte_errno;
 }
 
+/**
+ * Destroy DevX Receive Queue resources.
+ *
+ * @param[in] rq_res
+ *   DevX RQ resource to destroy.
+ */
+static void
+mlx5_devx_wq_res_destroy(struct mlx5_devx_wq_res *rq_res)
+{
+	if (rq_res->umem_obj)
+		claim_zero(mlx5_os_umem_dereg(rq_res->umem_obj));
+	if (rq_res->umem_buf)
+		mlx5_free((void *)(uintptr_t)rq_res->umem_buf);
+	memset(rq_res, 0, sizeof(*rq_res));
+}
+
+/**
+ * Destroy DevX Receive Memory Pool.
+ *
+ * @param[in] rmp
+ *   DevX RMP to destroy.
+ */
+static void
+mlx5_devx_rmp_destroy(struct mlx5_devx_rmp *rmp)
+{
+	MLX5_ASSERT(rmp->ref_cnt == 0);
+	if (rmp->rmp) {
+		claim_zero(mlx5_devx_cmd_destroy(rmp->rmp));
+		rmp->rmp = NULL;
+	}
+	mlx5_devx_wq_res_destroy(&rmp->wq);
+}
+
 /**
  * Destroy DevX Queue Pair.
  *

@@ -389,55 +422,48 @@ mlx5_devx_qp_create(void *ctx, struct mlx5_devx_qp *qp_obj, uint16_t log_wqbb_n,
 void
 mlx5_devx_rq_destroy(struct mlx5_devx_rq *rq)
 {
-	if (rq->rq)
+	if (rq->rq) {
 		claim_zero(mlx5_devx_cmd_destroy(rq->rq));
-	if (rq->umem_obj)
-		claim_zero(mlx5_os_umem_dereg(rq->umem_obj));
-	if (rq->umem_buf)
-		mlx5_free((void *)(uintptr_t)rq->umem_buf);
+		rq->rq = NULL;
+		if (rq->rmp)
+			rq->rmp->ref_cnt--;
+	}
+	if (rq->rmp == NULL) {
+		mlx5_devx_wq_res_destroy(&rq->wq);
+	} else {
+		if (rq->rmp->ref_cnt == 0)
+			mlx5_devx_rmp_destroy(rq->rmp);
+	}
 }
 
 /**
- * Create Receive Queue using DevX API.
- *
- * Get a pointer to partially initialized attributes structure, and updates the
- * following fields:
- *   wq_umem_valid
- *   wq_umem_id
- *   wq_umem_offset
- *   dbr_umem_valid
- *   dbr_umem_id
- *   dbr_addr
- *   log_wq_pg_sz
- * All other fields are updated by caller.
+ * Create WQ resources using DevX API.
  *
  * @param[in] ctx
  *   Context returned from mlx5 open_device() glue function.
- * @param[in/out] rq_obj
- *   Pointer to RQ to create.
  * @param[in] wqe_size
  *   Size of WQE structure.
  * @param[in] log_wqbb_n
  *   Log of number of WQBBs in queue.
- * @param[in] attr
- *   Pointer to RQ attributes structure.
  * @param[in] socket
  *   Socket to use for allocation.
+ * @param[out] wq_attr
+ *   Pointer to WQ attributes structure.
+ * @param[out] wq_res
+ *   Pointer to WQ resource to create.
  *
  * @return
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
-int
-mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj, uint32_t wqe_size,
-		    uint16_t log_wqbb_n,
-		    struct mlx5_devx_create_rq_attr *attr, int socket)
+static int
+mlx5_devx_wq_init(void *ctx, uint32_t wqe_size, uint16_t log_wqbb_n, int socket,
+		  struct mlx5_devx_wq_attr *wq_attr,
+		  struct mlx5_devx_wq_res *wq_res)
 {
-	struct mlx5_devx_obj *rq = NULL;
 	struct mlx5dv_devx_umem *umem_obj = NULL;
 	void *umem_buf = NULL;
 	size_t alignment = MLX5_WQE_BUF_ALIGNMENT;
 	uint32_t umem_size, umem_dbrec;
-	uint16_t rq_size = 1 << log_wqbb_n;
 	int ret;
 
 	if (alignment == (size_t)-1) {

@@ -446,7 +472,7 @@ mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj, uint32_t wqe_size,
 		return -rte_errno;
 	}
 	/* Allocate memory buffer for WQEs and doorbell record. */
-	umem_size = wqe_size * rq_size;
+	umem_size = wqe_size * (1 << log_wqbb_n);
 	umem_dbrec = RTE_ALIGN(umem_size, MLX5_DBR_SIZE);
 	umem_size += MLX5_DBR_SIZE;
 	umem_buf = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, umem_size,

@@ -464,14 +490,60 @@ mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj, uint32_t wqe_size,
 		rte_errno = errno;
 		goto error;
 	}
+	/* Fill WQ attributes for RQ/RMP object creation. */
+	wq_attr->wq_umem_valid = 1;
+	wq_attr->wq_umem_id = mlx5_os_get_umem_id(umem_obj);
+	wq_attr->wq_umem_offset = 0;
+	wq_attr->dbr_umem_valid = 1;
+	wq_attr->dbr_umem_id = wq_attr->wq_umem_id;
+	wq_attr->dbr_addr = umem_dbrec;
+	wq_attr->log_wq_pg_sz = MLX5_LOG_PAGE_SIZE;
 	/* Fill attributes for RQ object creation. */
-	attr->wq_attr.wq_umem_valid = 1;
-	attr->wq_attr.wq_umem_id = mlx5_os_get_umem_id(umem_obj);
-	attr->wq_attr.wq_umem_offset = 0;
-	attr->wq_attr.dbr_umem_valid = 1;
-	attr->wq_attr.dbr_umem_id = attr->wq_attr.wq_umem_id;
-	attr->wq_attr.dbr_addr = umem_dbrec;
-	attr->wq_attr.log_wq_pg_sz = MLX5_LOG_PAGE_SIZE;
+	wq_res->umem_buf = umem_buf;
+	wq_res->umem_obj = umem_obj;
+	wq_res->db_rec = RTE_PTR_ADD(umem_buf, umem_dbrec);
+	return 0;
+error:
+	ret = rte_errno;
+	if (umem_obj)
+		claim_zero(mlx5_os_umem_dereg(umem_obj));
+	if (umem_buf)
+		mlx5_free((void *)(uintptr_t)umem_buf);
+	rte_errno = ret;
+	return -rte_errno;
+}
+
+/**
+ * Create standalone Receive Queue using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param[in/out] rq_obj
+ *   Pointer to RQ to create.
+ * @param[in] wqe_size
+ *   Size of WQE structure.
+ * @param[in] log_wqbb_n
+ *   Log of number of WQBBs in queue.
+ * @param[in] attr
+ *   Pointer to RQ attributes structure.
+ * @param[in] socket
+ *   Socket to use for allocation.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+mlx5_devx_rq_std_create(void *ctx, struct mlx5_devx_rq *rq_obj,
+			uint32_t wqe_size, uint16_t log_wqbb_n,
+			struct mlx5_devx_create_rq_attr *attr, int socket)
+{
+	struct mlx5_devx_obj *rq;
+	int ret;
+
+	ret = mlx5_devx_wq_init(ctx, wqe_size, log_wqbb_n, socket,
+				&attr->wq_attr, &rq_obj->wq);
+	if (ret != 0)
+		return ret;
 	/* Create receive queue object with DevX. */
 	rq = mlx5_devx_cmd_create_rq(ctx, attr, socket);
 	if (!rq) {

@@ -479,21 +551,160 @@ mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj, uint32_t wqe_size,
 		rte_errno = ENOMEM;
 		goto error;
 	}
-	rq_obj->umem_buf = umem_buf;
-	rq_obj->umem_obj = umem_obj;
 	rq_obj->rq = rq;
-	rq_obj->db_rec = RTE_PTR_ADD(rq_obj->umem_buf, umem_dbrec);
 	return 0;
 error:
 	ret = rte_errno;
-	if (umem_obj)
-		claim_zero(mlx5_os_umem_dereg(umem_obj));
-	if (umem_buf)
-		mlx5_free((void *)(uintptr_t)umem_buf);
+	mlx5_devx_wq_res_destroy(&rq_obj->wq);
+	rte_errno = ret;
+	return -rte_errno;
+}
+
+/**
+ * Create Receive Memory Pool using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param[in/out] rq_obj
+ *   Pointer to RQ to create.
+ * @param[in] wqe_size
+ *   Size of WQE structure.
+ * @param[in] log_wqbb_n
+ *   Log of number of WQBBs in queue.
+ * @param[in] attr
+ *   Pointer to RQ attributes structure.
+ * @param[in] socket
+ *   Socket to use for allocation.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+mlx5_devx_rmp_create(void *ctx, struct mlx5_devx_rmp *rmp_obj,
+		     uint32_t wqe_size, uint16_t log_wqbb_n,
+		     struct mlx5_devx_wq_attr *wq_attr, int socket)
+{
+	struct mlx5_devx_create_rmp_attr rmp_attr = { 0 };
+	int ret;
+
+	if (rmp_obj->rmp != NULL)
+		return 0;
+	rmp_attr.wq_attr = *wq_attr;
+	ret = mlx5_devx_wq_init(ctx, wqe_size, log_wqbb_n, socket,
+				&rmp_attr.wq_attr, &rmp_obj->wq);
+	if (ret != 0)
+		return ret;
+	rmp_attr.state = MLX5_RMPC_STATE_RDY;
+	rmp_attr.basic_cyclic_rcv_wqe =
+		wq_attr->wq_type != MLX5_WQ_TYPE_CYCLIC_STRIDING_RQ;
+	/* Create receive memory pool object with DevX. */
+	rmp_obj->rmp = mlx5_devx_cmd_create_rmp(ctx, &rmp_attr, socket);
+	if (rmp_obj->rmp == NULL) {
+		DRV_LOG(ERR, "Can't create DevX RMP object.");
+		rte_errno = ENOMEM;
+		goto error;
+	}
+	return 0;
+error:
+	ret = rte_errno;
+	mlx5_devx_wq_res_destroy(&rmp_obj->wq);
+	rte_errno = ret;
+	return -rte_errno;
+}
+
+/**
+ * Create Shared Receive Queue based on RMP using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param[in/out] rq_obj
+ *   Pointer to RQ to create.
+ * @param[in] wqe_size
+ *   Size of WQE structure.
+ * @param[in] log_wqbb_n
+ *   Log of number of WQBBs in queue.
+ * @param[in] attr
+ *   Pointer to RQ attributes structure.
+ * @param[in] socket
+ *   Socket to use for allocation.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+mlx5_devx_rq_shared_create(void *ctx, struct mlx5_devx_rq *rq_obj,
+			   uint32_t wqe_size, uint16_t log_wqbb_n,
+			   struct mlx5_devx_create_rq_attr *attr, int socket)
+{
+	struct mlx5_devx_obj *rq;
+	int ret;
+
+	ret = mlx5_devx_rmp_create(ctx, rq_obj->rmp, wqe_size, log_wqbb_n,
+				   &attr->wq_attr, socket);
+	if (ret != 0)
+		return ret;
+	attr->mem_rq_type = MLX5_RQC_MEM_RQ_TYPE_MEMORY_RQ_RMP;
+	attr->rmpn = rq_obj->rmp->rmp->id;
+	attr->flush_in_error_en = 0;
+	memset(&attr->wq_attr, 0, sizeof(attr->wq_attr));
+	/* Create receive queue object with DevX. */
+	rq = mlx5_devx_cmd_create_rq(ctx, attr, socket);
+	if (!rq) {
+		DRV_LOG(ERR, "Can't create DevX RMP RQ object.");
+		rte_errno = ENOMEM;
+		goto error;
+	}
+	rq_obj->rq = rq;
+	rq_obj->rmp->ref_cnt++;
+	return 0;
+error:
+	ret = rte_errno;
+	mlx5_devx_rq_destroy(rq_obj);
 	rte_errno = ret;
 	return -rte_errno;
 }
 
+/**
+ * Create Receive Queue using DevX API. Shared RQ is created only if rmp set.
+ *
+ * Get a pointer to partially initialized attributes structure, and updates the
+ * following fields:
+ *   wq_umem_valid
+ *   wq_umem_id
+ *   wq_umem_offset
+ *   dbr_umem_valid
+ *   dbr_umem_id
+ *   dbr_addr
+ *   log_wq_pg_sz
+ * All other fields are updated by caller.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param[in/out] rq_obj
+ *   Pointer to RQ to create.
+ * @param[in] wqe_size
+ *   Size of WQE structure.
+ * @param[in] log_wqbb_n
+ *   Log of number of WQBBs in queue.
+ * @param[in] attr
+ *   Pointer to RQ attributes structure.
+ * @param[in] socket
+ *   Socket to use for allocation.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_devx_rq_create(void *ctx, struct mlx5_devx_rq *rq_obj,
+		    uint32_t wqe_size, uint16_t log_wqbb_n,
+		    struct mlx5_devx_create_rq_attr *attr, int socket)
+{
+	if (rq_obj->rmp == NULL)
+		return mlx5_devx_rq_std_create(ctx, rq_obj, wqe_size,
+					       log_wqbb_n, attr, socket);
+	return mlx5_devx_rq_shared_create(ctx, rq_obj, wqe_size,
+					  log_wqbb_n, attr, socket);
+}
 
 /**
  * Change QP state to RTS.

drivers/common/mlx5/mlx5_common_devx.h +16 -3

index f699405f69b..7ceac040f8b 100644
@@ -45,14 +45,27 @@ struct mlx5_devx_qp {
 	volatile uint32_t *db_rec; /* The QP doorbell record. */
 };
 
-/* DevX Receive Queue structure. */
-struct mlx5_devx_rq {
-	struct mlx5_devx_obj *rq; /* The RQ DevX object. */
+/* DevX Receive Queue resource structure. */
+struct mlx5_devx_wq_res {
 	void *umem_obj; /* The RQ umem object. */
 	volatile void *umem_buf;
 	volatile uint32_t *db_rec; /* The RQ doorbell record. */
 };
 
+/* DevX Receive Memory Pool structure. */
+struct mlx5_devx_rmp {
+	struct mlx5_devx_obj *rmp; /* The RMP DevX object. */
+	uint32_t ref_cnt; /* Reference count. */
+	struct mlx5_devx_wq_res wq;
+};
+
+/* DevX Receive Queue structure. */
+struct mlx5_devx_rq {
+	struct mlx5_devx_obj *rq; /* The RQ DevX object. */
+	struct mlx5_devx_rmp *rmp; /* Shared RQ RMP object. */
+	struct mlx5_devx_wq_res wq; /* WQ resource of standalone RQ. */
+};
+
 /* mlx5_common_devx.c */
 
 __rte_internal

drivers/net/mlx5/mlx5_devx.c +2 -2

index 424f77be790..443252df05d 100644
@@ -515,8 +515,8 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	ret = mlx5_devx_modify_rq(tmpl, MLX5_RXQ_MOD_RST2RDY);
 	if (ret)
 		goto error;
-	rxq_data->wqes = (void *)(uintptr_t)tmpl->rq_obj.umem_buf;
-	rxq_data->rq_db = (uint32_t *)(uintptr_t)tmpl->rq_obj.db_rec;
+	rxq_data->wqes = (void *)(uintptr_t)tmpl->rq_obj.wq.umem_buf;
+	rxq_data->rq_db = (uint32_t *)(uintptr_t)tmpl->rq_obj.wq.db_rec;
 	rxq_data->cq_arm_sn = 0;
 	rxq_data->cq_ci = 0;
 	mlx5_rxq_initialize(rxq_data);

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:35 UTC #253 lore raw

PATCH v4 5/14 net/mlx5: fix Rx queue memory allocation return value

In reply to Xueming Li

If error happened during Rx queue mbuf allocation, boolean value returned. From description, return value should be error number.

This patch returns negative error number.

Fixes 0f20acbf5eda ("net/mlx5: implement vectorized MPRQ burst")

Cc [email protected]

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/net/mlx5/mlx5_rxq.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

drivers/net/mlx5/mlx5_rxq.c +5 -3

index 9220bb2c15c..4567b43c1b6 100644
@@ -129,7 +129,7 @@ rxq_alloc_elts_mprq(struct mlx5_rxq_ctrl *rxq_ctrl)
  *   Pointer to RX queue structure.
  *
  * @return
- *   0 on success, errno value on failure.
+ *   0 on success, negative errno value on failure.
  */
 static int
 rxq_alloc_elts_sprq(struct mlx5_rxq_ctrl *rxq_ctrl)

@@ -220,7 +220,7 @@ rxq_alloc_elts_sprq(struct mlx5_rxq_ctrl *rxq_ctrl)
  *   Pointer to RX queue structure.
  *
  * @return
- *   0 on success, errno value on failure.
+ *   0 on success, negative errno value on failure.
  */
 int
 rxq_alloc_elts(struct mlx5_rxq_ctrl *rxq_ctrl)

@@ -233,7 +233,9 @@ rxq_alloc_elts(struct mlx5_rxq_ctrl *rxq_ctrl)
 	 */
 	if (mlx5_rxq_mprq_enabled(&rxq_ctrl->rxq))
 		ret = rxq_alloc_elts_mprq(rxq_ctrl);
-	return (ret || rxq_alloc_elts_sprq(rxq_ctrl));
+	if (ret == 0)
+		ret = rxq_alloc_elts_sprq(rxq_ctrl);
+	return ret;
 }
 
 /**

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:34 UTC #254 lore raw

PATCH v4 6/14 net/mlx5: clean Rx queue code

In reply to Xueming Li

This patch removes unused Rx queue code.

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/net/mlx5/mlx5_rxq.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

drivers/net/mlx5/mlx5_rxq.c +2 -6

index 4567b43c1b6..b2e4389ad60 100644
@@ -674,9 +674,7 @@ mlx5_rx_queue_setup(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc,
 		    struct rte_mempool *mp)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq = (*priv->rxqs)[idx];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_rxq_ctrl *rxq_ctrl;
 	struct rte_eth_rxseg_split *rx_seg =
 				(struct rte_eth_rxseg_split *)conf->rx_seg;
 	struct rte_eth_rxseg_split rx_single = {.mp = mp};

@@ -743,9 +741,7 @@ mlx5_rx_hairpin_queue_setup(struct rte_eth_dev *dev, uint16_t idx,
 			    const struct rte_eth_hairpin_conf *hairpin_conf)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq = (*priv->rxqs)[idx];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_rxq_ctrl *rxq_ctrl;
 	int res;
 
 	res = mlx5_rx_queue_pre_setup(dev, idx, &desc);

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:35 UTC #255 lore raw

PATCH v4 7/14 net/mlx5: split Rx queue into shareable and private

In reply to Xueming Li

To prepare shared Rx queue, splits RxQ data into shareable and private. Struct mlx5_rxq_priv is per queue data.
Struct mlx5_rxq_ctrl is shared queue resources and data.

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/net/mlx5/mlx5.c        |  4 +++
 drivers/net/mlx5/mlx5.h        |  5 ++-
 drivers/net/mlx5/mlx5_ethdev.c | 10 ++++++
 drivers/net/mlx5/mlx5_rx.h     | 17 +++++++--
 drivers/net/mlx5/mlx5_rxq.c    | 66 ++++++++++++++++++++++++++++------
 5 files changed, 88 insertions(+), 14 deletions(-)

drivers/net/mlx5/mlx5.c +4 -0

index dc15688f216..374cc9757aa 100644
@@ -1700,6 +1700,10 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 		mlx5_free(dev->intr_handle);
 		dev->intr_handle = NULL;
 	}
+	if (priv->rxq_privs != NULL) {
+		mlx5_free(priv->rxq_privs);
+		priv->rxq_privs = NULL;
+	}
 	if (priv->txqs != NULL) {
 		/* XXX race condition if mlx5_tx_burst() is still running. */
 		rte_delay_us_sleep(1000);

drivers/net/mlx5/mlx5.h +4 -1

index 74af88ec194..4e99fe7d068 100644
@@ -1345,6 +1345,8 @@ enum mlx5_txq_modify_type {
 	MLX5_TXQ_MOD_ERR2RDY, /* modify state from error to ready. */
 };
 
+struct mlx5_rxq_priv;
+
 /* HW objects operations structure. */
 struct mlx5_obj_ops {
 	int (*rxq_obj_modify_vlan_strip)(struct mlx5_rxq_obj *rxq_obj, int on);

@@ -1408,7 +1410,8 @@ struct mlx5_priv {
 	/* RX/TX queues. */
 	unsigned int rxqs_n; /* RX queues array size. */
 	unsigned int txqs_n; /* TX queues array size. */
-	struct mlx5_rxq_data *(*rxqs)[]; /* RX queues. */
+	struct mlx5_rxq_priv *(*rxq_privs)[]; /* RX queue non-shared data. */
+	struct mlx5_rxq_data *(*rxqs)[]; /* (Shared) RX queues. */
 	struct mlx5_txq_data *(*txqs)[]; /* TX queues. */
 	struct rte_mempool *mprq_mp; /* Mempool for Multi-Packet RQ. */
 	struct rte_eth_rss_conf rss_conf; /* RSS configuration. */

drivers/net/mlx5/mlx5_ethdev.c +10 -0

index 81fa8845bb5..cde505955df 100644
@@ -104,6 +104,16 @@ mlx5_dev_configure(struct rte_eth_dev *dev)
 	       MLX5_RSS_HASH_KEY_LEN);
 	priv->rss_conf.rss_key_len = MLX5_RSS_HASH_KEY_LEN;
 	priv->rss_conf.rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+	priv->rxq_privs = mlx5_realloc(priv->rxq_privs,
+				       MLX5_MEM_RTE | MLX5_MEM_ZERO,
+				       sizeof(void *) * rxqs_n, 0,
+				       SOCKET_ID_ANY);
+	if (priv->rxq_privs == NULL) {
+		DRV_LOG(ERR, "port %u cannot allocate rxq private data",
+			dev->data->port_id);
+		rte_errno = ENOMEM;
+		return -rte_errno;
+	}
 	priv->rxqs = (void *)dev->data->rx_queues;
 	priv->txqs = (void *)dev->data->tx_queues;
 	if (txqs_n != priv->txqs_n) {

drivers/net/mlx5/mlx5_rx.h +15 -2

index 69b1263339e..fa24f5cdf3a 100644
@@ -150,10 +150,14 @@ struct mlx5_rxq_ctrl {
 	struct mlx5_rxq_data rxq; /* Data path structure. */
 	LIST_ENTRY(mlx5_rxq_ctrl) next; /* Pointer to the next element. */
 	uint32_t refcnt; /* Reference counter. */
+	LIST_HEAD(priv, mlx5_rxq_priv) owners; /* Owner rxq list. */
 	struct mlx5_rxq_obj *obj; /* Verbs/DevX elements. */
+	struct mlx5_dev_ctx_shared *sh; /* Shared context. */
 	struct mlx5_priv *priv; /* Back pointer to private data. */
 	enum mlx5_rxq_type type; /* Rxq type. */
 	unsigned int socket; /* CPU socket ID for allocations. */
+	uint32_t share_group; /* Group ID of shared RXQ. */
+	uint16_t share_qid; /* Shared RxQ ID in group. */
 	unsigned int irq:1; /* Whether IRQ is enabled. */
 	uint32_t flow_mark_n; /* Number of Mark/Flag flows using this Queue. */
 	uint32_t flow_tunnels_n[MLX5_FLOW_TUNNEL]; /* Tunnels counters. */

@@ -163,6 +167,14 @@ struct mlx5_rxq_ctrl {
 	uint32_t hairpin_status; /* Hairpin binding status. */
 };
 
+/* RX queue private data. */
+struct mlx5_rxq_priv {
+	uint16_t idx; /* Queue index. */
+	struct mlx5_rxq_ctrl *ctrl; /* Shared Rx Queue. */
+	LIST_ENTRY(mlx5_rxq_priv) owner_entry; /* Entry in shared rxq_ctrl. */
+	struct mlx5_priv *priv; /* Back pointer to private data. */
+};
+
 /* mlx5_rxq.c */
 
 extern uint8_t rss_hash_default_key[];

@@ -186,13 +198,14 @@ void mlx5_rx_intr_vec_disable(struct rte_eth_dev *dev);
 int mlx5_rx_intr_enable(struct rte_eth_dev *dev, uint16_t rx_queue_id);
 int mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id);
 int mlx5_rxq_obj_verify(struct rte_eth_dev *dev);
-struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx,
+struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev,
+				   struct mlx5_rxq_priv *rxq,
 				   uint16_t desc, unsigned int socket,
 				   const struct rte_eth_rxconf *conf,
 				   const struct rte_eth_rxseg_split *rx_seg,
 				   uint16_t n_seg);
 struct mlx5_rxq_ctrl *mlx5_rxq_hairpin_new
-	(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc,
+	(struct rte_eth_dev *dev, struct mlx5_rxq_priv *rxq, uint16_t desc,
 	 const struct rte_eth_hairpin_conf *hairpin_conf);
 struct mlx5_rxq_ctrl *mlx5_rxq_get(struct rte_eth_dev *dev, uint16_t idx);
 int mlx5_rxq_release(struct rte_eth_dev *dev, uint16_t idx);

drivers/net/mlx5/mlx5_rxq.c +55 ~~-11~~

163 lines not on this page · raw message

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:35 UTC #256 lore raw

PATCH v4 8/14 net/mlx5: move Rx queue reference count

In reply to Xueming Li

Rx queue reference count is counter of RQ, used to count reference to RQ object. To prepare for shared Rx queue, this patch moves it from rxq_ctrl to Rx queue private data.

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/net/mlx5/mlx5_rx.h      |   8 +-
 drivers/net/mlx5/mlx5_rxq.c     | 169 +++++++++++++++++++++-----------
 drivers/net/mlx5/mlx5_trigger.c |  57 +++++------
 3 files changed, 142 insertions(+), 92 deletions(-)

drivers/net/mlx5/mlx5_rx.h +6 -2

index fa24f5cdf3a..eccfbf1108d 100644
@@ -149,7 +149,6 @@ enum mlx5_rxq_type {
 struct mlx5_rxq_ctrl {
 	struct mlx5_rxq_data rxq; /* Data path structure. */
 	LIST_ENTRY(mlx5_rxq_ctrl) next; /* Pointer to the next element. */
-	uint32_t refcnt; /* Reference counter. */
 	LIST_HEAD(priv, mlx5_rxq_priv) owners; /* Owner rxq list. */
 	struct mlx5_rxq_obj *obj; /* Verbs/DevX elements. */
 	struct mlx5_dev_ctx_shared *sh; /* Shared context. */

@@ -170,6 +169,7 @@ struct mlx5_rxq_ctrl {
 /* RX queue private data. */
 struct mlx5_rxq_priv {
 	uint16_t idx; /* Queue index. */
+	uint32_t refcnt; /* Reference counter. */
 	struct mlx5_rxq_ctrl *ctrl; /* Shared Rx Queue. */
 	LIST_ENTRY(mlx5_rxq_priv) owner_entry; /* Entry in shared rxq_ctrl. */
 	struct mlx5_priv *priv; /* Back pointer to private data. */

@@ -207,7 +207,11 @@ struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev,
 struct mlx5_rxq_ctrl *mlx5_rxq_hairpin_new
 	(struct rte_eth_dev *dev, struct mlx5_rxq_priv *rxq, uint16_t desc,
 	 const struct rte_eth_hairpin_conf *hairpin_conf);
-struct mlx5_rxq_ctrl *mlx5_rxq_get(struct rte_eth_dev *dev, uint16_t idx);
+struct mlx5_rxq_priv *mlx5_rxq_ref(struct rte_eth_dev *dev, uint16_t idx);
+uint32_t mlx5_rxq_deref(struct rte_eth_dev *dev, uint16_t idx);
+struct mlx5_rxq_priv *mlx5_rxq_get(struct rte_eth_dev *dev, uint16_t idx);
+struct mlx5_rxq_ctrl *mlx5_rxq_ctrl_get(struct rte_eth_dev *dev, uint16_t idx);
+struct mlx5_rxq_data *mlx5_rxq_data_get(struct rte_eth_dev *dev, uint16_t idx);
 int mlx5_rxq_release(struct rte_eth_dev *dev, uint16_t idx);
 int mlx5_rxq_verify(struct rte_eth_dev *dev);
 int rxq_alloc_elts(struct mlx5_rxq_ctrl *rxq_ctrl);

drivers/net/mlx5/mlx5_rxq.c +113 ~~-56~~

307 lines not on this page · raw message

drivers/net/mlx5/mlx5_trigger.c +23 ~~-34~~

227 lines not on this page · raw message

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:35 UTC #257 lore raw

PATCH v4 9/14 net/mlx5: move Rx queue hairpin info to private data

In reply to Xueming Li

Hairpin info of Rx queue can't be shared, moves to private queue data.

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/net/mlx5/mlx5_rx.h      |  4 ++--
 drivers/net/mlx5/mlx5_rxq.c     | 13 +++++--------
 drivers/net/mlx5/mlx5_trigger.c | 24 ++++++++++++------------
 3 files changed, 19 insertions(+), 22 deletions(-)

drivers/net/mlx5/mlx5_rx.h +2 -2

index eccfbf1108d..b21918223b8 100644
@@ -162,8 +162,6 @@ struct mlx5_rxq_ctrl {
 	uint32_t flow_tunnels_n[MLX5_FLOW_TUNNEL]; /* Tunnels counters. */
 	uint32_t wqn; /* WQ number. */
 	uint16_t dump_file_n; /* Number of dump files. */
-	struct rte_eth_hairpin_conf hairpin_conf; /* Hairpin configuration. */
-	uint32_t hairpin_status; /* Hairpin binding status. */
 };
 
 /* RX queue private data. */

@@ -173,6 +171,8 @@ struct mlx5_rxq_priv {
 	struct mlx5_rxq_ctrl *ctrl; /* Shared Rx Queue. */
 	LIST_ENTRY(mlx5_rxq_priv) owner_entry; /* Entry in shared rxq_ctrl. */
 	struct mlx5_priv *priv; /* Back pointer to private data. */
+	struct rte_eth_hairpin_conf hairpin_conf; /* Hairpin configuration. */
+	uint32_t hairpin_status; /* Hairpin binding status. */
 };
 
 /* mlx5_rxq.c */

drivers/net/mlx5/mlx5_rxq.c +5 -8

index 8071ddbd61c..7b637fda643 100644
@@ -1695,8 +1695,8 @@ mlx5_rxq_hairpin_new(struct rte_eth_dev *dev, struct mlx5_rxq_priv *rxq,
 	tmpl->rxq.elts_n = log2above(desc);
 	tmpl->rxq.elts = NULL;
 	tmpl->rxq.mr_ctrl.cache_bh = (struct mlx5_mr_btree) { 0 };
-	tmpl->hairpin_conf = *hairpin_conf;
 	tmpl->rxq.idx = idx;
+	rxq->hairpin_conf = *hairpin_conf;
 	mlx5_rxq_ref(dev, idx);
 	LIST_INSERT_HEAD(&priv->rxqsctrl, tmpl, next);
 	return tmpl;

@@ -1913,14 +1913,11 @@ const struct rte_eth_hairpin_conf *
 mlx5_rxq_get_hairpin_conf(struct rte_eth_dev *dev, uint16_t idx)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_ctrl *rxq_ctrl = NULL;
+	struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, idx);
 
-	if (idx < priv->rxqs_n && (*priv->rxqs)[idx]) {
-		rxq_ctrl = container_of((*priv->rxqs)[idx],
-					struct mlx5_rxq_ctrl,
-					rxq);
-		if (rxq_ctrl->type == MLX5_RXQ_TYPE_HAIRPIN)
-			return &rxq_ctrl->hairpin_conf;
+	if (idx < priv->rxqs_n && rxq != NULL) {
+		if (rxq->ctrl->type == MLX5_RXQ_TYPE_HAIRPIN)
+			return &rxq->hairpin_conf;
 	}
 	return NULL;
 }

drivers/net/mlx5/mlx5_trigger.c +12 ~~-12~~

index e5d74d275f8..a124f74fcda 100644
@@ -324,7 +324,7 @@ mlx5_hairpin_auto_bind(struct rte_eth_dev *dev)
 		}
 		rxq_ctrl = rxq->ctrl;
 		if (rxq_ctrl->type != MLX5_RXQ_TYPE_HAIRPIN ||
-		    rxq_ctrl->hairpin_conf.peers[0].queue != i) {
+		    rxq->hairpin_conf.peers[0].queue != i) {
 			rte_errno = ENOMEM;
 			DRV_LOG(ERR, "port %u Tx queue %d can't be binded to "
 				"Rx queue %d", dev->data->port_id,

@@ -354,7 +354,7 @@ mlx5_hairpin_auto_bind(struct rte_eth_dev *dev)
 		if (ret)
 			goto error;
 		/* Qs with auto-bind will be destroyed directly. */
-		rxq_ctrl->hairpin_status = 1;
+		rxq->hairpin_status = 1;
 		txq_ctrl->hairpin_status = 1;
 		mlx5_txq_release(dev, i);
 	}

@@ -457,9 +457,9 @@ mlx5_hairpin_queue_peer_update(struct rte_eth_dev *dev, uint16_t peer_queue,
 		}
 		peer_info->qp_id = rxq_ctrl->obj->rq->id;
 		peer_info->vhca_id = priv->config.hca_attr.vhca_id;
-		peer_info->peer_q = rxq_ctrl->hairpin_conf.peers[0].queue;
-		peer_info->tx_explicit = rxq_ctrl->hairpin_conf.tx_explicit;
-		peer_info->manual_bind = rxq_ctrl->hairpin_conf.manual_bind;
+		peer_info->peer_q = rxq->hairpin_conf.peers[0].queue;
+		peer_info->tx_explicit = rxq->hairpin_conf.tx_explicit;
+		peer_info->manual_bind = rxq->hairpin_conf.manual_bind;
 	}
 	return 0;
 }

@@ -581,20 +581,20 @@ mlx5_hairpin_queue_peer_bind(struct rte_eth_dev *dev, uint16_t cur_queue,
 				dev->data->port_id, cur_queue);
 			return -rte_errno;
 		}
-		if (rxq_ctrl->hairpin_status != 0) {
+		if (rxq->hairpin_status != 0) {
 			DRV_LOG(DEBUG, "port %u Rx queue %d is already bound",
 				dev->data->port_id, cur_queue);
 			return 0;
 		}
 		if (peer_info->tx_explicit !=
-		    rxq_ctrl->hairpin_conf.tx_explicit) {
+		    rxq->hairpin_conf.tx_explicit) {
 			rte_errno = EINVAL;
 			DRV_LOG(ERR, "port %u Rx queue %d and peer Tx rule mode"
 				" mismatch", dev->data->port_id, cur_queue);
 			return -rte_errno;
 		}
 		if (peer_info->manual_bind !=
-		    rxq_ctrl->hairpin_conf.manual_bind) {
+		    rxq->hairpin_conf.manual_bind) {
 			rte_errno = EINVAL;
 			DRV_LOG(ERR, "port %u Rx queue %d and peer binding mode"
 				" mismatch", dev->data->port_id, cur_queue);

@@ -606,7 +606,7 @@ mlx5_hairpin_queue_peer_bind(struct rte_eth_dev *dev, uint16_t cur_queue,
 		rq_attr.hairpin_peer_vhca = peer_info->vhca_id;
 		ret = mlx5_devx_cmd_modify_rq(rxq_ctrl->obj->rq, &rq_attr);
 		if (ret == 0)
-			rxq_ctrl->hairpin_status = 1;
+			rxq->hairpin_status = 1;
 	}
 	return ret;
 }

@@ -688,7 +688,7 @@ mlx5_hairpin_queue_peer_unbind(struct rte_eth_dev *dev, uint16_t cur_queue,
 				dev->data->port_id, cur_queue);
 			return -rte_errno;
 		}
-		if (rxq_ctrl->hairpin_status == 0) {
+		if (rxq->hairpin_status == 0) {
 			DRV_LOG(DEBUG, "port %u Rx queue %d is already unbound",
 				dev->data->port_id, cur_queue);
 			return 0;

@@ -703,7 +703,7 @@ mlx5_hairpin_queue_peer_unbind(struct rte_eth_dev *dev, uint16_t cur_queue,
 		rq_attr.rq_state = MLX5_SQC_STATE_RST;
 		ret = mlx5_devx_cmd_modify_rq(rxq_ctrl->obj->rq, &rq_attr);
 		if (ret == 0)
-			rxq_ctrl->hairpin_status = 0;
+			rxq->hairpin_status = 0;
 	}
 	return ret;
 }

@@ -1041,7 +1041,7 @@ mlx5_hairpin_get_peer_ports(struct rte_eth_dev *dev, uint16_t *peer_ports,
 			rxq_ctrl = rxq->ctrl;
 			if (rxq_ctrl->type != MLX5_RXQ_TYPE_HAIRPIN)
 				continue;
-			pp = rxq_ctrl->hairpin_conf.peers[0].port;
+			pp = rxq->hairpin_conf.peers[0].port;
 			if (pp >= RTE_MAX_ETHPORTS) {
 				rte_errno = ERANGE;
 				DRV_LOG(ERR, "port %hu queue %u peer port "

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:35 UTC #258 lore raw

PATCH v4 10/14 net/mlx5: remove port info from shareable Rx queue

In reply to Xueming Li

To prepare for shared Rx queue, removes port info from shareable Rx queue control.

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/net/mlx5/mlx5_devx.c     |  2 +-
 drivers/net/mlx5/mlx5_rx.c       | 15 +++--------
 drivers/net/mlx5/mlx5_rx.h       |  7 ++++--
 drivers/net/mlx5/mlx5_rxq.c      | 43 ++++++++++++++++++++++----------
 drivers/net/mlx5/mlx5_rxtx_vec.c |  2 +-
 drivers/net/mlx5/mlx5_trigger.c  | 13 +++++-----
 6 files changed, 47 insertions(+), 35 deletions(-)

drivers/net/mlx5/mlx5_devx.c +1 -1

index 443252df05d..8b3651f5034 100644
@@ -918,7 +918,7 @@ mlx5_rxq_devx_obj_drop_create(struct rte_eth_dev *dev)
 	}
 	rxq->rxq_ctrl = rxq_ctrl;
 	rxq_ctrl->type = MLX5_RXQ_TYPE_STANDARD;
-	rxq_ctrl->priv = priv;
+	rxq_ctrl->sh = priv->sh;
 	rxq_ctrl->obj = rxq;
 	rxq_data = &rxq_ctrl->rxq;
 	/* Create CQ using DevX API. */

drivers/net/mlx5/mlx5_rx.c +3 ~~-12~~

index 258a6453144..d41905a2a04 100644
@@ -118,15 +118,7 @@ int
 mlx5_rx_descriptor_status(void *rx_queue, uint16_t offset)
 {
 	struct mlx5_rxq_data *rxq = rx_queue;
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-			container_of(rxq, struct mlx5_rxq_ctrl, rxq);
-	struct rte_eth_dev *dev = ETH_DEV(rxq_ctrl->priv);
 
-	if (dev->rx_pkt_burst == NULL ||
-	    dev->rx_pkt_burst == removed_rx_burst) {
-		rte_errno = ENOTSUP;
-		return -rte_errno;
-	}
 	if (offset >= (1 << rxq->cqe_n)) {
 		rte_errno = EINVAL;
 		return -rte_errno;

@@ -438,10 +430,10 @@ mlx5_rx_err_handle(struct mlx5_rxq_data *rxq, uint8_t vec)
 		sm.is_wq = 1;
 		sm.queue_id = rxq->idx;
 		sm.state = IBV_WQS_RESET;
-		if (mlx5_queue_state_modify(ETH_DEV(rxq_ctrl->priv), &sm))
+		if (mlx5_queue_state_modify(RXQ_DEV(rxq_ctrl), &sm))
 			return -1;
 		if (rxq_ctrl->dump_file_n <
-		    rxq_ctrl->priv->config.max_dump_files_num) {
+		    RXQ_PORT(rxq_ctrl)->config.max_dump_files_num) {
 			MKSTR(err_str, "Unexpected CQE error syndrome "
 			      "0x%02x CQN = %u RQN = %u wqe_counter = %u"
 			      " rq_ci = %u cq_ci = %u", u.err_cqe->syndrome,

@@ -478,8 +470,7 @@ mlx5_rx_err_handle(struct mlx5_rxq_data *rxq, uint8_t vec)
 			sm.is_wq = 1;
 			sm.queue_id = rxq->idx;
 			sm.state = IBV_WQS_RDY;
-			if (mlx5_queue_state_modify(ETH_DEV(rxq_ctrl->priv),
-						    &sm))
+			if (mlx5_queue_state_modify(RXQ_DEV(rxq_ctrl), &sm))
 				return -1;
 			if (vec) {
 				const uint32_t elts_n =

drivers/net/mlx5/mlx5_rx.h +5 -2

index b21918223b8..c04c0c73349 100644
@@ -22,6 +22,10 @@
 /* Support tunnel matching. */
 #define MLX5_FLOW_TUNNEL 10
 
+#define RXQ_PORT(rxq_ctrl) LIST_FIRST(&(rxq_ctrl)->owners)->priv
+#define RXQ_DEV(rxq_ctrl) ETH_DEV(RXQ_PORT(rxq_ctrl))
+#define RXQ_PORT_ID(rxq_ctrl) PORT_ID(RXQ_PORT(rxq_ctrl))
+
 /* First entry must be NULL for comparison. */
 #define mlx5_mr_btree_len(bt) ((bt)->len - 1)

@@ -152,7 +156,6 @@ struct mlx5_rxq_ctrl {
 	LIST_HEAD(priv, mlx5_rxq_priv) owners; /* Owner rxq list. */
 	struct mlx5_rxq_obj *obj; /* Verbs/DevX elements. */
 	struct mlx5_dev_ctx_shared *sh; /* Shared context. */
-	struct mlx5_priv *priv; /* Back pointer to private data. */
 	enum mlx5_rxq_type type; /* Rxq type. */
 	unsigned int socket; /* CPU socket ID for allocations. */
 	uint32_t share_group; /* Group ID of shared RXQ. */

@@ -318,7 +321,7 @@ mlx5_rx_addr2mr(struct mlx5_rxq_data *rxq, uintptr_t addr)
 	 */
 	rxq_ctrl = container_of(rxq, struct mlx5_rxq_ctrl, rxq);
 	mp = mlx5_rxq_mprq_enabled(rxq) ? rxq->mprq_mp : rxq->mp;
-	return mlx5_mr_mempool2mr_bh(&rxq_ctrl->priv->sh->cdev->mr_scache,
+	return mlx5_mr_mempool2mr_bh(&rxq_ctrl->sh->cdev->mr_scache,
 				     mr_ctrl, mp, addr);
 }

drivers/net/mlx5/mlx5_rxq.c +30 ~~-13~~

79 lines not on this page · raw message

drivers/net/mlx5/mlx5_rxtx_vec.c +1 -1

index ecd273e00a8..511681841ca 100644
@@ -550,7 +550,7 @@ mlx5_rxq_check_vec_support(struct mlx5_rxq_data *rxq)
 	struct mlx5_rxq_ctrl *ctrl =
 		container_of(rxq, struct mlx5_rxq_ctrl, rxq);
 
-	if (!ctrl->priv->config.rx_vec_en || rxq->sges_n != 0)
+	if (!RXQ_PORT(ctrl)->config.rx_vec_en || rxq->sges_n != 0)
 		return -ENOTSUP;
 	if (rxq->lro)
 		return -ENOTSUP;

drivers/net/mlx5/mlx5_trigger.c +7 -6

index a124f74fcda..caafdf27e8f 100644
@@ -131,9 +131,11 @@ mlx5_rxq_mempool_register_cb(struct rte_mempool *mp, void *opaque,
  *   0 on success, (-1) on failure and rte_errno is set.
  */
 static int
-mlx5_rxq_mempool_register(struct mlx5_rxq_ctrl *rxq_ctrl)
+mlx5_rxq_mempool_register(struct rte_eth_dev *dev,
+			  struct mlx5_rxq_ctrl *rxq_ctrl)
 {
-	struct mlx5_priv *priv = rxq_ctrl->priv;
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_dev_ctx_shared *sh = rxq_ctrl->sh;
 	struct rte_mempool *mp;
 	uint32_t s;
 	int ret = 0;

@@ -148,9 +150,8 @@ mlx5_rxq_mempool_register(struct mlx5_rxq_ctrl *rxq_ctrl)
 	}
 	for (s = 0; s < rxq_ctrl->rxq.rxseg_n; s++) {
 		mp = rxq_ctrl->rxq.rxseg[s].mp;
-		ret = mlx5_mr_mempool_register(&priv->sh->cdev->mr_scache,
-					       priv->sh->cdev->pd, mp,
-					       &priv->mp_id);
+		ret = mlx5_mr_mempool_register(&sh->cdev->mr_scache,
+					       sh->cdev->pd, mp, &priv->mp_id);
 		if (ret < 0 && rte_errno != EEXIST)
 			return ret;
 		rte_mempool_mem_iter(mp, mlx5_rxq_mempool_register_cb,

@@ -213,7 +214,7 @@ mlx5_rxq_start(struct rte_eth_dev *dev)
 			 * the implicit registration is enabled or not,
 			 * Rx mempool destruction is tracked to free MRs.
 			 */
-			if (mlx5_rxq_mempool_register(rxq_ctrl) < 0)
+			if (mlx5_rxq_mempool_register(dev, rxq_ctrl) < 0)
 				goto error;
 			ret = rxq_alloc_elts(rxq_ctrl);
 			if (ret)

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:35 UTC #259 lore raw

PATCH v4 11/14 net/mlx5: move Rx queue DevX resource

In reply to Xueming Li

To support shared RX queue, moves DevX RQ which is per queue resource to Rx queue private data.

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/net/mlx5/linux/mlx5_verbs.c | 154 +++++++++++--------
 drivers/net/mlx5/mlx5.h             |  11 +-
 drivers/net/mlx5/mlx5_devx.c        | 227 +++++++++++++---------------
 drivers/net/mlx5/mlx5_rx.h          |   1 +
 drivers/net/mlx5/mlx5_rxq.c         |  44 +++---
 drivers/net/mlx5/mlx5_rxtx.c        |   6 +-
 drivers/net/mlx5/mlx5_trigger.c     |   2 +-
 drivers/net/mlx5/mlx5_vlan.c        |  16 +-
 8 files changed, 240 insertions(+), 221 deletions(-)

drivers/net/mlx5/linux/mlx5_verbs.c +90 ~~-64~~

index 4779b37aa65..5d4ae3ea752 100644
@@ -29,13 +29,13 @@
 /**
  * Modify Rx WQ vlan stripping offload
  *
- * @param rxq_obj
- *   Rx queue object.
+ * @param rxq
+ *   Rx queue.
  *
  * @return 0 on success, non-0 otherwise
  */
 static int
-mlx5_rxq_obj_modify_wq_vlan_strip(struct mlx5_rxq_obj *rxq_obj, int on)
+mlx5_rxq_obj_modify_wq_vlan_strip(struct mlx5_rxq_priv *rxq, int on)
 {
 	uint16_t vlan_offloads =
 		(on ? IBV_WQ_FLAGS_CVLAN_STRIPPING : 0) |

@@ -47,14 +47,14 @@ mlx5_rxq_obj_modify_wq_vlan_strip(struct mlx5_rxq_obj *rxq_obj, int on)
 		.flags = vlan_offloads,
 	};
 
-	return mlx5_glue->modify_wq(rxq_obj->wq, &mod);
+	return mlx5_glue->modify_wq(rxq->ctrl->obj->wq, &mod);
 }
 
 /**
  * Modifies the attributes for the specified WQ.
  *
- * @param rxq_obj
- *   Verbs Rx queue object.
+ * @param rxq
+ *   Verbs Rx queue.
  * @param type
  *   Type of change queue state.
  *

@@ -62,14 +62,14 @@ mlx5_rxq_obj_modify_wq_vlan_strip(struct mlx5_rxq_obj *rxq_obj, int on)
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-mlx5_ibv_modify_wq(struct mlx5_rxq_obj *rxq_obj, uint8_t type)
+mlx5_ibv_modify_wq(struct mlx5_rxq_priv *rxq, uint8_t type)
 {
 	struct ibv_wq_attr mod = {
 		.attr_mask = IBV_WQ_ATTR_STATE,
 		.wq_state = (enum ibv_wq_state)type,
 	};
 
-	return mlx5_glue->modify_wq(rxq_obj->wq, &mod);
+	return mlx5_glue->modify_wq(rxq->ctrl->obj->wq, &mod);
 }
 
 /**

@@ -139,21 +139,18 @@ mlx5_ibv_modify_qp(struct mlx5_txq_obj *obj, enum mlx5_txq_modify_type type,
 /**
  * Create a CQ Verbs object.
  *
- * @param dev
- *   Pointer to Ethernet device.
- * @param idx
- *   Queue index in DPDK Rx queue array.
+ * @param rxq
+ *   Pointer to Rx queue.
  *
  * @return
  *   The Verbs CQ object initialized, NULL otherwise and rte_errno is set.
  */
 static struct ibv_cq *
-mlx5_rxq_ibv_cq_create(struct rte_eth_dev *dev, uint16_t idx)
+mlx5_rxq_ibv_cq_create(struct mlx5_rxq_priv *rxq)
 {
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[idx];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_priv *priv = rxq->priv;
+	struct mlx5_rxq_ctrl *rxq_ctrl = rxq->ctrl;
+	struct mlx5_rxq_data *rxq_data = &rxq_ctrl->rxq;
 	struct mlx5_rxq_obj *rxq_obj = rxq_ctrl->obj;
 	unsigned int cqe_n = mlx5_rxq_cqe_num(rxq_data);
 	struct {

@@ -199,7 +196,7 @@ mlx5_rxq_ibv_cq_create(struct rte_eth_dev *dev, uint16_t idx)
 		DRV_LOG(DEBUG,
 			"Port %u Rx CQE compression is disabled for HW"
 			" timestamp.",
-			dev->data->port_id);
+			priv->dev_data->port_id);
 	}
 #ifdef HAVE_IBV_MLX5_MOD_CQE_128B_PAD
 	if (RTE_CACHE_LINE_SIZE == 128) {

@@ -216,21 +213,18 @@ mlx5_rxq_ibv_cq_create(struct rte_eth_dev *dev, uint16_t idx)
 /**
  * Create a WQ Verbs object.
  *
- * @param dev
- *   Pointer to Ethernet device.
- * @param idx
- *   Queue index in DPDK Rx queue array.
+ * @param rxq
+ *   Pointer to Rx queue.
  *
  * @return
  *   The Verbs WQ object initialized, NULL otherwise and rte_errno is set.
  */
 static struct ibv_wq *
-mlx5_rxq_ibv_wq_create(struct rte_eth_dev *dev, uint16_t idx)
+mlx5_rxq_ibv_wq_create(struct mlx5_rxq_priv *rxq)
 {
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[idx];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_priv *priv = rxq->priv;
+	struct mlx5_rxq_ctrl *rxq_ctrl = rxq->ctrl;
+	struct mlx5_rxq_data *rxq_data = &rxq_ctrl->rxq;
 	struct mlx5_rxq_obj *rxq_obj = rxq_ctrl->obj;
 	unsigned int wqe_n = 1 << rxq_data->elts_n;
 	struct {

@@ -297,7 +291,7 @@ mlx5_rxq_ibv_wq_create(struct rte_eth_dev *dev, uint16_t idx)
 			DRV_LOG(ERR,
 				"Port %u Rx queue %u requested %u*%u but got"
 				" %u*%u WRs*SGEs.",
-				dev->data->port_id, idx,
+				priv->dev_data->port_id, rxq->idx,
 				wqe_n >> rxq_data->sges_n,
 				(1 << rxq_data->sges_n),
 				wq_attr.ibv.max_wr, wq_attr.ibv.max_sge);

@@ -312,21 +306,20 @@ mlx5_rxq_ibv_wq_create(struct rte_eth_dev *dev, uint16_t idx)
 /**
  * Create the Rx queue Verbs object.
  *
- * @param dev
- *   Pointer to Ethernet device.
- * @param idx
- *   Queue index in DPDK Rx queue array.
+ * @param rxq
+ *   Pointer to Rx queue.
  *
  * @return
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
+mlx5_rxq_ibv_obj_new(struct mlx5_rxq_priv *rxq)
 {
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq_data = (*priv->rxqs)[idx];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
+	uint16_t idx = rxq->idx;
+	struct mlx5_priv *priv = rxq->priv;
+	uint16_t port_id = priv->dev_data->port_id;
+	struct mlx5_rxq_ctrl *rxq_ctrl = rxq->ctrl;
+	struct mlx5_rxq_data *rxq_data = &rxq_ctrl->rxq;
 	struct mlx5_rxq_obj *tmpl = rxq_ctrl->obj;
 	struct mlx5dv_cq cq_info;
 	struct mlx5dv_rwq rwq;

@@ -341,17 +334,17 @@ mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 			mlx5_glue->create_comp_channel(priv->sh->cdev->ctx);
 		if (!tmpl->ibv_channel) {
 			DRV_LOG(ERR, "Port %u: comp channel creation failure.",
-				dev->data->port_id);
+				port_id);
 			rte_errno = ENOMEM;
 			goto error;
 		}
 		tmpl->fd = ((struct ibv_comp_channel *)(tmpl->ibv_channel))->fd;
 	}
 	/* Create CQ using Verbs API. */
-	tmpl->ibv_cq = mlx5_rxq_ibv_cq_create(dev, idx);
+	tmpl->ibv_cq = mlx5_rxq_ibv_cq_create(rxq);
 	if (!tmpl->ibv_cq) {
 		DRV_LOG(ERR, "Port %u Rx queue %u CQ creation failure.",
-			dev->data->port_id, idx);
+			port_id, idx);
 		rte_errno = ENOMEM;
 		goto error;
 	}

@@ -366,7 +359,7 @@ mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 		DRV_LOG(ERR,
 			"Port %u wrong MLX5_CQE_SIZE environment "
 			"variable value: it should be set to %u.",
-			dev->data->port_id, RTE_CACHE_LINE_SIZE);
+			port_id, RTE_CACHE_LINE_SIZE);
 		rte_errno = EINVAL;
 		goto error;
 	}

@@ -377,19 +370,19 @@ mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	rxq_data->cq_uar = cq_info.cq_uar;
 	rxq_data->cqn = cq_info.cqn;
 	/* Create WQ (RQ) using Verbs API. */
-	tmpl->wq = mlx5_rxq_ibv_wq_create(dev, idx);
+	tmpl->wq = mlx5_rxq_ibv_wq_create(rxq);
 	if (!tmpl->wq) {
 		DRV_LOG(ERR, "Port %u Rx queue %u WQ creation failure.",
-			dev->data->port_id, idx);
+			port_id, idx);
 		rte_errno = ENOMEM;
 		goto error;
 	}
 	/* Change queue state to ready. */
-	ret = mlx5_ibv_modify_wq(tmpl, IBV_WQS_RDY);
+	ret = mlx5_ibv_modify_wq(rxq, IBV_WQS_RDY);
 	if (ret) {
 		DRV_LOG(ERR,
 			"Port %u Rx queue %u WQ state to IBV_WQS_RDY failed.",
-			dev->data->port_id, idx);
+			port_id, idx);
 		rte_errno = ret;
 		goto error;
 	}

@@ -405,7 +398,7 @@ mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	rxq_data->cq_arm_sn = 0;
 	mlx5_rxq_initialize(rxq_data);
 	rxq_data->cq_ci = 0;
-	dev->data->rx_queue_state[idx] = RTE_ETH_QUEUE_STATE_STARTED;
+	priv->dev_data->rx_queue_state[idx] = RTE_ETH_QUEUE_STATE_STARTED;
 	rxq_ctrl->wqn = ((struct ibv_wq *)(tmpl->wq))->wq_num;
 	return 0;
 error:

@@ -423,12 +416,14 @@ mlx5_rxq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 /**
  * Release an Rx verbs queue object.
  *
- * @param rxq_obj
- *   Verbs Rx queue object.
+ * @param rxq
+ *   Pointer to Rx queue.
  */
 static void
-mlx5_rxq_ibv_obj_release(struct mlx5_rxq_obj *rxq_obj)
+mlx5_rxq_ibv_obj_release(struct mlx5_rxq_priv *rxq)
 {
+	struct mlx5_rxq_obj *rxq_obj = rxq->ctrl->obj;
+
 	MLX5_ASSERT(rxq_obj);
 	MLX5_ASSERT(rxq_obj->wq);
 	MLX5_ASSERT(rxq_obj->ibv_cq);

@@ -652,12 +647,24 @@ static void
 mlx5_rxq_ibv_obj_drop_release(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_obj *rxq = priv->drop_queue.rxq;
+	struct mlx5_rxq_priv *rxq = priv->drop_queue.rxq;
+	struct mlx5_rxq_obj *rxq_obj;
 
-	if (rxq->wq)
-		claim_zero(mlx5_glue->destroy_wq(rxq->wq));
-	if (rxq->ibv_cq)
-		claim_zero(mlx5_glue->destroy_cq(rxq->ibv_cq));
+	if (rxq == NULL)
+		return;
+	if (rxq->ctrl == NULL)
+		goto free_priv;
+	rxq_obj = rxq->ctrl->obj;
+	if (rxq_obj == NULL)
+		goto free_ctrl;
+	if (rxq_obj->wq)
+		claim_zero(mlx5_glue->destroy_wq(rxq_obj->wq));
+	if (rxq_obj->ibv_cq)
+		claim_zero(mlx5_glue->destroy_cq(rxq_obj->ibv_cq));
+	mlx5_free(rxq_obj);
+free_ctrl:
+	mlx5_free(rxq->ctrl);
+free_priv:
 	mlx5_free(rxq);
 	priv->drop_queue.rxq = NULL;
 }

@@ -676,39 +683,58 @@ mlx5_rxq_ibv_obj_drop_create(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
 	struct ibv_context *ctx = priv->sh->cdev->ctx;
-	struct mlx5_rxq_obj *rxq = priv->drop_queue.rxq;
+	struct mlx5_rxq_priv *rxq = priv->drop_queue.rxq;
+	struct mlx5_rxq_ctrl *rxq_ctrl = NULL;
+	struct mlx5_rxq_obj *rxq_obj = NULL;
 
-	if (rxq)
+	if (rxq != NULL)
 		return 0;
 	rxq = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*rxq), 0, SOCKET_ID_ANY);
-	if (!rxq) {
+	if (rxq == NULL) {
 		DRV_LOG(DEBUG, "Port %u cannot allocate drop Rx queue memory.",
 		      dev->data->port_id);
 		rte_errno = ENOMEM;
 		return -rte_errno;
 	}
 	priv->drop_queue.rxq = rxq;
-	rxq->ibv_cq = mlx5_glue->create_cq(ctx, 1, NULL, NULL, 0);
-	if (!rxq->ibv_cq) {
+	rxq_ctrl = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*rxq_ctrl), 0,
+			       SOCKET_ID_ANY);
+	if (rxq_ctrl == NULL) {
+		DRV_LOG(DEBUG, "Port %u cannot allocate drop Rx queue control memory.",
+		      dev->data->port_id);
+		rte_errno = ENOMEM;
+		goto error;
+	}
+	rxq->ctrl = rxq_ctrl;
+	rxq_obj = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*rxq_obj), 0,
+			      SOCKET_ID_ANY);
+	if (rxq_obj == NULL) {
+		DRV_LOG(DEBUG, "Port %u cannot allocate drop Rx queue memory.",
+		      dev->data->port_id);
+		rte_errno = ENOMEM;
+		goto error;
+	}
+	rxq_ctrl->obj = rxq_obj;
+	rxq_obj->ibv_cq = mlx5_glue->create_cq(ctx, 1, NULL, NULL, 0);
+	if (!rxq_obj->ibv_cq) {
 		DRV_LOG(DEBUG, "Port %u cannot allocate CQ for drop queue.",
 		      dev->data->port_id);
 		rte_errno = errno;
 		goto error;
 	}
-	rxq->wq = mlx5_glue->create_wq(ctx, &(struct ibv_wq_init_attr){
+	rxq_obj->wq = mlx5_glue->create_wq(ctx, &(struct ibv_wq_init_attr){
 						    .wq_type = IBV_WQT_RQ,
 						    .max_wr = 1,
 						    .max_sge = 1,
 						    .pd = priv->sh->cdev->pd,
-						    .cq = rxq->ibv_cq,
+						    .cq = rxq_obj->ibv_cq,
 					      });
-	if (!rxq->wq) {
+	if (!rxq_obj->wq) {
 		DRV_LOG(DEBUG, "Port %u cannot allocate WQ for drop queue.",
 		      dev->data->port_id);
 		rte_errno = errno;
 		goto error;
 	}
-	priv->drop_queue.rxq = rxq;
 	return 0;
 error:
 	mlx5_rxq_ibv_obj_drop_release(dev);

@@ -737,7 +763,7 @@ mlx5_ibv_drop_action_create(struct rte_eth_dev *dev)
 	ret = mlx5_rxq_ibv_obj_drop_create(dev);
 	if (ret < 0)
 		goto error;
-	rxq = priv->drop_queue.rxq;
+	rxq = priv->drop_queue.rxq->ctrl->obj;
 	ind_tbl = mlx5_glue->create_rwq_ind_table
 				(priv->sh->cdev->ctx,
 				 &(struct ibv_rwq_ind_table_init_attr){

drivers/net/mlx5/mlx5.h +5 -6

index 4e99fe7d068..967d92b4ad6 100644
@@ -300,7 +300,7 @@ struct mlx5_vf_vlan {
 /* Flow drop context necessary due to Verbs API. */
 struct mlx5_drop {
 	struct mlx5_hrxq *hrxq; /* Hash Rx queue queue. */
-	struct mlx5_rxq_obj *rxq; /* Rx queue object. */
+	struct mlx5_rxq_priv *rxq; /* Rx queue. */
 };
 
 /* Loopback dummy queue resources required due to Verbs API. */

@@ -1267,7 +1267,6 @@ struct mlx5_rxq_obj {
 		};
 		struct mlx5_devx_obj *rq; /* DevX RQ object for hairpin. */
 		struct {
-			struct mlx5_devx_rq rq_obj; /* DevX RQ object. */
 			struct mlx5_devx_cq cq_obj; /* DevX CQ object. */
 			void *devx_channel;
 		};

@@ -1349,11 +1348,11 @@ struct mlx5_rxq_priv;
 
 /* HW objects operations structure. */
 struct mlx5_obj_ops {
-	int (*rxq_obj_modify_vlan_strip)(struct mlx5_rxq_obj *rxq_obj, int on);
-	int (*rxq_obj_new)(struct rte_eth_dev *dev, uint16_t idx);
+	int (*rxq_obj_modify_vlan_strip)(struct mlx5_rxq_priv *rxq, int on);
+	int (*rxq_obj_new)(struct mlx5_rxq_priv *rxq);
 	int (*rxq_event_get)(struct mlx5_rxq_obj *rxq_obj);
-	int (*rxq_obj_modify)(struct mlx5_rxq_obj *rxq_obj, uint8_t type);
-	void (*rxq_obj_release)(struct mlx5_rxq_obj *rxq_obj);
+	int (*rxq_obj_modify)(struct mlx5_rxq_priv *rxq, uint8_t type);
+	void (*rxq_obj_release)(struct mlx5_rxq_priv *rxq);
 	int (*ind_table_new)(struct rte_eth_dev *dev, const unsigned int log_n,
 			     struct mlx5_ind_table_obj *ind_tbl);
 	int (*ind_table_modify)(struct rte_eth_dev *dev,

drivers/net/mlx5/mlx5_devx.c +109 ~~-118~~

467 lines not on this page · raw message

drivers/net/mlx5/mlx5_rx.h +1 -0

index c04c0c73349..337dcca59fb 100644
@@ -174,6 +174,7 @@ struct mlx5_rxq_priv {
 	struct mlx5_rxq_ctrl *ctrl; /* Shared Rx Queue. */
 	LIST_ENTRY(mlx5_rxq_priv) owner_entry; /* Entry in shared rxq_ctrl. */
 	struct mlx5_priv *priv; /* Back pointer to private data. */
+	struct mlx5_devx_rq devx_rq;
 	struct rte_eth_hairpin_conf hairpin_conf; /* Hairpin configuration. */
 	uint32_t hairpin_status; /* Hairpin binding status. */
 };

drivers/net/mlx5/mlx5_rxq.c +24 ~~-20~~

93 lines not on this page · raw message

drivers/net/mlx5/mlx5_rxtx.c +2 -4

index 0bcdff1b116..54d410b513b 100644
@@ -373,11 +373,9 @@ mlx5_queue_state_modify_primary(struct rte_eth_dev *dev,
 	struct mlx5_priv *priv = dev->data->dev_private;
 
 	if (sm->is_wq) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[sm->queue_id];
-		struct mlx5_rxq_ctrl *rxq_ctrl =
-			container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+		struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, sm->queue_id);
 
-		ret = priv->obj_ops.rxq_obj_modify(rxq_ctrl->obj, sm->state);
+		ret = priv->obj_ops.rxq_obj_modify(rxq, sm->state);
 		if (ret) {
 			DRV_LOG(ERR, "Cannot change Rx WQ state to %u  - %s",
 					sm->state, strerror(errno));

drivers/net/mlx5/mlx5_trigger.c +1 -1

index caafdf27e8f..2cf62a9780d 100644
@@ -231,7 +231,7 @@ mlx5_rxq_start(struct rte_eth_dev *dev)
 			rte_errno = ENOMEM;
 			goto error;
 		}
-		ret = priv->obj_ops.rxq_obj_new(dev, i);
+		ret = priv->obj_ops.rxq_obj_new(rxq);
 		if (ret) {
 			mlx5_free(rxq_ctrl->obj);
 			rxq_ctrl->obj = NULL;

drivers/net/mlx5/mlx5_vlan.c +8 -8

index 07792fc5d94..ea841bb32fb 100644
@@ -91,11 +91,11 @@ void
 mlx5_vlan_strip_queue_set(struct rte_eth_dev *dev, uint16_t queue, int on)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq = (*priv->rxqs)[queue];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, queue);
+	struct mlx5_rxq_data *rxq_data = &rxq->ctrl->rxq;
 	int ret = 0;
 
+	MLX5_ASSERT(rxq != NULL && rxq->ctrl != NULL);
 	/* Validate hw support */
 	if (!priv->config.hw_vlan_strip) {
 		DRV_LOG(ERR, "port %u VLAN stripping is not supported",

@@ -109,20 +109,20 @@ mlx5_vlan_strip_queue_set(struct rte_eth_dev *dev, uint16_t queue, int on)
 		return;
 	}
 	DRV_LOG(DEBUG, "port %u set VLAN stripping offloads %d for port %uqueue %d",
-		dev->data->port_id, on, rxq->port_id, queue);
-	if (!rxq_ctrl->obj) {
+		dev->data->port_id, on, rxq_data->port_id, queue);
+	if (rxq->ctrl->obj == NULL) {
 		/* Update related bits in RX queue. */
-		rxq->vlan_strip = !!on;
+		rxq_data->vlan_strip = !!on;
 		return;
 	}
-	ret = priv->obj_ops.rxq_obj_modify_vlan_strip(rxq_ctrl->obj, on);
+	ret = priv->obj_ops.rxq_obj_modify_vlan_strip(rxq, on);
 	if (ret) {
 		DRV_LOG(ERR, "Port %u failed to modify object stripping mode:"
 			" %s", dev->data->port_id, strerror(rte_errno));
 		return;
 	}
 	/* Update related bits in RX queue. */
-	rxq->vlan_strip = !!on;
+	rxq_data->vlan_strip = !!on;
 }
 
 /**

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:35 UTC #260 lore raw

PATCH v4 12/14 net/mlx5: remove Rx queue data list from device

In reply to Xueming Li

Rx queue data list(priv->rxqs) can be replaced by Rx queue list(priv->rxq_privs), removes it and replaces with universal wrapper API.

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/net/mlx5/linux/mlx5_verbs.c |  7 ++---
 drivers/net/mlx5/mlx5.c             | 10 +-----
 drivers/net/mlx5/mlx5.h             |  1 -
 drivers/net/mlx5/mlx5_devx.c        | 12 +++++---
 drivers/net/mlx5/mlx5_ethdev.c      |  6 +---
 drivers/net/mlx5/mlx5_flow.c        | 47 +++++++++++++++--------------
 drivers/net/mlx5/mlx5_rss.c         |  6 ++--
 drivers/net/mlx5/mlx5_rx.c          | 15 +++++----
 drivers/net/mlx5/mlx5_rx.h          |  9 +++---
 drivers/net/mlx5/mlx5_rxq.c         | 43 ++++++++++++--------------
 drivers/net/mlx5/mlx5_rxtx_vec.c    |  6 ++--
 drivers/net/mlx5/mlx5_stats.c       |  9 +++---
 drivers/net/mlx5/mlx5_trigger.c     |  2 +-
 13 files changed, 79 insertions(+), 94 deletions(-)

drivers/net/mlx5/linux/mlx5_verbs.c +3 -4

index 5d4ae3ea752..f78916c868f 100644
@@ -486,11 +486,10 @@ mlx5_ibv_ind_table_new(struct rte_eth_dev *dev, const unsigned int log_n,
 
 	MLX5_ASSERT(ind_tbl);
 	for (i = 0; i != ind_tbl->queues_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[ind_tbl->queues[i]];
-		struct mlx5_rxq_ctrl *rxq_ctrl =
-				container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+		struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev,
+							 ind_tbl->queues[i]);
 
-		wq[i] = rxq_ctrl->obj->wq;
+		wq[i] = rxq->ctrl->obj->wq;
 	}
 	MLX5_ASSERT(i > 0);
 	/* Finalise indirection table. */

drivers/net/mlx5/mlx5.c +1 -9

index 374cc9757aa..8614b8ffddd 100644
@@ -1687,20 +1687,12 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	/* Free the eCPRI flex parser resource. */
 	mlx5_flex_parser_ecpri_release(dev);
 	mlx5_flex_item_port_cleanup(dev);
-	if (priv->rxqs != NULL) {
+	if (priv->rxq_privs != NULL) {
 		/* XXX race condition if mlx5_rx_burst() is still running. */
 		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->rxqs_n); ++i)
 			mlx5_rxq_release(dev, i);
 		priv->rxqs_n = 0;
-		priv->rxqs = NULL;
-	}
-	if (priv->representor) {
-		/* Each representor has a dedicated interrupts handler */
-		mlx5_free(dev->intr_handle);
-		dev->intr_handle = NULL;
-	}
-	if (priv->rxq_privs != NULL) {
 		mlx5_free(priv->rxq_privs);
 		priv->rxq_privs = NULL;
 	}

drivers/net/mlx5/mlx5.h +0 -1

index 967d92b4ad6..a037a33debf 100644
@@ -1410,7 +1410,6 @@ struct mlx5_priv {
 	unsigned int rxqs_n; /* RX queues array size. */
 	unsigned int txqs_n; /* TX queues array size. */
 	struct mlx5_rxq_priv *(*rxq_privs)[]; /* RX queue non-shared data. */
-	struct mlx5_rxq_data *(*rxqs)[]; /* (Shared) RX queues. */
 	struct mlx5_txq_data *(*txqs)[]; /* TX queues. */
 	struct rte_mempool *mprq_mp; /* Mempool for Multi-Packet RQ. */
 	struct rte_eth_rss_conf rss_conf; /* RSS configuration. */

drivers/net/mlx5/mlx5_devx.c +7 -5

index b90a5d82458..668d47025e8 100644
@@ -684,15 +684,17 @@ mlx5_devx_tir_attr_set(struct rte_eth_dev *dev, const uint8_t *rss_key,
 
 	/* NULL queues designate drop queue. */
 	if (ind_tbl->queues != NULL) {
-		struct mlx5_rxq_data *rxq_data =
-					(*priv->rxqs)[ind_tbl->queues[0]];
 		struct mlx5_rxq_ctrl *rxq_ctrl =
-			container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
-		rxq_obj_type = rxq_ctrl->type;
+				mlx5_rxq_ctrl_get(dev, ind_tbl->queues[0]);
+		rxq_obj_type = rxq_ctrl != NULL ? rxq_ctrl->type :
+						  MLX5_RXQ_TYPE_STANDARD;
 
 		/* Enable TIR LRO only if all the queues were configured for. */
 		for (i = 0; i < ind_tbl->queues_n; ++i) {
-			if (!(*priv->rxqs)[ind_tbl->queues[i]]->lro) {
+			struct mlx5_rxq_data *rxq_i =
+				mlx5_rxq_data_get(dev, ind_tbl->queues[i]);
+
+			if (rxq_i != NULL && !rxq_i->lro) {
 				lro = false;
 				break;
 			}

drivers/net/mlx5/mlx5_ethdev.c +1 -5

index cde505955df..bb38d5d2ade 100644
@@ -114,7 +114,6 @@ mlx5_dev_configure(struct rte_eth_dev *dev)
 		rte_errno = ENOMEM;
 		return -rte_errno;
 	}
-	priv->rxqs = (void *)dev->data->rx_queues;
 	priv->txqs = (void *)dev->data->tx_queues;
 	if (txqs_n != priv->txqs_n) {
 		DRV_LOG(INFO, "port %u Tx queues number update: %u -> %u",

@@ -171,11 +170,8 @@ mlx5_dev_configure_rss_reta(struct rte_eth_dev *dev)
 		return -rte_errno;
 	}
 	for (i = 0, j = 0; i < rxqs_n; i++) {
-		struct mlx5_rxq_data *rxq_data;
-		struct mlx5_rxq_ctrl *rxq_ctrl;
+		struct mlx5_rxq_ctrl *rxq_ctrl = mlx5_rxq_ctrl_get(dev, i);
 
-		rxq_data = (*priv->rxqs)[i];
-		rxq_ctrl = container_of(rxq_data, struct mlx5_rxq_ctrl, rxq);
 		if (rxq_ctrl && rxq_ctrl->type == MLX5_RXQ_TYPE_STANDARD)
 			rss_queue_arr[j++] = i;
 	}

drivers/net/mlx5/mlx5_flow.c +24 ~~-23~~

109 lines not on this page · raw message

drivers/net/mlx5/mlx5_rss.c +4 -2

index a04e22398db..75af05b7b02 100644
@@ -65,9 +65,11 @@ mlx5_rss_hash_update(struct rte_eth_dev *dev,
 	priv->rss_conf.rss_hf = rss_conf->rss_hf;
 	/* Enable the RSS hash in all Rx queues. */
 	for (i = 0, idx = 0; idx != priv->rxqs_n; ++i) {
-		if (!(*priv->rxqs)[i])
+		struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, i);
+
+		if (rxq == NULL || rxq->ctrl == NULL)
 			continue;
-		(*priv->rxqs)[i]->rss_hash = !!rss_conf->rss_hf &&
+		rxq->ctrl->rxq.rss_hash = !!rss_conf->rss_hf &&
 			!!(dev->data->dev_conf.rxmode.mq_mode & RTE_ETH_MQ_RX_RSS);
 		++idx;
 	}

drivers/net/mlx5/mlx5_rx.c +7 -8

index d41905a2a04..1ffa1b95b88 100644
@@ -148,10 +148,8 @@ void
 mlx5_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 		  struct rte_eth_rxq_info *qinfo)
 {
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq = (*priv->rxqs)[rx_queue_id];
-	struct mlx5_rxq_ctrl *rxq_ctrl =
-		container_of(rxq, struct mlx5_rxq_ctrl, rxq);
+	struct mlx5_rxq_ctrl *rxq_ctrl = mlx5_rxq_ctrl_get(dev, rx_queue_id);
+	struct mlx5_rxq_data *rxq = mlx5_rxq_data_get(dev, rx_queue_id);
 
 	if (!rxq)
 		return;

@@ -162,7 +160,10 @@ mlx5_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 	qinfo->conf.rx_thresh.wthresh = 0;
 	qinfo->conf.rx_free_thresh = rxq->rq_repl_thresh;
 	qinfo->conf.rx_drop_en = 1;
-	qinfo->conf.rx_deferred_start = rxq_ctrl ? 0 : 1;
+	if (rxq_ctrl == NULL || rxq_ctrl->obj == NULL)
+		qinfo->conf.rx_deferred_start = 0;
+	else
+		qinfo->conf.rx_deferred_start = 1;
 	qinfo->conf.offloads = dev->data->dev_conf.rxmode.offloads;
 	qinfo->scattered_rx = dev->data->scattered_rx;
 	qinfo->nb_desc = mlx5_rxq_mprq_enabled(rxq) ?

@@ -191,10 +192,8 @@ mlx5_rx_burst_mode_get(struct rte_eth_dev *dev,
 		       struct rte_eth_burst_mode *mode)
 {
 	eth_rx_burst_t pkt_burst = dev->rx_pkt_burst;
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_rxq_data *rxq;
+	struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, rx_queue_id);
 
-	rxq = (*priv->rxqs)[rx_queue_id];
 	if (!rxq) {
 		rte_errno = EINVAL;
 		return -rte_errno;

drivers/net/mlx5/mlx5_rx.h +4 -5

index 337dcca59fb..413e36f6d8d 100644
@@ -603,14 +603,13 @@ mlx5_mprq_enabled(struct rte_eth_dev *dev)
 		return 0;
 	/* All the configured queues should be enabled. */
 	for (i = 0; i < priv->rxqs_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
-		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
-			(rxq, struct mlx5_rxq_ctrl, rxq);
+		struct mlx5_rxq_ctrl *rxq_ctrl = mlx5_rxq_ctrl_get(dev, i);
 
-		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
+		if (rxq_ctrl == NULL ||
+		    rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
 			continue;
 		n_ibv++;
-		if (mlx5_rxq_mprq_enabled(rxq))
+		if (mlx5_rxq_mprq_enabled(&rxq_ctrl->rxq))
 			++n;
 	}
 	/* Multi-Packet RQ can't be partially configured. */

drivers/net/mlx5/mlx5_rxq.c +19 ~~-24~~

107 lines not on this page · raw message

drivers/net/mlx5/mlx5_rxtx_vec.c +3 -3

index 511681841ca..6212ce8247d 100644
@@ -578,11 +578,11 @@ mlx5_check_vec_rx_support(struct rte_eth_dev *dev)
 		return -ENOTSUP;
 	/* All the configured queues should support. */
 	for (i = 0; i < priv->rxqs_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
+		struct mlx5_rxq_data *rxq_data = mlx5_rxq_data_get(dev, i);
 
-		if (!rxq)
+		if (!rxq_data)
 			continue;
-		if (mlx5_rxq_check_vec_support(rxq) < 0)
+		if (mlx5_rxq_check_vec_support(rxq_data) < 0)
 			break;
 	}
 	if (i != priv->rxqs_n)

drivers/net/mlx5/mlx5_stats.c +5 -4

index ae2f5668a74..732775954ad 100644
@@ -107,7 +107,7 @@ mlx5_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
 	memset(&tmp, 0, sizeof(tmp));
 	/* Add software counters. */
 	for (i = 0; (i != priv->rxqs_n); ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
+		struct mlx5_rxq_data *rxq = mlx5_rxq_data_get(dev, i);
 
 		if (rxq == NULL)
 			continue;

@@ -181,10 +181,11 @@ mlx5_stats_reset(struct rte_eth_dev *dev)
 	unsigned int i;
 
 	for (i = 0; (i != priv->rxqs_n); ++i) {
-		if ((*priv->rxqs)[i] == NULL)
+		struct mlx5_rxq_data *rxq_data = mlx5_rxq_data_get(dev, i);
+
+		if (rxq_data == NULL)
 			continue;
-		memset(&(*priv->rxqs)[i]->stats, 0,
-		       sizeof(struct mlx5_rxq_stats));
+		memset(&rxq_data->stats, 0, sizeof(struct mlx5_rxq_stats));
 	}
 	for (i = 0; (i != priv->txqs_n); ++i) {
 		if ((*priv->txqs)[i] == NULL)

drivers/net/mlx5/mlx5_trigger.c +1 -1

index 2cf62a9780d..72475e4b5b5 100644
@@ -227,7 +227,7 @@ mlx5_rxq_start(struct rte_eth_dev *dev)
 		if (!rxq_ctrl->obj) {
 			DRV_LOG(ERR,
 				"Port %u Rx queue %u can't allocate resources.",
-				dev->data->port_id, (*priv->rxqs)[i]->idx);
+				dev->data->port_id, i);
 			rte_errno = ENOMEM;
 			goto error;
 		}

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:36 UTC #261 lore raw

PATCH v4 13/14 net/mlx5: support shared Rx queue

In reply to Xueming Li

Signed-off-by Xueming Li <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 doc/guides/nics/features/mlx5.ini   |   1 +
 doc/guides/nics/mlx5.rst            |   6 +
 drivers/net/mlx5/linux/mlx5_os.c    |   2 +
 drivers/net/mlx5/linux/mlx5_verbs.c |   8 +-
 drivers/net/mlx5/mlx5.h             |   2 +
 drivers/net/mlx5/mlx5_devx.c        |  46 +++--
 drivers/net/mlx5/mlx5_ethdev.c      |   5 +
 drivers/net/mlx5/mlx5_rx.h          |   3 +
 drivers/net/mlx5/mlx5_rxq.c         | 273 ++++++++++++++++++++++++----
 drivers/net/mlx5/mlx5_trigger.c     |  61 ++++---
 10 files changed, 329 insertions(+), 78 deletions(-)

doc/guides/nics/features/mlx5.ini +1 -0

index 403f58cd7e2..7cbd11bb160 100644
@@ -11,6 +11,7 @@ Removal event        = Y
 Rx interrupt         = Y
 Fast mbuf free       = Y
 Queue start/stop     = Y
+Shared Rx queue      = Y
 Burst mode info      = Y
 Power mgmt address monitor = Y
 MTU update           = Y

doc/guides/nics/mlx5.rst +6 -0

index bb92520dff4..824971d89ae 100644
@@ -113,6 +113,7 @@ Features
 - Connection tracking.
 - Sub-Function representors.
 - Sub-Function.
+- Shared Rx queue.
 
 
 Limitations

@@ -465,6 +466,11 @@ Limitations
   - In order to achieve best insertion rate, application should manage the flows per lcore.
   - Better to disable memory reclaim by setting ``reclaim_mem_mode`` to 0 to accelerate the flow object allocation and release with cache.
 
+ Shared Rx queue:
+
+  - Counters of received packets and bytes number of devices in same share group are same.
+  - Counters of received packets and bytes number of queues in same group and queue ID are same.
+
 - HW hashed bonding
 
   - TXQ affinity subjects to HW hash once enabled.

drivers/net/mlx5/linux/mlx5_os.c +2 -0

index f51da8c3a38..e0304b685e5 100644
@@ -420,6 +420,7 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv)
 			mlx5_glue->dr_create_flow_action_default_miss();
 	if (!sh->default_miss_action)
 		DRV_LOG(WARNING, "Default miss action is not supported.");
+	LIST_INIT(&sh->shared_rxqs);
 	return 0;
 error:
 	/* Rollback the created objects. */

@@ -494,6 +495,7 @@ mlx5_os_free_shared_dr(struct mlx5_priv *priv)
 	MLX5_ASSERT(sh && sh->refcnt);
 	if (sh->refcnt > 1)
 		return;
+	MLX5_ASSERT(LIST_EMPTY(&sh->shared_rxqs));
 #ifdef HAVE_MLX5DV_DR
 	if (sh->rx_domain) {
 		mlx5_glue->dr_destroy_domain(sh->rx_domain);

drivers/net/mlx5/linux/mlx5_verbs.c +5 -3

index f78916c868f..9d299542614 100644
@@ -424,14 +424,16 @@ mlx5_rxq_ibv_obj_release(struct mlx5_rxq_priv *rxq)
 {
 	struct mlx5_rxq_obj *rxq_obj = rxq->ctrl->obj;
 
-	MLX5_ASSERT(rxq_obj);
-	MLX5_ASSERT(rxq_obj->wq);
-	MLX5_ASSERT(rxq_obj->ibv_cq);
+	if (rxq_obj == NULL || rxq_obj->wq == NULL)
+		return;
 	claim_zero(mlx5_glue->destroy_wq(rxq_obj->wq));
+	rxq_obj->wq = NULL;
+	MLX5_ASSERT(rxq_obj->ibv_cq);
 	claim_zero(mlx5_glue->destroy_cq(rxq_obj->ibv_cq));
 	if (rxq_obj->ibv_channel)
 		claim_zero(mlx5_glue->destroy_comp_channel
 							(rxq_obj->ibv_channel));
+	rxq->ctrl->started = false;
 }
 
 /**

drivers/net/mlx5/mlx5.h +2 -0

index a037a33debf..51f45788381 100644
@@ -1200,6 +1200,7 @@ struct mlx5_dev_ctx_shared {
 	struct mlx5_ecpri_parser_profile ecpri_parser;
 	/* Flex parser profiles information. */
 	void *devx_rx_uar; /* DevX UAR for Rx. */
+	LIST_HEAD(shared_rxqs, mlx5_rxq_ctrl) shared_rxqs; /* Shared RXQs. */
 	struct mlx5_aso_age_mng *aso_age_mng;
 	/* Management data for aging mechanism using ASO Flow Hit. */
 	struct mlx5_geneve_tlv_option_resource *geneve_tlv_option_resource;

@@ -1267,6 +1268,7 @@ struct mlx5_rxq_obj {
 		};
 		struct mlx5_devx_obj *rq; /* DevX RQ object for hairpin. */
 		struct {
+			struct mlx5_devx_rmp devx_rmp; /* RMP for shared RQ. */
 			struct mlx5_devx_cq cq_obj; /* DevX CQ object. */
 			void *devx_channel;
 		};

drivers/net/mlx5/mlx5_devx.c +34 ~~-12~~

107 lines not on this page · raw message

drivers/net/mlx5/mlx5_ethdev.c +5 -0

index bb38d5d2ade..dc647d5580c 100644
@@ -26,6 +26,7 @@
 #include "mlx5_rx.h"
 #include "mlx5_tx.h"
 #include "mlx5_autoconf.h"
+#include "mlx5_devx.h"
 
 /**
  * Get the interface index from device name.

@@ -336,9 +337,13 @@ mlx5_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *info)
 	info->flow_type_rss_offloads = ~MLX5_RSS_HF_MASK;
 	mlx5_set_default_params(dev, info);
 	mlx5_set_txlimit_params(dev, info);
+	if (priv->config.hca_attr.mem_rq_rmp &&
+	    priv->obj_ops.rxq_obj_new == devx_obj_ops.rxq_obj_new)
+		info->dev_capa |= RTE_ETH_DEV_CAPA_RXQ_SHARE;
 	info->switch_info.name = dev->data->name;
 	info->switch_info.domain_id = priv->domain_id;
 	info->switch_info.port_id = priv->representor_id;
+	info->switch_info.rx_domain = 0; /* No sub Rx domains. */
 	if (priv->representor) {
 		uint16_t port_id;

drivers/net/mlx5/mlx5_rx.h +3 -0

index 413e36f6d8d..eda6eca8dea 100644
@@ -96,6 +96,7 @@ struct mlx5_rxq_data {
 	unsigned int lro:1; /* Enable LRO. */
 	unsigned int dynf_meta:1; /* Dynamic metadata is configured. */
 	unsigned int mcqe_format:3; /* CQE compression format. */
+	unsigned int shared:1; /* Shared RXQ. */
 	volatile uint32_t *rq_db;
 	volatile uint32_t *cq_db;
 	uint16_t port_id;

@@ -158,8 +159,10 @@ struct mlx5_rxq_ctrl {
 	struct mlx5_dev_ctx_shared *sh; /* Shared context. */
 	enum mlx5_rxq_type type; /* Rxq type. */
 	unsigned int socket; /* CPU socket ID for allocations. */
+	LIST_ENTRY(mlx5_rxq_ctrl) share_entry; /* Entry in shared RXQ list. */
 	uint32_t share_group; /* Group ID of shared RXQ. */
 	uint16_t share_qid; /* Shared RxQ ID in group. */
+	unsigned int started:1; /* Whether (shared) RXQ has been started. */
 	unsigned int irq:1; /* Whether IRQ is enabled. */
 	uint32_t flow_mark_n; /* Number of Mark/Flag flows using this Queue. */
 	uint32_t flow_tunnels_n[MLX5_FLOW_TUNNEL]; /* Tunnels counters. */

drivers/net/mlx5/mlx5_rxq.c +234 ~~-39~~

372 lines not on this page · raw message

drivers/net/mlx5/mlx5_trigger.c +37 ~~-24~~

80 lines not on this page · raw message

Signature · 1 line

2.33.0

Xueming Li

author

2021-11-04 12:36 UTC #262 lore raw

PATCH v4 14/14 net/mlx5: add shared Rx queue port datapath support

In reply to Xueming Li

From: Viacheslav Ovsiienko <[email protected]>

When receive packet, mlx5 PMD saves mbuf port number from RxQ data.

To support shared RxQ, save port number into RQ context as user index. Received packet resolve port number from CQE user index which derived from RQ context.

Legacy Verbs API doesn't support RQ user index setting, still read from RxQ port number.

Signed-off-by Xueming Li <[email protected]>

Signed-off-by Viacheslav Ovsiienko <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
 drivers/net/mlx5/mlx5_devx.c             |  1 +
 drivers/net/mlx5/mlx5_rx.c               |  1 +
 drivers/net/mlx5/mlx5_rxq.c              |  3 ++-
 drivers/net/mlx5/mlx5_rxtx_vec_altivec.h |  6 ++++++
 drivers/net/mlx5/mlx5_rxtx_vec_neon.h    | 12 +++++++++++-
 drivers/net/mlx5/mlx5_rxtx_vec_sse.h     |  8 +++++++-
 6 files changed, 28 insertions(+), 3 deletions(-)

drivers/net/mlx5/mlx5_devx.c +1 -0

index d3d189ab7f2..a9f9f4af700 100644
@@ -277,6 +277,7 @@ mlx5_rxq_create_devx_rq_resources(struct mlx5_rxq_priv *rxq)
 						MLX5_WQ_END_PAD_MODE_NONE;
 	rq_attr.wq_attr.pd = cdev->pdn;
 	rq_attr.counter_set_id = priv->counter_set_id;
+	rq_attr.user_index = rte_cpu_to_be_16(priv->dev_data->port_id);
 	if (rxq_data->shared) /* Create RMP based RQ. */
 		rxq->devx_rq.rmp = &rxq_ctrl->obj->devx_rmp;
 	/* Create RQ using DevX API. */

drivers/net/mlx5/mlx5_rx.c +1 -0

index 1ffa1b95b88..4d85f64accd 100644
@@ -709,6 +709,7 @@ rxq_cq_to_mbuf(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt,
 {
 	/* Update packet information. */
 	pkt->packet_type = rxq_cq_to_pkt_type(rxq, cqe, mcqe);
+	pkt->port = unlikely(rxq->shared) ? cqe->user_index_low : rxq->port_id;
 
 	if (rxq->rss_hash) {
 		uint32_t rss_hash_res = 0;

drivers/net/mlx5/mlx5_rxq.c +2 -1

index 8feb3e2c0fb..4515d531835 100644
@@ -186,7 +186,8 @@ rxq_alloc_elts_sprq(struct mlx5_rxq_ctrl *rxq_ctrl)
 		mbuf_init->data_off = RTE_PKTMBUF_HEADROOM;
 		rte_mbuf_refcnt_set(mbuf_init, 1);
 		mbuf_init->nb_segs = 1;
-		mbuf_init->port = rxq->port_id;
+		/* For shared queues port is provided in CQE */
+		mbuf_init->port = rxq->shared ? 0 : rxq->port_id;
 		if (priv->flags & RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF)
 			mbuf_init->ol_flags = RTE_MBUF_F_EXTERNAL;
 		/*

drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +6 -0

index 1d00c1c43d1..423e229508c 100644
@@ -1189,6 +1189,12 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 
 		/* D.5 fill in mbuf - rearm_data and packet_type. */
 		rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]);
+		if (unlikely(rxq->shared)) {
+			pkts[pos]->port = cq[pos].user_index_low;
+			pkts[pos + p1]->port = cq[pos + p1].user_index_low;
+			pkts[pos + p2]->port = cq[pos + p2].user_index_low;
+			pkts[pos + p3]->port = cq[pos + p3].user_index_low;
+		}
 		if (rxq->hw_timestamp) {
 			int offset = rxq->timestamp_offset;
 			if (rxq->rt_timestamp) {

drivers/net/mlx5/mlx5_rxtx_vec_neon.h +11 -1

index aa36df29a09..b1d16baa619 100644
@@ -787,7 +787,17 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 		/* C.4 fill in mbuf - rearm_data and packet_type. */
 		rxq_cq_to_ptype_oflags_v(rxq, ptype_info, flow_tag,
 					 opcode, &elts[pos]);
-		if (rxq->hw_timestamp) {
+		if (unlikely(rxq->shared)) {
+			elts[pos]->port = container_of(p0, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 1]->port = container_of(p1, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 2]->port = container_of(p2, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 3]->port = container_of(p3, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+		}
+		if (unlikely(rxq->hw_timestamp)) {
 			int offset = rxq->timestamp_offset;
 			if (rxq->rt_timestamp) {
 				struct mlx5_dev_ctx_shared *sh = rxq->sh;

drivers/net/mlx5/mlx5_rxtx_vec_sse.h +7 -1

index b0fc29d7b9e..f3d838389e2 100644
@@ -736,7 +736,13 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 		*err |= _mm_cvtsi128_si64(opcode);
 		/* D.5 fill in mbuf - rearm_data and packet_type. */
 		rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]);
-		if (rxq->hw_timestamp) {
+		if (unlikely(rxq->shared)) {
+			pkts[pos]->port = cq[pos].user_index_low;
+			pkts[pos + p1]->port = cq[pos + p1].user_index_low;
+			pkts[pos + p2]->port = cq[pos + p2].user_index_low;
+			pkts[pos + p3]->port = cq[pos + p3].user_index_low;
+		}
+		if (unlikely(rxq->hw_timestamp)) {
 			int offset = rxq->timestamp_offset;
 			if (rxq->rt_timestamp) {
 				struct mlx5_dev_ctx_shared *sh = rxq->sh;

Signature · 1 line

2.33.0

David Christensen

Reviewed-by on v4 14/14

2021-11-04 17:50 UTC #263 lore raw

Re: [dpdk-dev] [PATCH v4 14/14] net/mlx5: add shared Rx queue port datapath support

Quoted 112 lines · Xueming Li

From: Viacheslav Ovsiienko <[email protected]>

When receive packet, mlx5 PMD saves mbuf port number from RxQ data.

To support shared RxQ, save port number into RQ context as user index. Received packet resolve port number from CQE user index which derived from RQ context.

Legacy Verbs API doesn't support RQ user index setting, still read from RxQ port number.

Signed-off-by Xueming Li <[email protected]>

Signed-off-by Viacheslav Ovsiienko <[email protected]>

Acked-by Slava Ovsiienko <[email protected]>

---
  drivers/net/mlx5/mlx5_devx.c             |  1 +
  drivers/net/mlx5/mlx5_rx.c               |  1 +
  drivers/net/mlx5/mlx5_rxq.c              |  3 ++-
  drivers/net/mlx5/mlx5_rxtx_vec_altivec.h |  6 ++++++
  drivers/net/mlx5/mlx5_rxtx_vec_neon.h    | 12 +++++++++++-
  drivers/net/mlx5/mlx5_rxtx_vec_sse.h     |  8 +++++++-
  6 files changed, 28 insertions(+), 3 deletions(-)

drivers/net/mlx5/mlx5_devx.c +1 -0

index d3d189ab7f2..a9f9f4af700 100644
@@ -277,6 +277,7 @@ mlx5_rxq_create_devx_rq_resources(struct mlx5_rxq_priv *rxq)
  						MLX5_WQ_END_PAD_MODE_NONE;
  	rq_attr.wq_attr.pd = cdev->pdn;
  	rq_attr.counter_set_id = priv->counter_set_id;
+	rq_attr.user_index = rte_cpu_to_be_16(priv->dev_data->port_id);
  	if (rxq_data->shared) /* Create RMP based RQ. */
  		rxq->devx_rq.rmp = &rxq_ctrl->obj->devx_rmp;
  	/* Create RQ using DevX API. */

drivers/net/mlx5/mlx5_rx.c +1 -0

index 1ffa1b95b88..4d85f64accd 100644
@@ -709,6 +709,7 @@ rxq_cq_to_mbuf(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt,
  {
  	/* Update packet information. */
  	pkt->packet_type = rxq_cq_to_pkt_type(rxq, cqe, mcqe);
+	pkt->port = unlikely(rxq->shared) ? cqe->user_index_low : rxq->port_id;
 
  	if (rxq->rss_hash) {
  		uint32_t rss_hash_res = 0;

drivers/net/mlx5/mlx5_rxq.c +2 -1

index 8feb3e2c0fb..4515d531835 100644
@@ -186,7 +186,8 @@ rxq_alloc_elts_sprq(struct mlx5_rxq_ctrl *rxq_ctrl)
  		mbuf_init->data_off = RTE_PKTMBUF_HEADROOM;
  		rte_mbuf_refcnt_set(mbuf_init, 1);
  		mbuf_init->nb_segs = 1;
-		mbuf_init->port = rxq->port_id;
+		/* For shared queues port is provided in CQE */
+		mbuf_init->port = rxq->shared ? 0 : rxq->port_id;
  		if (priv->flags & RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF)
  			mbuf_init->ol_flags = RTE_MBUF_F_EXTERNAL;
  		/*

drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +6 -0

index 1d00c1c43d1..423e229508c 100644
@@ -1189,6 +1189,12 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
 
  		/* D.5 fill in mbuf - rearm_data and packet_type. */
  		rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]);
+		if (unlikely(rxq->shared)) {
+			pkts[pos]->port = cq[pos].user_index_low;
+			pkts[pos + p1]->port = cq[pos + p1].user_index_low;
+			pkts[pos + p2]->port = cq[pos + p2].user_index_low;
+			pkts[pos + p3]->port = cq[pos + p3].user_index_low;
+		}
  		if (rxq->hw_timestamp) {
  			int offset = rxq->timestamp_offset;
  			if (rxq->rt_timestamp) {

drivers/net/mlx5/mlx5_rxtx_vec_neon.h +11 -1

index aa36df29a09..b1d16baa619 100644
@@ -787,7 +787,17 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
  		/* C.4 fill in mbuf - rearm_data and packet_type. */
  		rxq_cq_to_ptype_oflags_v(rxq, ptype_info, flow_tag,
  					 opcode, &elts[pos]);
-		if (rxq->hw_timestamp) {
+		if (unlikely(rxq->shared)) {
+			elts[pos]->port = container_of(p0, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 1]->port = container_of(p1, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 2]->port = container_of(p2, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 3]->port = container_of(p3, struct mlx5_cqe,
+					      pkt_info)->user_index_low;
+		}
+		if (unlikely(rxq->hw_timestamp)) {
  			int offset = rxq->timestamp_offset;
  			if (rxq->rt_timestamp) {
  				struct mlx5_dev_ctx_shared *sh = rxq->sh;

drivers/net/mlx5/mlx5_rxtx_vec_sse.h +0 -1

index b0fc29d7b9e..f3d838389e2 100644
@@ -736,7 +736,13 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq,
  		*err |= _mm_cvtsi128_si64(opcode);
  		/* D.5 fill in mbuf - rearm_data and packet_type. */
  		rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]);
-		if (rxq->hw_timestamp) {

drivers/net/mlx5/mlx5_rxtx_vec_sse.h rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, · line 748

Quoted 10 lines

drivers/net/mlx5/mlx5_rxtx_vec_sse.h +7 -0

+		if (unlikely(rxq->shared)) {
+			pkts[pos]->port = cq[pos].user_index_low;
+			pkts[pos + p1]->port = cq[pos + p1].user_index_low;
+			pkts[pos + p2]->port = cq[pos + p2].user_index_low;
+			pkts[pos + p3]->port = cq[pos + p3].user_index_low;
+		}
+		if (unlikely(rxq->hw_timestamp)) {
  			int offset = rxq->timestamp_offset;
  			if (rxq->rt_timestamp) {
  				struct mlx5_dev_ctx_shared *sh = rxq->sh;

Reviewed-by David Christensen <[email protected]>

Ruifeng Wang

2021-11-05 06:41 UTC #264 lore raw

Re: [dpdk-dev] [PATCH v4 14/14] net/mlx5: add shared Rx queue port datapath support

In reply to Xueming Li

Quoted 102 lines

-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Thursday, November 4, 2021 8:33 PM
To: [email protected]
Cc: Viacheslav Ovsiienko <[email protected]>; [email protected]; Lior Margalit <[email protected]>; Matan Azrad <[email protected]>; David Christensen <[email protected]>; Ruifeng Wang <[email protected]>; Bruce Richardson <[email protected]>; Konstantin Ananyev <[email protected]>
Subject: [PATCH v4 14/14] net/mlx5: add shared Rx queue port datapath support
From: Viacheslav Ovsiienko <[email protected]>
When receive packet, mlx5 PMD saves mbuf port number from RxQ data.
To support shared RxQ, save port number into RQ context as user index. Received packet resolve port number from CQE user index which derived from RQ context.
Legacy Verbs API doesn't support RQ user index setting, still read from RxQ port number.
Signed-off-by Xueming Li <[email protected]>
Signed-off-by Viacheslav Ovsiienko <[email protected]>
Acked-by Slava Ovsiienko <[email protected]>
---
 drivers/net/mlx5/mlx5_devx.c             |  1 +
 drivers/net/mlx5/mlx5_rx.c               |  1 +
 drivers/net/mlx5/mlx5_rxq.c              |  3 ++-
 drivers/net/mlx5/mlx5_rxtx_vec_altivec.h |  6 ++++++
 drivers/net/mlx5/mlx5_rxtx_vec_neon.h    | 12 +++++++++++-
 drivers/net/mlx5/mlx5_rxtx_vec_sse.h     |  8 +++++++-
 6 files changed, 28 insertions(+), 3 deletions(-)
drivers/net/mlx5/mlx5_devx.c +1 -0
index d3d189ab7f2..a9f9f4af700 100644
@@ -277,6 +277,7 @@ mlx5_rxq_create_devx_rq_resources(struct
 mlx5_rxq_priv *rxq)
 
 	MLX5_WQ_END_PAD_MODE_NONE;
 	rq_attr.wq_attr.pd = cdev->pdn;
 	rq_attr.counter_set_id = priv->counter_set_id;
+	rq_attr.user_index = rte_cpu_to_be_16(priv->dev_data->port_id);
 	if (rxq_data->shared) /* Create RMP based RQ. */
drivers/net/mlx5/mlx5_devx.c +0 -0
 		rxq->devx_rq.rmp = &rxq_ctrl->obj->devx_rmp;
 	/* Create RQ using DevX API. */
drivers/net/mlx5/mlx5_rx.c index renamed from drivers/net/mlx5/mlx5_rx.c +0 -0
1ffa1b95b88..4d85f64accd 100644
drivers/net/mlx5/mlx5_rx.c +2 -0
@@ -709,6 +709,7 @@ rxq_cq_to_mbuf(struct mlx5_rxq_data *rxq, struct
 rte_mbuf *pkt,  {
 	/* Update packet information. */
 	pkt->packet_type = rxq_cq_to_pkt_type(rxq, cqe, mcqe);
+	pkt->port = unlikely(rxq->shared) ? cqe->user_index_low :
+rxq->port_id;
 
 	if (rxq->rss_hash) {
 		uint32_t rss_hash_res = 0;
drivers/net/mlx5/mlx5_rxq.c +2 -1
index 8feb3e2c0fb..4515d531835 100644
@@ -186,7 +186,8 @@ rxq_alloc_elts_sprq(struct mlx5_rxq_ctrl *rxq_ctrl)
 		mbuf_init->data_off = RTE_PKTMBUF_HEADROOM;
 		rte_mbuf_refcnt_set(mbuf_init, 1);
 		mbuf_init->nb_segs = 1;
-		mbuf_init->port = rxq->port_id;
+		/* For shared queues port is provided in CQE */
+		mbuf_init->port = rxq->shared ? 0 : rxq->port_id;
 		if (priv->flags & RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF)
 			mbuf_init->ol_flags = RTE_MBUF_F_EXTERNAL;
 		/*
drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +6 -0
index 1d00c1c43d1..423e229508c 100644
@@ -1189,6 +1189,12 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq,
 volatile struct mlx5_cqe *cq,
 
 		/* D.5 fill in mbuf - rearm_data and packet_type. */
 		rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]);
+		if (unlikely(rxq->shared)) {
+			pkts[pos]->port = cq[pos].user_index_low;
+			pkts[pos + p1]->port = cq[pos + p1].user_index_low;
+			pkts[pos + p2]->port = cq[pos + p2].user_index_low;
+			pkts[pos + p3]->port = cq[pos + p3].user_index_low;
+		}
 		if (rxq->hw_timestamp) {
 			int offset = rxq->timestamp_offset;
drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +0 -0
 			if (rxq->rt_timestamp) {
drivers/net/mlx5/mlx5_rxtx_vec_neon.h +0 -0
index aa36df29a09..b1d16baa619 100644

drivers/net/mlx5/mlx5_rxtx_vec_neon.h rxq_cq_process_v(struct mlx5_rxq_data *rxq, · line 793

Quoted 8 lines

drivers/net/mlx5/mlx5_rxtx_vec_neon.h +3 -1

@@ -787,7 +787,17 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq,
 volatile struct mlx5_cqe *cq,
 		/* C.4 fill in mbuf - rearm_data and packet_type. */
 		rxq_cq_to_ptype_oflags_v(rxq, ptype_info, flow_tag,
 					 opcode, &elts[pos]);
-		if (rxq->hw_timestamp) {
+		if (unlikely(rxq->shared)) {
+			elts[pos]->port = container_of(p0, struct mlx5_cqe,
+					      pkt_info)->user_index_low;

I don't know hardware details on CQE, just comparing with other parts.
1. Is there a need to convert 'user_index_low' with rte_be_to_cpu_16?
2. 'cq' is available as input parameter. Can we directly use cq[pos] instead of using container_of(p0), just like what was done on other architectures?

Thanks.

Quoted 37 lines

drivers/net/mlx5/mlx5_rxtx_vec_neon.h +5 -0

+			elts[pos + 1]->port = container_of(p1, struct
 mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 2]->port = container_of(p2, struct
 mlx5_cqe,
+					      pkt_info)->user_index_low;
+			elts[pos + 3]->port = container_of(p3, struct

mlx5_cqe,

drivers/net/mlx5/mlx5_rxtx_vec_neon.h +3 -0

+					      pkt_info)->user_index_low;
+		}
+		if (unlikely(rxq->hw_timestamp)) {
 			int offset = rxq->timestamp_offset;
 			if (rxq->rt_timestamp) {
 				struct mlx5_dev_ctx_shared *sh = rxq->sh;

drivers/net/mlx5/mlx5_rxtx_vec_sse.h +7 -1

index b0fc29d7b9e..f3d838389e2 100644
@@ -736,7 +736,13 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq,
 volatile struct mlx5_cqe *cq,
 		*err |= _mm_cvtsi128_si64(opcode);
 		/* D.5 fill in mbuf - rearm_data and packet_type. */
 		rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]);
-		if (rxq->hw_timestamp) {
+		if (unlikely(rxq->shared)) {
+			pkts[pos]->port = cq[pos].user_index_low;
+			pkts[pos + p1]->port = cq[pos + p1].user_index_low;
+			pkts[pos + p2]->port = cq[pos + p2].user_index_low;
+			pkts[pos + p3]->port = cq[pos + p3].user_index_low;
+		}
+		if (unlikely(rxq->hw_timestamp)) {
 			int offset = rxq->timestamp_offset;
 			if (rxq->rt_timestamp) {

struct mlx5_dev_ctx_shared *sh = rxq->sh;
--
2.33.0

Raslan Darawsheh

2021-11-04 20:06 UTC #265 lore raw

Re: [dpdk-dev] [PATCH v4 00/14] net/mlx5: support shared Rx queue

In reply to Xueming Li

Hi,

Quoted 72 lines

-----Original Message-----
From: dev <[email protected]> On Behalf Of Xueming Li
Sent: Thursday, November 4, 2021 2:33 PM
To: [email protected]
Cc: Xueming(Steven) Li <[email protected]>; Lior Margalit <[email protected]>
Subject: [dpdk-dev] [PATCH v4 00/14] net/mlx5: support shared Rx queue
Implemetation of Shared Rx queue.
v1:
- initial version
v2:
- rebased on latest dependent series
- fully tested
- support share_qid of RxQ configuration
v3:
- internally reviewed
- removed MPRQ support
- fixed multi-segment support
- fixed configure not applied after port restart
v4:
- rebase with latest code
Viacheslav Ovsiienko (1):
net/mlx5: add shared Rx queue port datapath support
Xueming Li (13):
common/mlx5: introduce user index field in completion net/mlx5: fix field reference for PPC common/mlx5: adds basic receive memory pool support common/mlx5: support receive memory pool net/mlx5: fix Rx queue memory allocation return value net/mlx5: clean Rx queue code
net/mlx5: split Rx queue into shareable and private net/mlx5: move Rx queue reference count net/mlx5: move Rx queue hairpin info to private data net/mlx5: remove port info from shareable Rx queue net/mlx5: move Rx queue DevX resource net/mlx5: remove Rx queue data list from device net/mlx5: support shared Rx queue
 doc/guides/nics/features/mlx5.ini        |   1 +
 doc/guides/nics/mlx5.rst                 |   6 +
 drivers/common/mlx5/mlx5_common_devx.c   | 295 +++++++++--
 drivers/common/mlx5/mlx5_common_devx.h   |  19 +-
 drivers/common/mlx5/mlx5_devx_cmds.c     |  52 ++
 drivers/common/mlx5/mlx5_devx_cmds.h     |  16 +
 drivers/common/mlx5/mlx5_prm.h           |  93 +++-
 drivers/common/mlx5/version.map          |   1 +
 drivers/net/mlx5/linux/mlx5_os.c         |   2 +
 drivers/net/mlx5/linux/mlx5_verbs.c      | 169 +++---
 drivers/net/mlx5/mlx5.c                  |  10 +-
 drivers/net/mlx5/mlx5.h                  |  17 +-
 drivers/net/mlx5/mlx5_devx.c             | 270 +++++-----
 drivers/net/mlx5/mlx5_ethdev.c           |  21 +-
 drivers/net/mlx5/mlx5_flow.c             |  47 +-
 drivers/net/mlx5/mlx5_rss.c              |   6 +-
 drivers/net/mlx5/mlx5_rx.c               |  31 +-
 drivers/net/mlx5/mlx5_rx.h               |  45 +-
 drivers/net/mlx5/mlx5_rxq.c              | 630 +++++++++++++++++------
 drivers/net/mlx5/mlx5_rxtx.c             |   6 +-
 drivers/net/mlx5/mlx5_rxtx_vec.c         |   8 +-
 drivers/net/mlx5/mlx5_rxtx_vec_altivec.h |  14 +-
 drivers/net/mlx5/mlx5_rxtx_vec_neon.h    |  12 +-
 drivers/net/mlx5/mlx5_rxtx_vec_sse.h     |   8 +-
 drivers/net/mlx5/mlx5_stats.c            |   9 +-
 drivers/net/mlx5/mlx5_trigger.c          | 155 +++---
 drivers/net/mlx5/mlx5_vlan.c             |  16 +-
 drivers/regex/mlx5/mlx5_regex_fastpath.c |   2 +-
 28 files changed, 1377 insertions(+), 584 deletions(-)

Quoted 2 lines

--
2.33.0

Series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh