Skip to contentSkip to conversation

fix queue stats mapping

RFC Superseded Min Hu (Connor) 3 participants 20 messages Oct 20, 2020 – Dec 8, 2020

lore
Share & tools
lore
#0 lore raw

v2 cover fix queue stats mapping

From: Huisong Li <[email protected]>

This patch set is used to adjust the unreasonable configurations
and runtime errors about rx/tx queue stats mapping.

---
V1 -> V2
* adjust commit log description.
---

Huisong Li (2):
app/testpmd: fix queue stats mapping configuration app/testpmd: fix starting failed with queue-stats-mapping

 app/test-pmd/config.c     | 180 ++++++++++++++++++++++++++++-----------------
 app/test-pmd/parameters.c |  63 ++++++++--------
 app/test-pmd/testpmd.c    | 182 ++++++++++++++++++++++++++++++++--------------
 app/test-pmd/testpmd.h    |  41 +++++------
 4 files changed, 293 insertions(+), 173 deletions(-)
Signature · 1 line
2.7.4
#1 lore raw

RFC v2 1/2 app/testpmd: fix queue stats mapping configuration

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) There is no need to keep cache line alignment for
'struct queue_stats_mappings'.
4) The mapping arrays, 'tx_queue_stats_mappings_array' & 'rx_queue_stats_mappings_array' are global and their sizes are based on fixed max port and queue size assumptions.
5) The configuration result does not take effect or can not be queried in real time.

Therefore, we have made the following adjustments:
1) If PMD supports queue stats mapping, configure to driver in real time after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Only display queues that mapping done by adding a new 'active' field in queue_stats_mappings struct.
3) Remove cache alignment for 'struct queue_stats_mappings'.
4) Add a new port_stats_mappings struct in rte_port.
The struct contains number of rx/txq stats mapping, rx/tx queue_stats_mapping_enabled flag, and rx/tx queue_stats_mapping array. Size of queue_stats_mapping_array is set to "RTE_ETHDEV_QUEUE_STAT_CNTRS"
to ensure that the same number of queues can be set for each port.

Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")
---
 app/test-pmd/config.c     | 180 +++++++++++++++++++++++++++++-----------------
 app/test-pmd/parameters.c |  63 ++++++++--------
 app/test-pmd/testpmd.c    | 180 ++++++++++++++++++++++++++++++++--------------
 app/test-pmd/testpmd.h    |  41 ++++++-----
 4 files changed, 292 insertions(+), 172 deletions(-)
app/test-pmd/config.c +115 -65
index ba17f3b..325c42f 100644
@@ -177,13 +177,13 @@ nic_stats_display(portid_t port_id)
 	static uint64_t prev_bytes_rx[RTE_MAX_ETHPORTS];
 	static uint64_t prev_bytes_tx[RTE_MAX_ETHPORTS];
 	static uint64_t prev_ns[RTE_MAX_ETHPORTS];
+	struct port_stats_mappings *p_stats_map;
 	struct timespec cur_time;
 	uint64_t diff_pkts_rx, diff_pkts_tx, diff_bytes_rx, diff_bytes_tx,
 								diff_ns;
 	uint64_t mpps_rx, mpps_tx, mbps_rx, mbps_tx;
 	struct rte_eth_stats stats;
-	struct rte_port *port = &ports[port_id];
-	uint8_t i;
+	struct rte_port *port;
 
 	static const char *nic_stats_border = "########################";
 
@@ -195,7 +195,10 @@ nic_stats_display(portid_t port_id)
 	printf("\n  %s NIC statistics for port %-2d %s\n",
 	       nic_stats_border, port_id, nic_stats_border);
 
-	if ((!port->rx_queue_stats_mapping_enabled) && (!port->tx_queue_stats_mapping_enabled)) {
+	port = &ports[port_id];
+	p_stats_map = &port->p_stats_map;
+	if ((!p_stats_map->rx_queue_stats_mapping_enabled) &&
+		(!p_stats_map->tx_queue_stats_mapping_enabled)) {
 		printf("  RX-packets: %-10"PRIu64" RX-missed: %-10"PRIu64" RX-bytes:  "
 		       "%-"PRIu64"\n",
 		       stats.ipackets, stats.imissed, stats.ibytes);
@@ -205,36 +208,20 @@ nic_stats_display(portid_t port_id)
 		printf("  TX-packets: %-10"PRIu64" TX-errors: %-10"PRIu64" TX-bytes:  "
 		       "%-"PRIu64"\n",
 		       stats.opackets, stats.oerrors, stats.obytes);
-	}
-	else {
-		printf("  RX-packets:              %10"PRIu64"    RX-errors: %10"PRIu64
-		       "    RX-bytes: %10"PRIu64"\n",
-		       stats.ipackets, stats.ierrors, stats.ibytes);
-		printf("  RX-errors:  %10"PRIu64"\n", stats.ierrors);
-		printf("  RX-nombuf:               %10"PRIu64"\n",
+	} else {
+		printf("  RX-packets:             %14"PRIu64"    RX-missed: "
+		       "%14"PRIu64"    RX-bytes: %14"PRIu64"\n",
+		       stats.ipackets, stats.imissed, stats.ibytes);
+		printf("  RX-errors:              %14"PRIu64"\n",
+		       stats.ierrors);
+		printf("  RX-nombuf:              %14"PRIu64"\n",
 		       stats.rx_nombuf);
-		printf("  TX-packets:              %10"PRIu64"    TX-errors: %10"PRIu64
-		       "    TX-bytes: %10"PRIu64"\n",
+		printf("  TX-packets:             %14"PRIu64"    TX-errors: "
+		       "%14"PRIu64"    TX-bytes: %14"PRIu64"\n",
 		       stats.opackets, stats.oerrors, stats.obytes);
 	}
 
-	if (port->rx_queue_stats_mapping_enabled) {
-		printf("\n");
-		for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS; i++) {
-			printf("  Stats reg %2d RX-packets: %10"PRIu64
-			       "    RX-errors: %10"PRIu64
-			       "    RX-bytes: %10"PRIu64"\n",
-			       i, stats.q_ipackets[i], stats.q_errors[i], stats.q_ibytes[i]);
-		}
-	}
-	if (port->tx_queue_stats_mapping_enabled) {
-		printf("\n");
-		for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS; i++) {
-			printf("  Stats reg %2d TX-packets: %10"PRIu64
-			       "                             TX-bytes: %10"PRIu64"\n",
-			       i, stats.q_opackets[i], stats.q_obytes[i]);
-		}
-	}
+	port_stats_mapping_display(port_id, &stats);
 
 	diff_ns = 0;
 	if (clock_gettime(CLOCK_TYPE_ID, &cur_time) == 0) {
@@ -400,7 +387,9 @@ nic_xstats_clear(portid_t port_id)
 void
 nic_stats_mapping_display(portid_t port_id)
 {
-	struct rte_port *port = &ports[port_id];
+	struct port_stats_mappings *p_stats_map;
+	struct queue_stats_mappings *q_stats_map;
+	struct rte_port *port;
 	uint16_t i;
 
 	static const char *nic_stats_mapping_border = "########################";
@@ -410,7 +399,10 @@ nic_stats_mapping_display(portid_t port_id)
 		return;
 	}
 
-	if ((!port->rx_queue_stats_mapping_enabled) && (!port->tx_queue_stats_mapping_enabled)) {
+	port = &ports[port_id];
+	p_stats_map = &port->p_stats_map;
+	if ((!p_stats_map->rx_queue_stats_mapping_enabled) &&
+		(!p_stats_map->tx_queue_stats_mapping_enabled)) {
 		printf("Port id %d - either does not support queue statistic mapping or"
 		       " no queue statistic mapping set\n", port_id);
 		return;
@@ -419,24 +411,26 @@ nic_stats_mapping_display(portid_t port_id)
 	printf("\n  %s NIC statistics mapping for port %-2d %s\n",
 	       nic_stats_mapping_border, port_id, nic_stats_mapping_border);
 
-	if (port->rx_queue_stats_mapping_enabled) {
-		for (i = 0; i < nb_rx_queue_stats_mappings; i++) {
-			if (rx_queue_stats_mappings[i].port_id == port_id) {
+	if (p_stats_map->rx_queue_stats_mapping_enabled) {
+		for (i = 0; i < p_stats_map->nb_rxq_stats_mappings; i++) {
+			q_stats_map = &p_stats_map->rxq_map_array[i];
+			if (q_stats_map->active) {
 				printf("  RX-queue %2d mapped to Stats Reg %2d\n",
-				       rx_queue_stats_mappings[i].queue_id,
-				       rx_queue_stats_mappings[i].stats_counter_id);
+				       q_stats_map->queue_id,
+				       q_stats_map->stats_counter_id);
 			}
 		}
 		printf("\n");
 	}
 
 
-	if (port->tx_queue_stats_mapping_enabled) {
-		for (i = 0; i < nb_tx_queue_stats_mappings; i++) {
-			if (tx_queue_stats_mappings[i].port_id == port_id) {
+	if (p_stats_map->tx_queue_stats_mapping_enabled) {
+		for (i = 0; i < p_stats_map->nb_txq_stats_mappings; i++) {
+			q_stats_map = &p_stats_map->txq_map_array[i];
+			if (q_stats_map->active) {
 				printf("  TX-queue %2d mapped to Stats Reg %2d\n",
-				       tx_queue_stats_mappings[i].queue_id,
-				       tx_queue_stats_mappings[i].stats_counter_id);
+				       q_stats_map->queue_id,
+				       q_stats_map->stats_counter_id);
 			}
 		}
 	}
@@ -4546,8 +4540,13 @@ tx_vlan_pvid_set(portid_t port_id, uint16_t vlan_id, int on)
 void
 set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value)
 {
+	struct port_stats_mappings *p_stats_map;
+	struct queue_stats_mappings *q_stats_map;
+	bool existing_mapping_found = false;
+	struct rte_port *port;
+	uint16_t cur_map_idx;
 	uint16_t i;
-	uint8_t existing_mapping_found = 0;
+	int ret;
 
 	if (port_id_is_invalid(port_id, ENABLED_WARN))
 		return;
@@ -4561,37 +4560,88 @@ set_qmap(portid_t port_id, uint8_t is_rx, uint16_t queue_id, uint8_t map_value)
 		return;
 	}
 
-	if (!is_rx) { /*then tx*/
-		for (i = 0; i < nb_tx_queue_stats_mappings; i++) {
-			if ((tx_queue_stats_mappings[i].port_id == port_id) &&
-			    (tx_queue_stats_mappings[i].queue_id == queue_id)) {
-				tx_queue_stats_mappings[i].stats_counter_id = map_value;
-				existing_mapping_found = 1;
+	port = &ports[port_id];
+	p_stats_map = &port->p_stats_map;
+	if (!is_rx) { /* tx */
+		for (i = 0; i < p_stats_map->nb_txq_stats_mappings; i++) {
+			q_stats_map = &p_stats_map->txq_map_array[i];
+			if (q_stats_map->queue_id == queue_id) {
+				ret =
+				rte_eth_dev_set_tx_queue_stats_mapping(port_id,
+							queue_id, map_value);
+				if (ret) {
+					printf("failed to set tx queue stats "
+						"mapping.\n");
+					return;
+				}
+
+				q_stats_map->stats_counter_id = map_value;
+				q_stats_map->active = true;
+				existing_mapping_found = true;
 				break;
 			}
 		}
-		if (!existing_mapping_found) { /* A new additional mapping... */
-			tx_queue_stats_mappings[nb_tx_queue_stats_mappings].port_id = port_id;
-			tx_queue_stats_mappings[nb_tx_queue_stats_mappings].queue_id = queue_id;
-			tx_queue_stats_mappings[nb_tx_queue_stats_mappings].stats_counter_id = map_value;
-			nb_tx_queue_stats_mappings++;
+
+		/* A new additional mapping... */
+		if (!existing_mapping_found) {
+			ret = rte_eth_dev_set_tx_queue_stats_mapping(port_id,
+								     queue_id,
+								     map_value);
+			if (ret) {
+				printf("failed to set tx queue stats "
+					"mapping.\n");
+				return;
+			}
+
+			cur_map_idx = p_stats_map->nb_txq_stats_mappings;
+			q_stats_map = &p_stats_map->txq_map_array[cur_map_idx];
+			q_stats_map->queue_id = queue_id;
+			q_stats_map->stats_counter_id = map_value;
+			q_stats_map->active = true;
+			p_stats_map->nb_txq_stats_mappings++;
 		}
-	}
-	else { /*rx*/
-		for (i = 0; i < nb_rx_queue_stats_mappings; i++) {
-			if ((rx_queue_stats_mappings[i].port_id == port_id) &&
-			    (rx_queue_stats_mappings[i].queue_id == queue_id)) {
-				rx_queue_stats_mappings[i].stats_counter_id = map_value;
-				existing_mapping_found = 1;
+
+		p_stats_map->tx_queue_stats_mapping_enabled = true;
+	} else { /* rx */
+		for (i = 0; i < p_stats_map->nb_rxq_stats_mappings; i++) {
+			q_stats_map = &p_stats_map->rxq_map_array[i];
+			if (q_stats_map->queue_id == queue_id) {
+				ret =
+				rte_eth_dev_set_rx_queue_stats_mapping(port_id,
+							queue_id, map_value);
+				if (ret) {
+					printf("failed to set rx queue stats "
+						"mapping.\n");
+					return;
+				}
+
+				q_stats_map->stats_counter_id = map_value;
+				q_stats_map->active = true;
+				existing_mapping_found = true;
 				break;
 			}
 		}
-		if (!existing_mapping_found) { /* A new additional mapping... */
-			rx_queue_stats_mappings[nb_rx_queue_stats_mappings].port_id = port_id;
-			rx_queue_stats_mappings[nb_rx_queue_stats_mappings].queue_id = queue_id;
-			rx_queue_stats_mappings[nb_rx_queue_stats_mappings].stats_counter_id = map_value;
-			nb_rx_queue_stats_mappings++;
+
+		/* A new additional mapping... */
+		if (!existing_mapping_found) {
+			ret = rte_eth_dev_set_rx_queue_stats_mapping(port_id,
+								     queue_id,
+								     map_value);
+			if (ret) {
+				printf("failed to set rx queue stats "
+					"mapping.\n");
+				return;
+			}
+
+			cur_map_idx = p_stats_map->nb_rxq_stats_mappings;
+			q_stats_map = &p_stats_map->rxq_map_array[cur_map_idx];
+			q_stats_map->queue_id = queue_id;
+			q_stats_map->stats_counter_id = map_value;
+			q_stats_map->active = true;
+			p_stats_map->nb_rxq_stats_mappings++;
 		}
+
+		p_stats_map->rx_queue_stats_mapping_enabled = true;
 	}
 }
 
app/test-pmd/parameters.c +30 -33

92 lines not on this page · raw message

app/test-pmd/testpmd.c +127 -53

261 lines not on this page · raw message

app/test-pmd/testpmd.h +20 -21

65 lines not on this page · raw message

Signature · 1 line
2.7.4
#2 lore raw

Re: [dpdk-dev] [RFC V2 1/2] app/testpmd: fix queue stats mapping configuration

Quoted 15 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) There is no need to keep cache line alignment for
'struct queue_stats_mappings'.
4) The mapping arrays, 'tx_queue_stats_mappings_array' & 'rx_queue_stats_mappings_array' are global and their sizes are based on fixed max port and queue size assumptions.
5) The configuration result does not take effect or can not be queried in real time.

Quoted 13 lines

Therefore, we have made the following adjustments:
1) If PMD supports queue stats mapping, configure to driver in real time after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Only display queues that mapping done by adding a new 'active' field in queue_stats_mappings struct.
3) Remove cache alignment for 'struct queue_stats_mappings'.
4) Add a new port_stats_mappings struct in rte_port.
The struct contains number of rx/txq stats mapping, rx/tx queue_stats_mapping_enabled flag, and rx/tx queue_stats_mapping array. Size of queue_stats_mapping_array is set to "RTE_ETHDEV_QUEUE_STAT_CNTRS"
to ensure that the same number of queues can be set for each port.

Hi Connor,

I think above adjustment are good, but after the decision to use xstats for the queue stats, what do you think about more simplification,

1)
What testpmd does is, records the queue stats mapping commands and registers them later on port start & forwarding start. What happens if recording and registering completely removed?
When "set stat_qmap .." issued, it just call the ethdev APIs to do the mapping in device.
This lets us removing record structures, "struct port_stats_mappings p_stats_map"
Also can remove 'map_port_queue_stats_mapping_registers()' and its sub functions.

2)
Also lets remove "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, which enables removing 'parse_queue_stats_mapping_config()' function too

3)
Another problem is to display the queue stats, in 'fwd_stats_display()' & 'nic_stats_display()', there is a check if the queue stats mapping enable or not ('rx_queue_stats_mapping_enabled' & 'tx_queue_stats_mapping_enabled'), I think displaying queue stats and queue stat mapping should be separate, why not drop checks for queue stats mapping and display queue stats for 'nb_rxq' & 'nb_txq' queues?

Does above make sense?

Majority of the drivers doesn't require queue stat mapping to get the queue stats, lets don't pollute main usage with this requirement.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")

<...>

#3 lore raw

Re: [dpdk-dev] [RFC V2 1/2] app/testpmd: fix queue stats mapping configuration

Hi Ferruh,

I agree with your proposal. But if we remove record structures, we will not be able to query the current queue stats mapping configuration. Or we can provide a query API for the PMD driver that uses the set_queue_stats_mapping API, and driver records these mapping information from user.

What do you think?

Quoted 63 lines · Ferruh Yigit
Quoted 15 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) There is no need to keep cache line alignment for
'struct queue_stats_mappings'.
4) The mapping arrays, 'tx_queue_stats_mappings_array' & 'rx_queue_stats_mappings_array' are global and their sizes are based on fixed max port and queue size assumptions.
5) The configuration result does not take effect or can not be queried in real time.

Quoted 13 lines

Therefore, we have made the following adjustments:
1) If PMD supports queue stats mapping, configure to driver in real time after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Only display queues that mapping done by adding a new 'active' field in queue_stats_mappings struct.
3) Remove cache alignment for 'struct queue_stats_mappings'.
4) Add a new port_stats_mappings struct in rte_port.
The struct contains number of rx/txq stats mapping, rx/tx queue_stats_mapping_enabled flag, and rx/tx queue_stats_mapping array. Size of queue_stats_mapping_array is set to "RTE_ETHDEV_QUEUE_STAT_CNTRS" to ensure that the same number of queues can be set for each port.

Hi Connor,

I think above adjustment are good, but after the decision to use xstats for the queue stats, what do you think about more simplification,

1)
What testpmd does is, records the queue stats mapping commands and registers them later on port start & forwarding start. What happens if recording and registering completely removed?
When "set stat_qmap .." issued, it just call the ethdev APIs to do the mapping in device.
This lets us removing record structures, "struct port_stats_mappings p_stats_map"
Also can remove 'map_port_queue_stats_mapping_registers()' and its sub functions.

2)
Also lets remove "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, which enables removing 'parse_queue_stats_mapping_config()' function too

3)
Another problem is to display the queue stats, in 'fwd_stats_display()' & 'nic_stats_display()', there is a check if the queue stats mapping enable or not ('rx_queue_stats_mapping_enabled' & 'tx_queue_stats_mapping_enabled'), I think displaying queue stats and queue stat mapping should be separate, why not drop checks for queue stats mapping and display queue stats for 'nb_rxq' & 'nb_txq' queues?

Does above make sense?

Majority of the drivers doesn't require queue stat mapping to get the queue stats, lets don't pollute main usage with this requirement.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")

<...>

.

#4 lore raw

Re: [dpdk-dev] [RFC V2 1/2] app/testpmd: fix queue stats mapping configuration

Hi Ferruh, any suggestions?

Quoted 75 lines · Min Hu (Connor)

Hi Ferruh,

I agree with your proposal. But if we remove record structures, we will not be able to query the current queue stats mapping configuration. Or we can provide a query API for the PMD driver that uses the set_queue_stats_mapping API, and driver records these mapping information from user.

What do you think?

Quoted 63 lines · Ferruh Yigit
Quoted 15 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) There is no need to keep cache line alignment for
'struct queue_stats_mappings'.
4) The mapping arrays, 'tx_queue_stats_mappings_array' & 'rx_queue_stats_mappings_array' are global and their sizes are based on fixed max port and queue size assumptions.
5) The configuration result does not take effect or can not be queried in real time.

Quoted 13 lines

Therefore, we have made the following adjustments:
1) If PMD supports queue stats mapping, configure to driver in real time after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Only display queues that mapping done by adding a new 'active' field in queue_stats_mappings struct.
3) Remove cache alignment for 'struct queue_stats_mappings'.
4) Add a new port_stats_mappings struct in rte_port.
The struct contains number of rx/txq stats mapping, rx/tx queue_stats_mapping_enabled flag, and rx/tx queue_stats_mapping array. Size of queue_stats_mapping_array is set to "RTE_ETHDEV_QUEUE_STAT_CNTRS" to ensure that the same number of queues can be set for each port.

Hi Connor,

I think above adjustment are good, but after the decision to use xstats for the queue stats, what do you think about more simplification,

1)
What testpmd does is, records the queue stats mapping commands and registers them later on port start & forwarding start. What happens if recording and registering completely removed?
When "set stat_qmap .." issued, it just call the ethdev APIs to do the mapping in device.
This lets us removing record structures, "struct port_stats_mappings p_stats_map"
Also can remove 'map_port_queue_stats_mapping_registers()' and its sub functions.

2)
Also lets remove "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, which enables removing 'parse_queue_stats_mapping_config()' function too

3)
Another problem is to display the queue stats, in 'fwd_stats_display()' & 'nic_stats_display()', there is a check if the queue stats mapping enable or not ('rx_queue_stats_mapping_enabled' & 'tx_queue_stats_mapping_enabled'), I think displaying queue stats and queue stat mapping should be separate, why not drop checks for queue stats mapping and display queue stats for 'nb_rxq' & 'nb_txq' queues?

Does above make sense?

Majority of the drivers doesn't require queue stat mapping to get the queue stats, lets don't pollute main usage with this requirement.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")

<...>

.

#5 lore raw

Re: [dpdk-dev] [RFC V2 1/2] app/testpmd: fix queue stats mapping configuration

Quoted 14 lines · Min Hu (Connor)

Hi Ferruh, any suggestions?

Quoted 10 lines · Min Hu (Connor)

Hi Ferruh,

I agree with your proposal. But if we remove record structures, we will not be able to query the current queue stats mapping configuration. Or we can provide a query API for the PMD driver that uses the set_queue_stats_mapping API, and driver records these mapping information from user.

What do you think?

Sorry for delay,

Yes that information will be lost, but since the queue stats mapping is not commonly used, I think it can be OK to loose it.

As you said another option is to add a new ethdev API to get queue stats mapping, but because of same reason not sure about adding it. We can add it later if there is a request for it.

Quoted 65 lines
Quoted 65 lines · Ferruh Yigit
Quoted 63 lines · Min Hu (Connor)
Quoted 15 lines

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) There is no need to keep cache line alignment for
'struct queue_stats_mappings'.
4) The mapping arrays, 'tx_queue_stats_mappings_array' & 'rx_queue_stats_mappings_array' are global and their sizes are based on fixed max port and queue size assumptions.
5) The configuration result does not take effect or can not be queried in real time.

Quoted 13 lines

Therefore, we have made the following adjustments:
1) If PMD supports queue stats mapping, configure to driver in real time after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Only display queues that mapping done by adding a new 'active' field in queue_stats_mappings struct.
3) Remove cache alignment for 'struct queue_stats_mappings'.
4) Add a new port_stats_mappings struct in rte_port.
The struct contains number of rx/txq stats mapping, rx/tx queue_stats_mapping_enabled flag, and rx/tx queue_stats_mapping array. Size of queue_stats_mapping_array is set to "RTE_ETHDEV_QUEUE_STAT_CNTRS" to ensure that the same number of queues can be set for each port.

Hi Connor,

I think above adjustment are good, but after the decision to use xstats for the queue stats, what do you think about more simplification,

1)
What testpmd does is, records the queue stats mapping commands and registers them later on port start & forwarding start. What happens if recording and registering completely removed?
When "set stat_qmap .." issued, it just call the ethdev APIs to do the mapping in device.
This lets us removing record structures, "struct port_stats_mappings p_stats_map"
Also can remove 'map_port_queue_stats_mapping_registers()' and its sub functions.

2)
Also lets remove "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, which enables removing 'parse_queue_stats_mapping_config()' function too

3)
Another problem is to display the queue stats, in 'fwd_stats_display()' & 'nic_stats_display()', there is a check if the queue stats mapping enable or not ('rx_queue_stats_mapping_enabled' & 'tx_queue_stats_mapping_enabled'), I think displaying queue stats and queue stat mapping should be separate, why not drop checks for queue stats mapping and display queue stats for 'nb_rxq' & 'nb_txq' queues?

Does above make sense?

Majority of the drivers doesn't require queue stat mapping to get the queue stats, lets don't pollute main usage with this requirement.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")

<...>

.

#6 lore raw

Re: [dpdk-dev] [RFC V2 1/2] app/testpmd: fix queue stats mapping configuration

HI, Ferruh,
Ok,I will send V3 patches. thanks.

Quoted 89 lines · Ferruh Yigit
Quoted 14 lines · Min Hu (Connor)

Hi Ferruh, any suggestions?

Quoted 10 lines · Min Hu (Connor)

Hi Ferruh,

I agree with your proposal. But if we remove record structures, we will not be able to query the current queue stats mapping configuration. Or we can provide a query API for the PMD driver that uses the set_queue_stats_mapping API, and driver records these mapping information from user.

What do you think?

Sorry for delay,

Yes that information will be lost, but since the queue stats mapping is not commonly used, I think it can be OK to loose it.

As you said another option is to add a new ethdev API to get queue stats mapping, but because of same reason not sure about adding it. We can add it later if there is a request for it.

Quoted 60 lines
Quoted 60 lines · Ferruh Yigit
Quoted 58 lines · Min Hu (Connor)
Quoted 15 lines

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) There is no need to keep cache line alignment for
'struct queue_stats_mappings'.
4) The mapping arrays, 'tx_queue_stats_mappings_array' & 'rx_queue_stats_mappings_array' are global and their sizes are based on fixed max port and queue size assumptions.
5) The configuration result does not take effect or can not be queried in real time.

Quoted 13 lines

Therefore, we have made the following adjustments:
1) If PMD supports queue stats mapping, configure to driver in real time after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Only display queues that mapping done by adding a new 'active' field in queue_stats_mappings struct.
3) Remove cache alignment for 'struct queue_stats_mappings'.
4) Add a new port_stats_mappings struct in rte_port.
The struct contains number of rx/txq stats mapping, rx/tx queue_stats_mapping_enabled flag, and rx/tx queue_stats_mapping array. Size of queue_stats_mapping_array is set to "RTE_ETHDEV_QUEUE_STAT_CNTRS" to ensure that the same number of queues can be set for each port.

Hi Connor,

I think above adjustment are good, but after the decision to use xstats for the queue stats, what do you think about more simplification,

1)
What testpmd does is, records the queue stats mapping commands and registers them later on port start & forwarding start. What happens if recording and registering completely removed?
When "set stat_qmap .." issued, it just call the ethdev APIs to do the mapping in device.
This lets us removing record structures, "struct port_stats_mappings p_stats_map"
Also can remove 'map_port_queue_stats_mapping_registers()' and its sub functions.

2)
Also lets remove "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, which enables removing 'parse_queue_stats_mapping_config()' function too

3)
Another problem is to display the queue stats, in 'fwd_stats_display()' & 'nic_stats_display()', there is a check if the queue stats mapping enable or not ('rx_queue_stats_mapping_enabled' & 'tx_queue_stats_mapping_enabled'), I think displaying queue stats and queue stat mapping should be separate, why not drop checks for queue stats mapping and display queue stats for 'nb_rxq' & 'nb_txq' queues?

Does above make sense?

Majority of the drivers doesn't require queue stat mapping to get the queue stats, lets don't pollute main usage with this requirement.

Quoted 4 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line

<...>

.

.

#7 lore raw

RFC v4 1/1 app/testpmd: fix queue stats mapping configuration

In reply to Ferruh Yigit

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following
simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats.
Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")

---
V3 -> V4
* remove unnecessary spaces.

V2 -> V3
* remove record structures, "rx/tx-queue-stats-mapping" parameters and related function.

V1 -> V2
* adjust commit log description.

---
 app/test-pmd/cmdline.c    |  17 ++---
 app/test-pmd/config.c     | 144 +++++-----------------------------
 app/test-pmd/parameters.c | 107 --------------------------
 app/test-pmd/testpmd.c    | 191 +++++-----------------------------------------
 app/test-pmd/testpmd.h    |  22 ------
 5 files changed, 46 insertions(+), 435 deletions(-)
app/test-pmd/cmdline.c +6 -11
index 5e2881e..4c1ee0d 100644
@@ -163,7 +163,7 @@ static void cmd_help_long_parsed(void *parsed_result,
 			"Display:\n"
 			"--------\n\n"
 
-			"show port (info|stats|summary|xstats|fdir|stat_qmap|dcb_tc|cap) (port_id|all)\n"
+			"show port (info|stats|summary|xstats|fdir|dcb_tc|cap) (port_id|all)\n"
 			"    Display information for port_id, or all.\n\n"
 
 			"show port port_id (module_eeprom|eeprom)\n"
@@ -177,7 +177,7 @@ static void cmd_help_long_parsed(void *parsed_result,
 			"show port (port_id) rss-hash [key]\n"
 			"    Display the RSS hash functions and RSS hash key of port\n\n"
 
-			"clear port (info|stats|xstats|fdir|stat_qmap) (port_id|all)\n"
+			"clear port (info|stats|xstats|fdir) (port_id|all)\n"
 			"    Clear information for port_id, or all.\n\n"
 
 			"show (rxq|txq) info (port_id) (queue_id)\n"
@@ -7552,9 +7552,6 @@ static void cmd_showportall_parsed(void *parsed_result,
 		RTE_ETH_FOREACH_DEV(i)
 			fdir_get_infos(i);
 #endif
-	else if (!strcmp(res->what, "stat_qmap"))
-		RTE_ETH_FOREACH_DEV(i)
-			nic_stats_mapping_display(i);
 	else if (!strcmp(res->what, "dcb_tc"))
 		RTE_ETH_FOREACH_DEV(i)
 			port_dcb_info_display(i);
@@ -7570,14 +7567,14 @@ cmdline_parse_token_string_t cmd_showportall_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_showportall_result, port, "port");
 cmdline_parse_token_string_t cmd_showportall_what =
 	TOKEN_STRING_INITIALIZER(struct cmd_showportall_result, what,
-				 "info#summary#stats#xstats#fdir#stat_qmap#dcb_tc#cap");
+				 "info#summary#stats#xstats#fdir#dcb_tc#cap");
 cmdline_parse_token_string_t cmd_showportall_all =
 	TOKEN_STRING_INITIALIZER(struct cmd_showportall_result, all, "all");
 cmdline_parse_inst_t cmd_showportall = {
 	.f = cmd_showportall_parsed,
 	.data = NULL,
 	.help_str = "show|clear port "
-		"info|summary|stats|xstats|fdir|stat_qmap|dcb_tc|cap all",
+		"info|summary|stats|xstats|fdir|dcb_tc|cap all",
 	.tokens = {
 		(void *)&cmd_showportall_show,
 		(void *)&cmd_showportall_port,
@@ -7619,8 +7616,6 @@ static void cmd_showport_parsed(void *parsed_result,
 	else if (!strcmp(res->what, "fdir"))
 		 fdir_get_infos(res->portnum);
 #endif
-	else if (!strcmp(res->what, "stat_qmap"))
-		nic_stats_mapping_display(res->portnum);
 	else if (!strcmp(res->what, "dcb_tc"))
 		port_dcb_info_display(res->portnum);
 	else if (!strcmp(res->what, "cap"))
@@ -7634,7 +7629,7 @@ cmdline_parse_token_string_t cmd_showport_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_showport_result, port, "port");
 cmdline_parse_token_string_t cmd_showport_what =
 	TOKEN_STRING_INITIALIZER(struct cmd_showport_result, what,
-				 "info#summary#stats#xstats#fdir#stat_qmap#dcb_tc#cap");
+				 "info#summary#stats#xstats#fdir#dcb_tc#cap");
 cmdline_parse_token_num_t cmd_showport_portnum =
 	TOKEN_NUM_INITIALIZER(struct cmd_showport_result, portnum, RTE_UINT16);
 
@@ -7642,7 +7637,7 @@ cmdline_parse_inst_t cmd_showport = {
 	.f = cmd_showport_parsed,
 	.data = NULL,
 	.help_str = "show|clear port "
-		"info|summary|stats|xstats|fdir|stat_qmap|dcb_tc|cap "
+		"info|summary|stats|xstats|fdir|dcb_tc|cap "
 		"<port_id>",
 	.tokens = {
 		(void *)&cmd_showport_show,
app/test-pmd/config.c +21 -123

184 lines not on this page · raw message

app/test-pmd/parameters.c +0 -107

131 lines not on this page · raw message

app/test-pmd/testpmd.c +19 -172

248 lines not on this page · raw message

app/test-pmd/testpmd.h +0 -22
index 6b901a8..5f23162 100644
@@ -206,8 +206,6 @@ struct rte_port {
 	uint16_t                tunnel_tso_segsz; /**< Segmentation offload MSS for tunneled pkts. */
 	uint16_t                tx_vlan_id;/**< The tag ID */
 	uint16_t                tx_vlan_id_outer;/**< The outer tag ID */
-	uint8_t                 tx_queue_stats_mapping_enabled;
-	uint8_t                 rx_queue_stats_mapping_enabled;
 	volatile uint16_t        port_status;    /**< port started or not */
 	uint8_t                 need_setup;     /**< port just attached */
 	uint8_t                 need_reconfig;  /**< need reconfiguring port or not */
@@ -326,25 +324,6 @@ enum dcb_mode_enable
 	DCB_ENABLED
 };
 
-#define MAX_TX_QUEUE_STATS_MAPPINGS 1024 /* MAX_PORT of 32 @ 32 tx_queues/port */
-#define MAX_RX_QUEUE_STATS_MAPPINGS 4096 /* MAX_PORT of 32 @ 128 rx_queues/port */
-
-struct queue_stats_mappings {
-	portid_t port_id;
-	uint16_t queue_id;
-	uint8_t stats_counter_id;
-} __rte_cache_aligned;
-
-extern struct queue_stats_mappings tx_queue_stats_mappings_array[];
-extern struct queue_stats_mappings rx_queue_stats_mappings_array[];
-
-/* Assign both tx and rx queue stats mappings to the same default values */
-extern struct queue_stats_mappings *tx_queue_stats_mappings;
-extern struct queue_stats_mappings *rx_queue_stats_mappings;
-
-extern uint16_t nb_tx_queue_stats_mappings;
-extern uint16_t nb_rx_queue_stats_mappings;
-
 extern uint8_t xstats_hide_zero; /**< Hide zero values for xstats display */
 
 /* globals used for configuration */
@@ -790,7 +769,6 @@ void nic_stats_display(portid_t port_id);
 void nic_stats_clear(portid_t port_id);
 void nic_xstats_display(portid_t port_id);
 void nic_xstats_clear(portid_t port_id);
-void nic_stats_mapping_display(portid_t port_id);
 void device_infos_display(const char *identifier);
 void port_infos_display(portid_t port_id);
 void port_summary_display(portid_t port_id);
Signature · 1 line
2.7.4
#8 lore raw

Re: [dpdk-dev] [RFC V4] app/testpmd: fix queue stats mapping configuration

Quoted 31 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following
simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats. Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")

Overall looks good to me.
I did a quick test didn't see anything unexpected. 'xstats' or 'dpdk-proc-info' app still can be used to get queue stats and "set stat_qmap ..." is working as expected.

But it is a little late for this release cycle, would you be OK to get this at the beginning of next release?

#9 lore raw

Re: [dpdk-dev] [RFC V4] app/testpmd: fix queue stats mapping configuration

Quoted 38 lines · On Fri, 20 Nov 2020 17:26:55 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 31 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following
simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats. Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")
Quoted 7 lines

Overall looks good to me.
I did a quick test didn't see anything unexpected. 'xstats' or 'dpdk-proc-info' app still can be used to get queue stats and "set stat_qmap ..." is working as expected.

But it is a little late for this release cycle, would you be OK to get this at the beginning of next release?

Could we plan to deprecate queue stats mapping in future when xstats work is done?

#10 lore raw

Re: [dpdk-dev] [RFC V4] app/testpmd: fix queue stats mapping configuration

Quoted 46 lines · Stephen Hemminger
Quoted 43 lines · On Fri, 20 Nov 2020 17:26:55 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 31 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following
simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats. Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")

Overall looks good to me.
I did a quick test didn't see anything unexpected. 'xstats' or 'dpdk-proc-info' app still can be used to get queue stats and "set stat_qmap ..." is working as expected.

Quoted 5 lines
Quoted 2 lines

But it is a little late for this release cycle, would you be OK to get this at the beginning of next release?

Could we plan to deprecate queue stats mapping in future when xstats work is done?

Even queue stats moved to xstats, a few PMDs still need this configuration and API. And this patch already cleans the queue stats mapping noise from testpmd.

What is the benefit/motivation to deprecate the queue stats mapping API?

#11 lore raw

Re: [dpdk-dev] [RFC V4] app/testpmd: fix queue stats mapping configuration

Quoted 50 lines · On Fri, 20 Nov 2020 23:33:40 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 49 lines · Stephen Hemminger
Quoted 45 lines · On Fri, 20 Nov 2020 17:26:55 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 31 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following
simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats. Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")

Overall looks good to me.
I did a quick test didn't see anything unexpected. 'xstats' or 'dpdk-proc-info' app still can be used to get queue stats and "set stat_qmap ..." is working as expected.

But it is a little late for this release cycle, would you be OK to get this at the beginning of next release?

Quoted 7 lines
Quoted 2 lines

Could we plan to deprecate queue stats mapping in future when xstats work is done?

Even queue stats moved to xstats, a few PMDs still need this configuration and API. And this patch already cleans the queue stats mapping noise from testpmd.

What is the benefit/motivation to deprecate the queue stats mapping API?

Mostly because so few drivers implement it, that any application using it is going to be non-portable.

#12 lore raw

Re: [dpdk-dev] [RFC V4] app/testpmd: fix queue stats mapping configuration

Quoted 56 lines · Stephen Hemminger
Quoted 53 lines · On Fri, 20 Nov 2020 23:33:40 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 51 lines · Stephen Hemminger
Quoted 38 lines · On Fri, 20 Nov 2020 17:26:55 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 31 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following
simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats. Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")
Quoted 7 lines

Overall looks good to me.
I did a quick test didn't see anything unexpected. 'xstats' or 'dpdk-proc-info' app still can be used to get queue stats and "set stat_qmap ..." is working as expected.

But it is a little late for this release cycle, would you be OK to get this at the beginning of next release?

Could we plan to deprecate queue stats mapping in future when xstats work is done?

Quoted 8 lines
Quoted 4 lines

Even queue stats moved to xstats, a few PMDs still need this configuration and API. And this patch already cleans the queue stats mapping noise from testpmd.

What is the benefit/motivation to deprecate the queue stats mapping API?

Mostly because so few drivers implement it, that any application using it is going to be non-portable.
Hi, Stephen,

For PMDs with few hardware queue statistics resources, such as
ixgbe, all queue stats are recorded in statistics register 0 by default. If users want to get detailed queue stats, users have to call the queue stats mapping API. From this point of view, it is necessary for these PMDs.

Quoted 1 line
#13 lore raw

Re: [dpdk-dev] [RFC V4] app/testpmd: fix queue stats mapping configuration

In reply to Stephen Hemminger
Quoted 56 lines · Stephen Hemminger
Quoted 53 lines · On Fri, 20 Nov 2020 23:33:40 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 51 lines · Stephen Hemminger
Quoted 38 lines · On Fri, 20 Nov 2020 17:26:55 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 31 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following
simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats. Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")
Quoted 7 lines

Overall looks good to me.
I did a quick test didn't see anything unexpected. 'xstats' or 'dpdk-proc-info' app still can be used to get queue stats and "set stat_qmap ..." is working as expected.

But it is a little late for this release cycle, would you be OK to get this at the beginning of next release?

Could we plan to deprecate queue stats mapping in future when xstats work is done?

Quoted 8 lines
Quoted 4 lines

Even queue stats moved to xstats, a few PMDs still need this configuration and API. And this patch already cleans the queue stats mapping noise from testpmd.

What is the benefit/motivation to deprecate the queue stats mapping API?

Mostly because so few drivers implement it, that any application using it is going to be non-portable.

'rte_eth_dev_set_.*_queue_stats_mapping()' APIs return '-ENOTSUP' if underlying PMD doesn't support it, applications can check this for portable code.

#14 lore raw

Re: [dpdk-dev] [RFC V4] app/testpmd: fix queue stats mapping configuration

Hi, Ferruh, and all,
are there any comments about this set of patch?

Quoted 66 lines · Ferruh Yigit
Quoted 54 lines · Stephen Hemminger
Quoted 51 lines · On Fri, 20 Nov 2020 23:33:40 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 49 lines · Stephen Hemminger
Quoted 38 lines · On Fri, 20 Nov 2020 17:26:55 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 31 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats. Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")
Quoted 7 lines

Overall    looks good to me.
I did a quick test didn't see anything unexpected. 'xstats' or 'dpdk-proc-info' app still can be used to get queue stats and "set stat_qmap ..." is working as expected.

But it is a little late for this release cycle, would you be OK to get this at the beginning of next release?

Could we plan to deprecate queue stats mapping in future when xstats work is done?

Quoted 8 lines
Quoted 4 lines

Even queue stats moved to xstats, a few PMDs still need this configuration and API. And this patch already cleans the queue stats mapping noise from testpmd.

What is the benefit/motivation to deprecate the queue stats mapping API?

Mostly because so few drivers implement it, that any application using it is  going to be non-portable.

'rte_eth_dev_set_.*_queue_stats_mapping()' APIs return '-ENOTSUP' if underlying PMD doesn't support it, applications can check this for portable code. .

#15 lore raw

Re: [dpdk-dev] [RFC V4] app/testpmd: fix queue stats mapping configuration

Quoted 3 lines · Min Hu (Connor)

Hi, Ferruh, and all,
    are there any comments about this set of patch?

I am good with this one, would you mind sending an actual patch (not RFC) on top of the latest head?

Thanks,
ferruh

Quoted 67 lines · Ferruh Yigit
Quoted 66 lines · Stephen Hemminger
Quoted 54 lines
Quoted 51 lines · On Fri, 20 Nov 2020 23:33:40 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 49 lines · Stephen Hemminger
Quoted 38 lines · On Fri, 20 Nov 2020 17:26:55 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 31 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats. Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")
Quoted 7 lines

Overall    looks good to me.
I did a quick test didn't see anything unexpected. 'xstats' or 'dpdk-proc-info' app still can be used to get queue stats and "set stat_qmap ..." is working as expected.

But it is a little late for this release cycle, would you be OK to get this at the beginning of next release?

Could we plan to deprecate queue stats mapping in future when xstats work is done?

Quoted 8 lines
Quoted 4 lines

Even queue stats moved to xstats, a few PMDs still need this configuration and API. And this patch already cleans the queue stats mapping noise from testpmd.

What is the benefit/motivation to deprecate the queue stats mapping API?

Mostly because so few drivers implement it, that any application using it is  going to be non-portable.

'rte_eth_dev_set_.*_queue_stats_mapping()' APIs return '-ENOTSUP' if underlying PMD doesn't support it, applications can check this for portable code. .

#16 lore raw

PATCH 1/1 app/testpmd: fix queue stats mapping configuration

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following
simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats.
Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")
---
 app/test-pmd/cmdline.c    |  17 ++---
 app/test-pmd/config.c     | 144 +++++-----------------------------
 app/test-pmd/parameters.c | 107 --------------------------
 app/test-pmd/testpmd.c    | 191 +++++-----------------------------------------
 app/test-pmd/testpmd.h    |  22 ------
 5 files changed, 46 insertions(+), 435 deletions(-)
app/test-pmd/cmdline.c +6 -11
index 5e2881e..4c1ee0d 100644
@@ -163,7 +163,7 @@ static void cmd_help_long_parsed(void *parsed_result,
 			"Display:\n"
 			"--------\n\n"
 
-			"show port (info|stats|summary|xstats|fdir|stat_qmap|dcb_tc|cap) (port_id|all)\n"
+			"show port (info|stats|summary|xstats|fdir|dcb_tc|cap) (port_id|all)\n"
 			"    Display information for port_id, or all.\n\n"
 
 			"show port port_id (module_eeprom|eeprom)\n"
@@ -177,7 +177,7 @@ static void cmd_help_long_parsed(void *parsed_result,
 			"show port (port_id) rss-hash [key]\n"
 			"    Display the RSS hash functions and RSS hash key of port\n\n"
 
-			"clear port (info|stats|xstats|fdir|stat_qmap) (port_id|all)\n"
+			"clear port (info|stats|xstats|fdir) (port_id|all)\n"
 			"    Clear information for port_id, or all.\n\n"
 
 			"show (rxq|txq) info (port_id) (queue_id)\n"
@@ -7552,9 +7552,6 @@ static void cmd_showportall_parsed(void *parsed_result,
 		RTE_ETH_FOREACH_DEV(i)
 			fdir_get_infos(i);
 #endif
-	else if (!strcmp(res->what, "stat_qmap"))
-		RTE_ETH_FOREACH_DEV(i)
-			nic_stats_mapping_display(i);
 	else if (!strcmp(res->what, "dcb_tc"))
 		RTE_ETH_FOREACH_DEV(i)
 			port_dcb_info_display(i);
@@ -7570,14 +7567,14 @@ cmdline_parse_token_string_t cmd_showportall_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_showportall_result, port, "port");
 cmdline_parse_token_string_t cmd_showportall_what =
 	TOKEN_STRING_INITIALIZER(struct cmd_showportall_result, what,
-				 "info#summary#stats#xstats#fdir#stat_qmap#dcb_tc#cap");
+				 "info#summary#stats#xstats#fdir#dcb_tc#cap");
 cmdline_parse_token_string_t cmd_showportall_all =
 	TOKEN_STRING_INITIALIZER(struct cmd_showportall_result, all, "all");
 cmdline_parse_inst_t cmd_showportall = {
 	.f = cmd_showportall_parsed,
 	.data = NULL,
 	.help_str = "show|clear port "
-		"info|summary|stats|xstats|fdir|stat_qmap|dcb_tc|cap all",
+		"info|summary|stats|xstats|fdir|dcb_tc|cap all",
 	.tokens = {
 		(void *)&cmd_showportall_show,
 		(void *)&cmd_showportall_port,
@@ -7619,8 +7616,6 @@ static void cmd_showport_parsed(void *parsed_result,
 	else if (!strcmp(res->what, "fdir"))
 		 fdir_get_infos(res->portnum);
 #endif
-	else if (!strcmp(res->what, "stat_qmap"))
-		nic_stats_mapping_display(res->portnum);
 	else if (!strcmp(res->what, "dcb_tc"))
 		port_dcb_info_display(res->portnum);
 	else if (!strcmp(res->what, "cap"))
@@ -7634,7 +7629,7 @@ cmdline_parse_token_string_t cmd_showport_port =
 	TOKEN_STRING_INITIALIZER(struct cmd_showport_result, port, "port");
 cmdline_parse_token_string_t cmd_showport_what =
 	TOKEN_STRING_INITIALIZER(struct cmd_showport_result, what,
-				 "info#summary#stats#xstats#fdir#stat_qmap#dcb_tc#cap");
+				 "info#summary#stats#xstats#fdir#dcb_tc#cap");
 cmdline_parse_token_num_t cmd_showport_portnum =
 	TOKEN_NUM_INITIALIZER(struct cmd_showport_result, portnum, RTE_UINT16);
 
@@ -7642,7 +7637,7 @@ cmdline_parse_inst_t cmd_showport = {
 	.f = cmd_showport_parsed,
 	.data = NULL,
 	.help_str = "show|clear port "
-		"info|summary|stats|xstats|fdir|stat_qmap|dcb_tc|cap "
+		"info|summary|stats|xstats|fdir|dcb_tc|cap "
 		"<port_id>",
 	.tokens = {
 		(void *)&cmd_showport_show,
app/test-pmd/config.c +21 -123

184 lines not on this page · raw message

app/test-pmd/parameters.c +0 -107

131 lines not on this page · raw message

app/test-pmd/testpmd.c +19 -172

248 lines not on this page · raw message

app/test-pmd/testpmd.h +0 -22
index 6b901a8..5f23162 100644
@@ -206,8 +206,6 @@ struct rte_port {
 	uint16_t                tunnel_tso_segsz; /**< Segmentation offload MSS for tunneled pkts. */
 	uint16_t                tx_vlan_id;/**< The tag ID */
 	uint16_t                tx_vlan_id_outer;/**< The outer tag ID */
-	uint8_t                 tx_queue_stats_mapping_enabled;
-	uint8_t                 rx_queue_stats_mapping_enabled;
 	volatile uint16_t        port_status;    /**< port started or not */
 	uint8_t                 need_setup;     /**< port just attached */
 	uint8_t                 need_reconfig;  /**< need reconfiguring port or not */
@@ -326,25 +324,6 @@ enum dcb_mode_enable
 	DCB_ENABLED
 };
 
-#define MAX_TX_QUEUE_STATS_MAPPINGS 1024 /* MAX_PORT of 32 @ 32 tx_queues/port */
-#define MAX_RX_QUEUE_STATS_MAPPINGS 4096 /* MAX_PORT of 32 @ 128 rx_queues/port */
-
-struct queue_stats_mappings {
-	portid_t port_id;
-	uint16_t queue_id;
-	uint8_t stats_counter_id;
-} __rte_cache_aligned;
-
-extern struct queue_stats_mappings tx_queue_stats_mappings_array[];
-extern struct queue_stats_mappings rx_queue_stats_mappings_array[];
-
-/* Assign both tx and rx queue stats mappings to the same default values */
-extern struct queue_stats_mappings *tx_queue_stats_mappings;
-extern struct queue_stats_mappings *rx_queue_stats_mappings;
-
-extern uint16_t nb_tx_queue_stats_mappings;
-extern uint16_t nb_rx_queue_stats_mappings;
-
 extern uint8_t xstats_hide_zero; /**< Hide zero values for xstats display */
 
 /* globals used for configuration */
@@ -790,7 +769,6 @@ void nic_stats_display(portid_t port_id);
 void nic_stats_clear(portid_t port_id);
 void nic_xstats_display(portid_t port_id);
 void nic_xstats_clear(portid_t port_id);
-void nic_stats_mapping_display(portid_t port_id);
 void device_infos_display(const char *identifier);
 void port_infos_display(portid_t port_id);
 void port_summary_display(portid_t port_id);
Signature · 1 line
2.7.4
Reviewed-by on v1 1/1 Applied
#17 lore raw

Re: [dpdk-dev] [PATCH V1] app/testpmd: fix queue stats mapping configuration

Quoted 31 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following
simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats. Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")

Applied to dpdk-next-net/main, thanks.

Thanks for this clean up.

#18 lore raw

Re: [dpdk-dev] [RFC V4] app/testpmd: fix queue stats mapping configuration

In reply to Ferruh Yigit

Hi, Ferruh,
I have sent V1 patch, please check it out, thanks;

https://patches.dpdk.org/patch/84716/

Quoted 79 lines · Ferruh Yigit
Quoted 3 lines · Min Hu (Connor)

Hi, Ferruh, and all,
     are there any comments about this set of patch?

I am good with this one, would you mind sending an actual patch (not RFC) on top of the latest head?

Thanks,
ferruh

Quoted 59 lines · Ferruh Yigit
Quoted 58 lines · Stephen Hemminger
Quoted 57 lines
Quoted 54 lines · On Fri, 20 Nov 2020 23:33:40 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 45 lines · Stephen Hemminger
Quoted 43 lines · On Fri, 20 Nov 2020 17:26:55 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 31 lines · Min Hu (Connor)

From: Huisong Li <[email protected]>

Currently, the queue stats mapping has the following problems:
1) Many PMD drivers don't support queue stats mapping. But there is no failure message after executing the command "set stat_qmap rx 0 2 2".
2) Once queue mapping is set, unrelated and unmapped queues are also displayed.
3) The configuration result does not take effect or can not be queried in real time.
4) The mapping arrays, "tx_queue_stats_mappings_array" & "rx_queue_stats_mappings_array" are global and their sizes are based on fixed max port and queue size assumptions.
5) These record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions are redundant for majority of drivers.
6) The display of the queue stats and queue stats mapping is mixed together.

Since xstats is used to obtain queue statistics, we have made the following simplifications and adjustments:
1) If PMD requires and supports queue stats mapping, configure to driver in real time by calling ethdev API after executing the command "set stat_qmap rx/tx ...". If not, the command can not be accepted.
2) Based on the above adjustments, these record structures, 'map_port_queue_stats_mapping_registers()' and its sub functions can be removed. "tx-queue-stats-mapping" & "rx-queue-stats-mapping" parameters, and 'parse_queue_stats_mapping_config()' can be removed too.
3) remove display of queue stats mapping in 'fwd_stats_display()' & 'nic_stats_display()', and obtain queue stats by xstats. Since the record structures are removed, 'nic_stats_mapping_display()' can be deleted.

Quoted 5 lines
Fixes 4dccdc789bf4b ("app/testpmd: simplify handling of stats mappings error")
Fixes 013af9b6b64f6 ("app/testpmd: various updates")
Fixes ed30d9b691b21 ("app/testpmd: add stats per queue")

Overall    looks good to me.
I did a quick test didn't see anything unexpected. 'xstats' or 'dpdk-proc-info' app still can be used to get queue stats and "set stat_qmap ..." is working as expected.

Quoted 4 lines
Quoted 2 lines

But it is a little late for this release cycle, would you be OK to get this at the beginning of next release?

Could we plan to deprecate queue stats mapping in future when xstats work is done?

Even queue stats moved to xstats, a few PMDs still need this configuration and API. And this patch already cleans the queue stats mapping noise from testpmd.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 1 line

What is the benefit/motivation to deprecate the queue stats mapping API?

Mostly because so few drivers implement it, that any application using it is  going to be non-portable.

'rte_eth_dev_set_.*_queue_stats_mapping()' APIs return '-ENOTSUP' if underlying PMD doesn't support it, applications can check this for portable code. .

.

#19 lore raw

RFC v2 2/2 app/testpmd: fix starting failed with queue-stats-mapping

In reply to Min Hu (Connor)

From: Huisong Li <[email protected]>

testpmd fails to start with "--rx-queue-stats-mapping" and "--tx-queue-stats-mapping", which is caused by the failure and exit of 'map_port_queue_stats_mapping_registers' function. By default, the configuration of queue statistics mapping in the initialization process is implemented in the 'init_port_config' function. However, the dev_configure interface is not called to configure the NIC and dev->data-nb_rx/tx_queues is zero. As a result, 'rte_eth_dev_set_tx/rx_queue_stats_mapping' function fails to verify the queue_id. Therefore, it is necessary to move 'map_port_queue_stats_mapping_registers' from 'init_port_config' to 'start_port' function.

Fixes 013af9b6b64f6e7 ("app/testpmd: various updates")
---
 app/test-pmd/testpmd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
app/test-pmd/testpmd.c +1 -1
index 86e3271..e4fe2a6 100644
@@ -2742,6 +2742,7 @@ start_port(portid_t pid)
 		need_check_link_status = 1;
 
 		pl[cfg_pi++] = pi;
+		map_port_queue_stats_mapping_registers(pi, port);
 	}
 
 	if (need_check_link_status == 1 && !no_link_check)
@@ -3600,7 +3601,6 @@ init_port_config(void)
 		if (ret != 0)
 			return;
 
-		map_port_queue_stats_mapping_registers(pid, port);
 #if defined RTE_LIBRTE_IXGBE_PMD && defined RTE_LIBRTE_IXGBE_BYPASS
 		rte_pmd_ixgbe_bypass_init(pid);
 #endif
Signature · 1 line
2.7.4