Skip to contentSkip to conversation

Enable ETS-based TX QoS on PF

PATCH Superseded Wenjun Wu 3 participants 109 messages Mar 29, 2022 – May 17, 2022

Subsystems (2)
lore
Share & tools
lore

Wenjun Wu

author
#0 lore raw

cover Enable ETS-based TX QoS on PF

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

Ting Xu (1):
net/ice: support queue bandwidth limit

Wenjun Wu (8):
net/ice/base: fix dead lock issue when getting node from ID type net/base/ice: support priority configuration of the exact node net/ice/base: support queue BW allocation configuration net/ice: support queue group bandwidth limit
net/ice: support queue priority configuration net/ice: support queue weight configuration
net/ice: support queue group priority configuration net/ice: add warning log for unsupported configuration

 drivers/net/ice/base/ice_sched.c |  89 +++-
 drivers/net/ice/base/ice_sched.h |   6 +
 drivers/net/ice/ice_ethdev.c     |  19 +
 drivers/net/ice/ice_ethdev.h     |  55 ++
 drivers/net/ice/ice_tm.c         | 845 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   1 +
 6 files changed, 1013 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#1 lore raw

PATCH 1/9 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#2 lore raw

PATCH 2/9 net/base/ice: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 21 +++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 24 insertions(+)
drivers/net/ice/base/ice_sched.c +21 -0
index e697c579be..c0f90b762b 100644
@@ -3613,6 +3613,27 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_cfg_node_priority - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi, struct ice_sched_node *node,
+		      u8 priority)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..e1dc6e18a4 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi,
+		      struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#3 lore raw

PATCH 3/9 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index c0f90b762b..4b7fdb2f13 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_node_priority - config priority of node
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index e1dc6e18a4..454a1570bb 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_node_priority(struct ice_port_info *pi,
 		      struct ice_sched_node *node, u8 priority);
 enum ice_status
Signature · 1 line
2.25.1
author
#4 lore raw

PATCH 4/9 net/ice: support queue bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues. Only TC0 is valid.

---
 drivers/net/ice/ice_ethdev.c |  19 ++
 drivers/net/ice/ice_ethdev.h |  48 +++
 drivers/net/ice/ice_tm.c     | 599 +++++++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build  |   1 +
 4 files changed, 667 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
drivers/net/ice/ice_ethdev.c +19 -0
index 13adcf90ed..37897765c8 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2312,6 +2325,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2492,6 +2508,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +48 -0

76 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +599 -0

599 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#5 lore raw

PATCH 5/9 net/ice: support queue group bandwidth limit

In reply to Wenjun Wu

To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes.

This patch adds queue group configuration support and queue group bandwidth limit support.

---
 drivers/net/ice/ice_ethdev.h |   9 +-
 drivers/net/ice/ice_tm.c     | 240 ++++++++++++++++++++++++++++++++---
 2 files changed, 233 insertions(+), 16 deletions(-)
drivers/net/ice/ice_ethdev.h +8 -1
index 0841e1866c..6ddbcc9972 100644
@@ -474,6 +474,7 @@ struct ice_tm_node {
 	uint32_t weight;
 	uint32_t reference_count;
 	struct ice_tm_node *parent;
+	struct ice_tm_node **children;
 	struct ice_tm_shaper_profile *shaper_profile;
 	struct rte_tm_node_params params;
 };
@@ -482,6 +483,8 @@ struct ice_tm_node {
 enum ice_tm_node_type {
 	ICE_TM_NODE_TYPE_PORT,
 	ICE_TM_NODE_TYPE_TC,
+	ICE_TM_NODE_TYPE_VSI,
+	ICE_TM_NODE_TYPE_QGROUP,
 	ICE_TM_NODE_TYPE_QUEUE,
 	ICE_TM_NODE_TYPE_MAX,
 };
@@ -489,10 +492,14 @@ enum ice_tm_node_type {
 /* Struct to store all the Traffic Manager configuration. */
 struct ice_tm_conf {
 	struct ice_shaper_profile_list shaper_profile_list;
-	struct ice_tm_node *root; /* root node - vf vsi */
+	struct ice_tm_node *root; /* root node - port */
 	struct ice_tm_node_list tc_list; /* node list for all the TCs */
+	struct ice_tm_node_list vsi_list; /* node list for all the VSIs */
+	struct ice_tm_node_list qgroup_list; /* node list for all the queue groups */
 	struct ice_tm_node_list queue_list; /* node list for all the queues */
 	uint32_t nb_tc_node;
+	uint32_t nb_vsi_node;
+	uint32_t nb_qgroup_node;
 	uint32_t nb_queue_node;
 	bool committed;
 };
drivers/net/ice/ice_tm.c +225 -15

348 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#6 lore raw

PATCH 6/9 net/ice: support queue priority configuration

In reply to Wenjun Wu

This patch adds queue priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 drivers/net/ice/ice_tm.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +12 -2
index 33f3aae5be..9552158b53 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -780,6 +781,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -795,6 +797,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#7 lore raw

PATCH 7/9 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 drivers/net/ice/ice_tm.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +11 -2
index 9552158b53..36981f78dd 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -805,6 +805,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#8 lore raw

PATCH 8/9 net/ice: support queue group priority configuration

In reply to Wenjun Wu

This patch adds queue group priority configuration support.

---
 drivers/net/ice/ice_tm.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/net/ice/ice_tm.c +8 -0
index 36981f78dd..fdbb415eda 100644
@@ -765,6 +765,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_node_priority(hw->port_info, qgroup_sched_node, priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
Signature · 1 line
2.25.1
author
#9 lore raw

PATCH 9/9 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index fdbb415eda..d214b6bc91 100644
@@ -515,6 +515,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 		}
 	}
 
+	if (tm_node->priority != 0 && (level_id != ICE_TM_NODE_TYPE_QUEUE ||
+	    level_id != ICE_TM_NODE_TYPE_QGROUP))
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	/* add the TC or VSI or queue group or queue node */
 	tm_node = rte_zmalloc("ice_tm_node",
 			      sizeof(struct ice_tm_node),
Signature · 1 line
2.25.1
author
#10 lore raw

v2 cover Enable ETS-based TX QoS on PF

In reply to Wenjun Wu

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.

Ting Xu (1):
net/ice: support queue bandwidth limit

Wenjun Wu (8):
net/ice/base: fix dead lock issue when getting node from ID type net/base/ice: support priority configuration of the exact node net/ice/base: support queue BW allocation configuration net/ice: support queue group bandwidth limit
net/ice: support queue priority configuration net/ice: support queue weight configuration
net/ice: support queue group priority configuration net/ice: add warning log for unsupported configuration

 drivers/net/ice/base/ice_sched.c |  89 +++-
 drivers/net/ice/base/ice_sched.h |   6 +
 drivers/net/ice/ice_ethdev.c     |  19 +
 drivers/net/ice/ice_ethdev.h     |  55 ++
 drivers/net/ice/ice_tm.c         | 844 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   1 +
 6 files changed, 1012 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#11 lore raw

PATCH v2 1/9 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#12 lore raw

PATCH v2 2/9 net/base/ice: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 21 +++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 24 insertions(+)
drivers/net/ice/base/ice_sched.c +21 -0
index e697c579be..c0f90b762b 100644
@@ -3613,6 +3613,27 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_cfg_node_priority - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi, struct ice_sched_node *node,
+		      u8 priority)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..e1dc6e18a4 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi,
+		      struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#13 lore raw

PATCH v2 3/9 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index c0f90b762b..4b7fdb2f13 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_node_priority - config priority of node
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index e1dc6e18a4..454a1570bb 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_node_priority(struct ice_port_info *pi,
 		      struct ice_sched_node *node, u8 priority);
 enum ice_status
Signature · 1 line
2.25.1
author
#14 lore raw

PATCH v2 4/9 net/ice: support queue bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues. Only TC0 is valid.

---
 drivers/net/ice/ice_ethdev.c |  19 ++
 drivers/net/ice/ice_ethdev.h |  48 +++
 drivers/net/ice/ice_tm.c     | 599 +++++++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build  |   1 +
 4 files changed, 667 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
drivers/net/ice/ice_ethdev.c +19 -0
index 13adcf90ed..37897765c8 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2312,6 +2325,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2492,6 +2508,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +48 -0

76 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +599 -0

599 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#15 lore raw

PATCH v2 5/9 net/ice: support queue group bandwidth limit

In reply to Wenjun Wu

To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes.

This patch adds queue group configuration support and queue group bandwidth limit support.

---
 drivers/net/ice/ice_ethdev.h |   9 +-
 drivers/net/ice/ice_tm.c     | 239 ++++++++++++++++++++++++++++++++---
 2 files changed, 232 insertions(+), 16 deletions(-)
drivers/net/ice/ice_ethdev.h +8 -1
index 0841e1866c..6ddbcc9972 100644
@@ -474,6 +474,7 @@ struct ice_tm_node {
 	uint32_t weight;
 	uint32_t reference_count;
 	struct ice_tm_node *parent;
+	struct ice_tm_node **children;
 	struct ice_tm_shaper_profile *shaper_profile;
 	struct rte_tm_node_params params;
 };
@@ -482,6 +483,8 @@ struct ice_tm_node {
 enum ice_tm_node_type {
 	ICE_TM_NODE_TYPE_PORT,
 	ICE_TM_NODE_TYPE_TC,
+	ICE_TM_NODE_TYPE_VSI,
+	ICE_TM_NODE_TYPE_QGROUP,
 	ICE_TM_NODE_TYPE_QUEUE,
 	ICE_TM_NODE_TYPE_MAX,
 };
@@ -489,10 +492,14 @@ enum ice_tm_node_type {
 /* Struct to store all the Traffic Manager configuration. */
 struct ice_tm_conf {
 	struct ice_shaper_profile_list shaper_profile_list;
-	struct ice_tm_node *root; /* root node - vf vsi */
+	struct ice_tm_node *root; /* root node - port */
 	struct ice_tm_node_list tc_list; /* node list for all the TCs */
+	struct ice_tm_node_list vsi_list; /* node list for all the VSIs */
+	struct ice_tm_node_list qgroup_list; /* node list for all the queue groups */
 	struct ice_tm_node_list queue_list; /* node list for all the queues */
 	uint32_t nb_tc_node;
+	uint32_t nb_vsi_node;
+	uint32_t nb_qgroup_node;
 	uint32_t nb_queue_node;
 	bool committed;
 };
drivers/net/ice/ice_tm.c +224 -15

347 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#16 lore raw

PATCH v2 6/9 net/ice: support queue priority configuration

In reply to Wenjun Wu

This patch adds queue priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 drivers/net/ice/ice_tm.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +12 -2
index 83dbc09505..5ebf9abeb9 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -779,6 +780,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -794,6 +796,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#17 lore raw

PATCH v2 7/9 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 drivers/net/ice/ice_tm.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +11 -2
index 5ebf9abeb9..070cb7db41 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -804,6 +804,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#18 lore raw

PATCH v2 8/9 net/ice: support queue group priority configuration

In reply to Wenjun Wu

This patch adds queue group priority configuration support.

---
 drivers/net/ice/ice_tm.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/net/ice/ice_tm.c +8 -0
index 070cb7db41..8a71bfa599 100644
@@ -764,6 +764,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_node_priority(hw->port_info, qgroup_sched_node, priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
Signature · 1 line
2.25.1
author
#19 lore raw

PATCH v2 9/9 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index 8a71bfa599..9fa959b4a0 100644
@@ -515,6 +515,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 		}
 	}
 
+	if (tm_node->priority != 0 && (level_id != ICE_TM_NODE_TYPE_QUEUE ||
+	    level_id != ICE_TM_NODE_TYPE_QGROUP))
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	/* add the TC or VSI or queue group or queue node */
 	tm_node = rte_zmalloc("ice_tm_node",
 			      sizeof(struct ice_tm_node),
Signature · 1 line
2.25.1
author
#20 lore raw

v3 cover Enable ETS-based TX QoS on PF

In reply to Wenjun Wu

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue

Ting Xu (1):
net/ice: support queue bandwidth limit

Wenjun Wu (8):
net/ice/base: fix dead lock issue when getting node from ID type net/base/ice: support priority configuration of the exact node net/ice/base: support queue BW allocation configuration net/ice: support queue group bandwidth limit
net/ice: support queue priority configuration net/ice: support queue weight configuration
net/ice: support queue group priority configuration net/ice: add warning log for unsupported configuration

 drivers/net/ice/base/ice_sched.c |  89 +++-
 drivers/net/ice/base/ice_sched.h |   6 +
 drivers/net/ice/ice_ethdev.c     |  19 +
 drivers/net/ice/ice_ethdev.h     |  55 ++
 drivers/net/ice/ice_tm.c         | 844 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   1 +
 6 files changed, 1012 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#21 lore raw

PATCH v3 1/9 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#22 lore raw

PATCH v3 2/9 net/base/ice: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 21 +++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 24 insertions(+)
drivers/net/ice/base/ice_sched.c +21 -0
index e697c579be..c0f90b762b 100644
@@ -3613,6 +3613,27 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_cfg_node_priority - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi, struct ice_sched_node *node,
+		      u8 priority)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..e1dc6e18a4 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi,
+		      struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#23 lore raw

PATCH v3 3/9 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index c0f90b762b..4b7fdb2f13 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_node_priority - config priority of node
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index e1dc6e18a4..454a1570bb 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_node_priority(struct ice_port_info *pi,
 		      struct ice_sched_node *node, u8 priority);
 enum ice_status
Signature · 1 line
2.25.1
author
#24 lore raw

PATCH v3 4/9 net/ice: support queue bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues. Only TC0 is valid.

---
 drivers/net/ice/ice_ethdev.c |  19 ++
 drivers/net/ice/ice_ethdev.h |  48 +++
 drivers/net/ice/ice_tm.c     | 599 +++++++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build  |   1 +
 4 files changed, 667 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
drivers/net/ice/ice_ethdev.c +19 -0
index 13adcf90ed..37897765c8 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2312,6 +2325,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2492,6 +2508,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +48 -0

76 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +599 -0

599 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#25 lore raw

PATCH v3 5/9 net/ice: support queue group bandwidth limit

In reply to Wenjun Wu

To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes.

This patch adds queue group configuration support and queue group bandwidth limit support.

---
 drivers/net/ice/ice_ethdev.h |   9 +-
 drivers/net/ice/ice_tm.c     | 239 ++++++++++++++++++++++++++++++++---
 2 files changed, 232 insertions(+), 16 deletions(-)
drivers/net/ice/ice_ethdev.h +8 -1
index 0841e1866c..6ddbcc9972 100644
@@ -474,6 +474,7 @@ struct ice_tm_node {
 	uint32_t weight;
 	uint32_t reference_count;
 	struct ice_tm_node *parent;
+	struct ice_tm_node **children;
 	struct ice_tm_shaper_profile *shaper_profile;
 	struct rte_tm_node_params params;
 };
@@ -482,6 +483,8 @@ struct ice_tm_node {
 enum ice_tm_node_type {
 	ICE_TM_NODE_TYPE_PORT,
 	ICE_TM_NODE_TYPE_TC,
+	ICE_TM_NODE_TYPE_VSI,
+	ICE_TM_NODE_TYPE_QGROUP,
 	ICE_TM_NODE_TYPE_QUEUE,
 	ICE_TM_NODE_TYPE_MAX,
 };
@@ -489,10 +492,14 @@ enum ice_tm_node_type {
 /* Struct to store all the Traffic Manager configuration. */
 struct ice_tm_conf {
 	struct ice_shaper_profile_list shaper_profile_list;
-	struct ice_tm_node *root; /* root node - vf vsi */
+	struct ice_tm_node *root; /* root node - port */
 	struct ice_tm_node_list tc_list; /* node list for all the TCs */
+	struct ice_tm_node_list vsi_list; /* node list for all the VSIs */
+	struct ice_tm_node_list qgroup_list; /* node list for all the queue groups */
 	struct ice_tm_node_list queue_list; /* node list for all the queues */
 	uint32_t nb_tc_node;
+	uint32_t nb_vsi_node;
+	uint32_t nb_qgroup_node;
 	uint32_t nb_queue_node;
 	bool committed;
 };
drivers/net/ice/ice_tm.c +224 -15

347 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#26 lore raw

PATCH v3 6/9 net/ice: support queue priority configuration

In reply to Wenjun Wu

This patch adds queue priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 drivers/net/ice/ice_tm.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +12 -2
index 83dbc09505..5ebf9abeb9 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -779,6 +780,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -794,6 +796,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#27 lore raw

PATCH v3 7/9 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 drivers/net/ice/ice_tm.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +11 -2
index 5ebf9abeb9..070cb7db41 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -804,6 +804,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#28 lore raw

PATCH v3 8/9 net/ice: support queue group priority configuration

In reply to Wenjun Wu

This patch adds queue group priority configuration support.

---
 drivers/net/ice/ice_tm.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/net/ice/ice_tm.c +8 -0
index 070cb7db41..8a71bfa599 100644
@@ -764,6 +764,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_node_priority(hw->port_info, qgroup_sched_node, priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
Signature · 1 line
2.25.1
author
#29 lore raw

PATCH v3 9/9 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index 8a71bfa599..8ae0b3930c 100644
@@ -531,6 +531,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 			rte_calloc(NULL, 256, (sizeof(struct ice_tm_node *)), 0);
 	tm_node->parent->children[tm_node->parent->reference_count] = tm_node;
 
+	if (tm_node->priority != 0 && (level_id != ICE_TM_NODE_TYPE_QUEUE ||
+	    level_id != ICE_TM_NODE_TYPE_QGROUP))
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	rte_memcpy(&tm_node->params, params,
 			 sizeof(struct rte_tm_node_params));
 	if (parent_node_type == ICE_TM_NODE_TYPE_PORT) {
Signature · 1 line
2.25.1
author
#30 lore raw

v4 cover Enable ETS-based TX QoS on PF

In reply to Wenjun Wu

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue
v4: fix logical issue

Ting Xu (1):
net/ice: support queue bandwidth limit

Wenjun Wu (8):
net/ice/base: fix dead lock issue when getting node from ID type net/base/ice: support priority configuration of the exact node net/ice/base: support queue BW allocation configuration net/ice: support queue group bandwidth limit
net/ice: support queue priority configuration net/ice: support queue weight configuration
net/ice: support queue group priority configuration net/ice: add warning log for unsupported configuration

 drivers/net/ice/base/ice_sched.c |  89 +++-
 drivers/net/ice/base/ice_sched.h |   6 +
 drivers/net/ice/ice_ethdev.c     |  19 +
 drivers/net/ice/ice_ethdev.h     |  55 ++
 drivers/net/ice/ice_tm.c         | 844 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   1 +
 6 files changed, 1012 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#31 lore raw

PATCH v4 1/9 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#32 lore raw

PATCH v4 2/9 net/base/ice: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 21 +++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 24 insertions(+)
drivers/net/ice/base/ice_sched.c +21 -0
index e697c579be..c0f90b762b 100644
@@ -3613,6 +3613,27 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_cfg_node_priority - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi, struct ice_sched_node *node,
+		      u8 priority)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..e1dc6e18a4 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi,
+		      struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#33 lore raw

PATCH v4 3/9 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index c0f90b762b..4b7fdb2f13 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_node_priority - config priority of node
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index e1dc6e18a4..454a1570bb 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_node_priority(struct ice_port_info *pi,
 		      struct ice_sched_node *node, u8 priority);
 enum ice_status
Signature · 1 line
2.25.1
author
#34 lore raw

PATCH v4 4/9 net/ice: support queue bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues. Only TC0 is valid.

---
 drivers/net/ice/ice_ethdev.c |  19 ++
 drivers/net/ice/ice_ethdev.h |  48 +++
 drivers/net/ice/ice_tm.c     | 599 +++++++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build  |   1 +
 4 files changed, 667 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
drivers/net/ice/ice_ethdev.c +19 -0
index 13adcf90ed..37897765c8 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2312,6 +2325,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2492,6 +2508,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +48 -0

76 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +599 -0

599 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#35 lore raw

PATCH v4 5/9 net/ice: support queue group bandwidth limit

In reply to Wenjun Wu

To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes.

This patch adds queue group configuration support and queue group bandwidth limit support.

---
 drivers/net/ice/ice_ethdev.h |   9 +-
 drivers/net/ice/ice_tm.c     | 239 ++++++++++++++++++++++++++++++++---
 2 files changed, 232 insertions(+), 16 deletions(-)
drivers/net/ice/ice_ethdev.h +8 -1
index 0841e1866c..6ddbcc9972 100644
@@ -474,6 +474,7 @@ struct ice_tm_node {
 	uint32_t weight;
 	uint32_t reference_count;
 	struct ice_tm_node *parent;
+	struct ice_tm_node **children;
 	struct ice_tm_shaper_profile *shaper_profile;
 	struct rte_tm_node_params params;
 };
@@ -482,6 +483,8 @@ struct ice_tm_node {
 enum ice_tm_node_type {
 	ICE_TM_NODE_TYPE_PORT,
 	ICE_TM_NODE_TYPE_TC,
+	ICE_TM_NODE_TYPE_VSI,
+	ICE_TM_NODE_TYPE_QGROUP,
 	ICE_TM_NODE_TYPE_QUEUE,
 	ICE_TM_NODE_TYPE_MAX,
 };
@@ -489,10 +492,14 @@ enum ice_tm_node_type {
 /* Struct to store all the Traffic Manager configuration. */
 struct ice_tm_conf {
 	struct ice_shaper_profile_list shaper_profile_list;
-	struct ice_tm_node *root; /* root node - vf vsi */
+	struct ice_tm_node *root; /* root node - port */
 	struct ice_tm_node_list tc_list; /* node list for all the TCs */
+	struct ice_tm_node_list vsi_list; /* node list for all the VSIs */
+	struct ice_tm_node_list qgroup_list; /* node list for all the queue groups */
 	struct ice_tm_node_list queue_list; /* node list for all the queues */
 	uint32_t nb_tc_node;
+	uint32_t nb_vsi_node;
+	uint32_t nb_qgroup_node;
 	uint32_t nb_queue_node;
 	bool committed;
 };
drivers/net/ice/ice_tm.c +224 -15

347 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#36 lore raw

PATCH v4 6/9 net/ice: support queue priority configuration

In reply to Wenjun Wu

This patch adds queue priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 drivers/net/ice/ice_tm.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +12 -2
index 83dbc09505..5ebf9abeb9 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -779,6 +780,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -794,6 +796,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#37 lore raw

PATCH v4 7/9 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 drivers/net/ice/ice_tm.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +11 -2
index 5ebf9abeb9..070cb7db41 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -804,6 +804,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#38 lore raw

PATCH v4 8/9 net/ice: support queue group priority configuration

In reply to Wenjun Wu

This patch adds queue group priority configuration support.

---
 drivers/net/ice/ice_tm.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/net/ice/ice_tm.c +8 -0
index 070cb7db41..8a71bfa599 100644
@@ -764,6 +764,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_node_priority(hw->port_info, qgroup_sched_node, priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
Signature · 1 line
2.25.1
author
#39 lore raw

PATCH v4 9/9 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index 8a71bfa599..3ae60e5594 100644
@@ -531,6 +531,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 			rte_calloc(NULL, 256, (sizeof(struct ice_tm_node *)), 0);
 	tm_node->parent->children[tm_node->parent->reference_count] = tm_node;
 
+	if (tm_node->priority != 0 && level_id != ICE_TM_NODE_TYPE_QUEUE &&
+	    level_id != ICE_TM_NODE_TYPE_QGROUP)
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	rte_memcpy(&tm_node->params, params,
 			 sizeof(struct rte_tm_node_params));
 	if (parent_node_type == ICE_TM_NODE_TYPE_PORT) {
Signature · 1 line
2.25.1
author
#40 lore raw

v5 cover Enable ETS-based TX QoS on PF

In reply to Wenjun Wu

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue
v4: fix logical issue
v5: fix CI testing issue. Add explicit cast.

Ting Xu (1):
net/ice: support queue bandwidth limit

Wenjun Wu (8):
net/ice/base: fix dead lock issue when getting node from ID type net/ice/base: support priority configuration of the exact node net/ice/base: support queue BW allocation configuration net/ice: support queue group bandwidth limit
net/ice: support queue priority configuration net/ice: support queue weight configuration
net/ice: support queue group priority configuration net/ice: add warning log for unsupported configuration

 drivers/net/ice/base/ice_sched.c |  89 +++-
 drivers/net/ice/base/ice_sched.h |   6 +
 drivers/net/ice/ice_ethdev.c     |  19 +
 drivers/net/ice/ice_ethdev.h     |  55 ++
 drivers/net/ice/ice_tm.c         | 844 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   1 +
 6 files changed, 1012 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#41 lore raw

PATCH v5 1/9 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#42 lore raw

PATCH v5 2/9 net/ice/base: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 21 +++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 24 insertions(+)
drivers/net/ice/base/ice_sched.c +21 -0
index e697c579be..c0f90b762b 100644
@@ -3613,6 +3613,27 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_cfg_node_priority - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi, struct ice_sched_node *node,
+		      u8 priority)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..e1dc6e18a4 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi,
+		      struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#43 lore raw

PATCH v5 3/9 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index c0f90b762b..4b7fdb2f13 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_node_priority - config priority of node
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index e1dc6e18a4..454a1570bb 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_node_priority(struct ice_port_info *pi,
 		      struct ice_sched_node *node, u8 priority);
 enum ice_status
Signature · 1 line
2.25.1
author
#44 lore raw

PATCH v5 4/9 net/ice: support queue bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues. Only TC0 is valid.

---
 drivers/net/ice/ice_ethdev.c |  19 ++
 drivers/net/ice/ice_ethdev.h |  48 +++
 drivers/net/ice/ice_tm.c     | 599 +++++++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build  |   1 +
 4 files changed, 667 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
drivers/net/ice/ice_ethdev.c +19 -0
index 13adcf90ed..37897765c8 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2312,6 +2325,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2492,6 +2508,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +48 -0

76 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +599 -0

599 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#45 lore raw

PATCH v5 5/9 net/ice: support queue group bandwidth limit

In reply to Wenjun Wu

To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes.

This patch adds queue group configuration support and queue group bandwidth limit support.

---
 drivers/net/ice/ice_ethdev.h |   9 +-
 drivers/net/ice/ice_tm.c     | 239 ++++++++++++++++++++++++++++++++---
 2 files changed, 232 insertions(+), 16 deletions(-)
drivers/net/ice/ice_ethdev.h +8 -1
index 0841e1866c..6ddbcc9972 100644
@@ -474,6 +474,7 @@ struct ice_tm_node {
 	uint32_t weight;
 	uint32_t reference_count;
 	struct ice_tm_node *parent;
+	struct ice_tm_node **children;
 	struct ice_tm_shaper_profile *shaper_profile;
 	struct rte_tm_node_params params;
 };
@@ -482,6 +483,8 @@ struct ice_tm_node {
 enum ice_tm_node_type {
 	ICE_TM_NODE_TYPE_PORT,
 	ICE_TM_NODE_TYPE_TC,
+	ICE_TM_NODE_TYPE_VSI,
+	ICE_TM_NODE_TYPE_QGROUP,
 	ICE_TM_NODE_TYPE_QUEUE,
 	ICE_TM_NODE_TYPE_MAX,
 };
@@ -489,10 +492,14 @@ enum ice_tm_node_type {
 /* Struct to store all the Traffic Manager configuration. */
 struct ice_tm_conf {
 	struct ice_shaper_profile_list shaper_profile_list;
-	struct ice_tm_node *root; /* root node - vf vsi */
+	struct ice_tm_node *root; /* root node - port */
 	struct ice_tm_node_list tc_list; /* node list for all the TCs */
+	struct ice_tm_node_list vsi_list; /* node list for all the VSIs */
+	struct ice_tm_node_list qgroup_list; /* node list for all the queue groups */
 	struct ice_tm_node_list queue_list; /* node list for all the queues */
 	uint32_t nb_tc_node;
+	uint32_t nb_vsi_node;
+	uint32_t nb_qgroup_node;
 	uint32_t nb_queue_node;
 	bool committed;
 };
drivers/net/ice/ice_tm.c +224 -15

347 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#46 lore raw

PATCH v5 6/9 net/ice: support queue priority configuration

In reply to Wenjun Wu

This patch adds queue priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 drivers/net/ice/ice_tm.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +12 -2
index d70d077286..91e420d653 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -779,6 +780,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -794,6 +796,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#47 lore raw

PATCH v5 7/9 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 drivers/net/ice/ice_tm.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +11 -2
index 91e420d653..4d7bb9102c 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -804,6 +804,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#48 lore raw

PATCH v5 8/9 net/ice: support queue group priority configuration

In reply to Wenjun Wu

This patch adds queue group priority configuration support.

---
 drivers/net/ice/ice_tm.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/net/ice/ice_tm.c +8 -0
index 4d7bb9102c..17f369994b 100644
@@ -764,6 +764,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_node_priority(hw->port_info, qgroup_sched_node, priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
Signature · 1 line
2.25.1
author
#49 lore raw

PATCH v5 9/9 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index 17f369994b..3e98c2f01e 100644
@@ -531,6 +531,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 			rte_calloc(NULL, 256, (sizeof(struct ice_tm_node *)), 0);
 	tm_node->parent->children[tm_node->parent->reference_count] = tm_node;
 
+	if (tm_node->priority != 0 && level_id != ICE_TM_NODE_TYPE_QUEUE &&
+	    level_id != ICE_TM_NODE_TYPE_QGROUP)
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	rte_memcpy(&tm_node->params, params,
 			 sizeof(struct rte_tm_node_params));
 	if (parent_node_type == ICE_TM_NODE_TYPE_PORT) {
Signature · 1 line
2.25.1
author
#50 lore raw

v6 cover Enable ETS-based TX QoS on PF

In reply to Wenjun Wu

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue.
v4: fix logical issue.
v5: fix CI testing issue. Add explicit cast.
v6: add release note.

Ting Xu (1):
net/ice: support queue bandwidth limit

Wenjun Wu (9):
net/ice/base: fix dead lock issue when getting node from ID type net/ice/base: support priority configuration of the exact node net/ice/base: support queue BW allocation configuration net/ice: support queue group bandwidth limit
net/ice: support queue priority configuration net/ice: support queue weight configuration
net/ice: support queue group priority configuration net/ice: add warning log for unsupported configuration
doc: add release notes for 22.07

 doc/guides/rel_notes/release_22_07.rst |   4 +
 drivers/net/ice/base/ice_sched.c       |  89 ++-
 drivers/net/ice/base/ice_sched.h       |   6 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  55 ++
 drivers/net/ice/ice_tm.c               | 844 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 7 files changed, 1016 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#51 lore raw

PATCH v6 1/10 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#52 lore raw

PATCH v6 2/10 net/ice/base: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 21 +++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 24 insertions(+)
drivers/net/ice/base/ice_sched.c +21 -0
index e697c579be..c0f90b762b 100644
@@ -3613,6 +3613,27 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_cfg_node_priority - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi, struct ice_sched_node *node,
+		      u8 priority)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..e1dc6e18a4 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi,
+		      struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#53 lore raw

PATCH v6 3/10 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index c0f90b762b..4b7fdb2f13 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_node_priority - config priority of node
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index e1dc6e18a4..454a1570bb 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_node_priority(struct ice_port_info *pi,
 		      struct ice_sched_node *node, u8 priority);
 enum ice_status
Signature · 1 line
2.25.1
author
#54 lore raw

PATCH v6 4/10 net/ice: support queue bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues. Only TC0 is valid.

---
 drivers/net/ice/ice_ethdev.c |  19 ++
 drivers/net/ice/ice_ethdev.h |  48 +++
 drivers/net/ice/ice_tm.c     | 599 +++++++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build  |   1 +
 4 files changed, 667 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
drivers/net/ice/ice_ethdev.c +19 -0
index 13adcf90ed..37897765c8 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2312,6 +2325,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2492,6 +2508,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +48 -0

76 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +599 -0

599 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#55 lore raw

PATCH v6 5/10 net/ice: support queue group bandwidth limit

In reply to Wenjun Wu

To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes.

This patch adds queue group configuration support and queue group bandwidth limit support.

---
 drivers/net/ice/ice_ethdev.h |   9 +-
 drivers/net/ice/ice_tm.c     | 239 ++++++++++++++++++++++++++++++++---
 2 files changed, 232 insertions(+), 16 deletions(-)
drivers/net/ice/ice_ethdev.h +8 -1
index 0841e1866c..6ddbcc9972 100644
@@ -474,6 +474,7 @@ struct ice_tm_node {
 	uint32_t weight;
 	uint32_t reference_count;
 	struct ice_tm_node *parent;
+	struct ice_tm_node **children;
 	struct ice_tm_shaper_profile *shaper_profile;
 	struct rte_tm_node_params params;
 };
@@ -482,6 +483,8 @@ struct ice_tm_node {
 enum ice_tm_node_type {
 	ICE_TM_NODE_TYPE_PORT,
 	ICE_TM_NODE_TYPE_TC,
+	ICE_TM_NODE_TYPE_VSI,
+	ICE_TM_NODE_TYPE_QGROUP,
 	ICE_TM_NODE_TYPE_QUEUE,
 	ICE_TM_NODE_TYPE_MAX,
 };
@@ -489,10 +492,14 @@ enum ice_tm_node_type {
 /* Struct to store all the Traffic Manager configuration. */
 struct ice_tm_conf {
 	struct ice_shaper_profile_list shaper_profile_list;
-	struct ice_tm_node *root; /* root node - vf vsi */
+	struct ice_tm_node *root; /* root node - port */
 	struct ice_tm_node_list tc_list; /* node list for all the TCs */
+	struct ice_tm_node_list vsi_list; /* node list for all the VSIs */
+	struct ice_tm_node_list qgroup_list; /* node list for all the queue groups */
 	struct ice_tm_node_list queue_list; /* node list for all the queues */
 	uint32_t nb_tc_node;
+	uint32_t nb_vsi_node;
+	uint32_t nb_qgroup_node;
 	uint32_t nb_queue_node;
 	bool committed;
 };
drivers/net/ice/ice_tm.c +224 -15

347 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#56 lore raw

PATCH v6 6/10 net/ice: support queue priority configuration

In reply to Wenjun Wu

This patch adds queue priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 drivers/net/ice/ice_tm.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +12 -2
index d70d077286..91e420d653 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -779,6 +780,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -794,6 +796,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#57 lore raw

PATCH v6 7/10 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 drivers/net/ice/ice_tm.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +11 -2
index 91e420d653..4d7bb9102c 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -804,6 +804,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#58 lore raw

PATCH v6 8/10 net/ice: support queue group priority configuration

In reply to Wenjun Wu

This patch adds queue group priority configuration support.

---
 drivers/net/ice/ice_tm.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/net/ice/ice_tm.c +8 -0
index 4d7bb9102c..17f369994b 100644
@@ -764,6 +764,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_node_priority(hw->port_info, qgroup_sched_node, priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
Signature · 1 line
2.25.1
author
#59 lore raw

PATCH v6 9/10 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index 17f369994b..3e98c2f01e 100644
@@ -531,6 +531,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 			rte_calloc(NULL, 256, (sizeof(struct ice_tm_node *)), 0);
 	tm_node->parent->children[tm_node->parent->reference_count] = tm_node;
 
+	if (tm_node->priority != 0 && level_id != ICE_TM_NODE_TYPE_QUEUE &&
+	    level_id != ICE_TM_NODE_TYPE_QGROUP)
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	rte_memcpy(&tm_node->params, params,
 			 sizeof(struct rte_tm_node_params));
 	if (parent_node_type == ICE_TM_NODE_TYPE_PORT) {
Signature · 1 line
2.25.1
author
#60 lore raw

PATCH v6 10/10 doc: add release notes for 22.07

In reply to Wenjun Wu

Add support for ETS-based TX QoS on PF

---
 doc/guides/rel_notes/release_22_07.rst | 4 ++++
 1 file changed, 4 insertions(+)
doc/guides/rel_notes/release_22_07.rst +4 -0
index 42a5f2d990..2ce3c99fb8 100644
@@ -55,6 +55,10 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Intel ice driver.**
+
+  * Added Tx QoS rate limitation and priority configuration support for queue and queue group.
+  * Added TX QoS queue weight configuration support.
 
 Removed Items
 -------------
Signature · 1 line
2.25.1
author
#61 lore raw

v7 cover Enable ETS-based TX QoS on PF

In reply to Wenjun Wu

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue.
v4: fix logical issue.
v5: fix CI testing issue. Add explicit cast.
v6: add release note.
v7: merge the release note with the previous patch.

Ting Xu (1):
net/ice: support queue bandwidth limit

Wenjun Wu (8):
net/ice/base: fix dead lock issue when getting node from ID type net/ice/base: support priority configuration of the exact node net/ice/base: support queue BW allocation configuration net/ice: support queue group bandwidth limit
net/ice: support queue priority configuration net/ice: support queue weight configuration
net/ice: support queue group priority configuration net/ice: add warning log for unsupported configuration

 doc/guides/rel_notes/release_22_07.rst |   4 +
 drivers/net/ice/base/ice_sched.c       |  89 ++-
 drivers/net/ice/base/ice_sched.h       |   6 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  55 ++
 drivers/net/ice/ice_tm.c               | 844 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 7 files changed, 1016 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#62 lore raw

PATCH v7 1/9 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#63 lore raw

PATCH v7 2/9 net/ice/base: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 21 +++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 24 insertions(+)
drivers/net/ice/base/ice_sched.c +21 -0
index e697c579be..c0f90b762b 100644
@@ -3613,6 +3613,27 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_cfg_node_priority - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi, struct ice_sched_node *node,
+		      u8 priority)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..e1dc6e18a4 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_node_priority(struct ice_port_info *pi,
+		      struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#64 lore raw

PATCH v7 3/9 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index c0f90b762b..4b7fdb2f13 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_node_priority - config priority of node
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index e1dc6e18a4..454a1570bb 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_node_priority(struct ice_port_info *pi,
 		      struct ice_sched_node *node, u8 priority);
 enum ice_status
Signature · 1 line
2.25.1
author
#65 lore raw

PATCH v7 4/9 net/ice: support queue bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues. Only TC0 is valid.

---
 doc/guides/rel_notes/release_22_07.rst |   4 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  48 ++
 drivers/net/ice/ice_tm.c               | 599 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 5 files changed, 671 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
doc/guides/rel_notes/release_22_07.rst +4 -0
index 42a5f2d990..2ce3c99fb8 100644
@@ -55,6 +55,10 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Intel ice driver.**
+
+  * Added Tx QoS rate limitation and priority configuration support for queue and queue group.
+  * Added TX QoS queue weight configuration support.
 
 Removed Items
 -------------
drivers/net/ice/ice_ethdev.c +19 -0
index 13adcf90ed..37897765c8 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2312,6 +2325,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2492,6 +2508,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +48 -0

76 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +599 -0

599 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#66 lore raw

PATCH v7 5/9 net/ice: support queue group bandwidth limit

In reply to Wenjun Wu

To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes.

This patch adds queue group configuration support and queue group bandwidth limit support.

---
 drivers/net/ice/ice_ethdev.h |   9 +-
 drivers/net/ice/ice_tm.c     | 239 ++++++++++++++++++++++++++++++++---
 2 files changed, 232 insertions(+), 16 deletions(-)
drivers/net/ice/ice_ethdev.h +8 -1
index 0841e1866c..6ddbcc9972 100644
@@ -474,6 +474,7 @@ struct ice_tm_node {
 	uint32_t weight;
 	uint32_t reference_count;
 	struct ice_tm_node *parent;
+	struct ice_tm_node **children;
 	struct ice_tm_shaper_profile *shaper_profile;
 	struct rte_tm_node_params params;
 };
@@ -482,6 +483,8 @@ struct ice_tm_node {
 enum ice_tm_node_type {
 	ICE_TM_NODE_TYPE_PORT,
 	ICE_TM_NODE_TYPE_TC,
+	ICE_TM_NODE_TYPE_VSI,
+	ICE_TM_NODE_TYPE_QGROUP,
 	ICE_TM_NODE_TYPE_QUEUE,
 	ICE_TM_NODE_TYPE_MAX,
 };
@@ -489,10 +492,14 @@ enum ice_tm_node_type {
 /* Struct to store all the Traffic Manager configuration. */
 struct ice_tm_conf {
 	struct ice_shaper_profile_list shaper_profile_list;
-	struct ice_tm_node *root; /* root node - vf vsi */
+	struct ice_tm_node *root; /* root node - port */
 	struct ice_tm_node_list tc_list; /* node list for all the TCs */
+	struct ice_tm_node_list vsi_list; /* node list for all the VSIs */
+	struct ice_tm_node_list qgroup_list; /* node list for all the queue groups */
 	struct ice_tm_node_list queue_list; /* node list for all the queues */
 	uint32_t nb_tc_node;
+	uint32_t nb_vsi_node;
+	uint32_t nb_qgroup_node;
 	uint32_t nb_queue_node;
 	bool committed;
 };
drivers/net/ice/ice_tm.c +224 -15

347 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#67 lore raw

PATCH v7 6/9 net/ice: support queue priority configuration

In reply to Wenjun Wu

This patch adds queue priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 drivers/net/ice/ice_tm.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +12 -2
index d70d077286..91e420d653 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -779,6 +780,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -794,6 +796,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#68 lore raw

PATCH v7 7/9 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 drivers/net/ice/ice_tm.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +11 -2
index 91e420d653..4d7bb9102c 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -804,6 +804,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#69 lore raw

PATCH v7 8/9 net/ice: support queue group priority configuration

In reply to Wenjun Wu

This patch adds queue group priority configuration support.

---
 drivers/net/ice/ice_tm.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/net/ice/ice_tm.c +8 -0
index 4d7bb9102c..17f369994b 100644
@@ -764,6 +764,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_node_priority(hw->port_info, qgroup_sched_node, priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
Signature · 1 line
2.25.1
author
#70 lore raw

PATCH v7 9/9 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index 17f369994b..3e98c2f01e 100644
@@ -531,6 +531,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 			rte_calloc(NULL, 256, (sizeof(struct ice_tm_node *)), 0);
 	tm_node->parent->children[tm_node->parent->reference_count] = tm_node;
 
+	if (tm_node->priority != 0 && level_id != ICE_TM_NODE_TYPE_QUEUE &&
+	    level_id != ICE_TM_NODE_TYPE_QGROUP)
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	rte_memcpy(&tm_node->params, params,
 			 sizeof(struct rte_tm_node_params));
 	if (parent_node_type == ICE_TM_NODE_TYPE_PORT) {
Signature · 1 line
2.25.1
Acked-by on the v7 series
#71 lore raw
In reply to Wenjun Wu

Hi,

Quoted 39 lines

-----Original Message-----
From: Wu, Wenjun1 <[email protected]>
Sent: 2022年4月22日 8:58
To: [email protected]; Yang, Qiming <[email protected]>; Zhang, Qi Z <[email protected]>
Subject: [PATCH v7 0/9] Enable ETS-based TX QoS on PF

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue.
v4: fix logical issue.
v5: fix CI testing issue. Add explicit cast.
v6: add release note.
v7: merge the release note with the previous patch.

Ting Xu (1):
net/ice: support queue bandwidth limit

Wenjun Wu (8):
net/ice/base: fix dead lock issue when getting node from ID type net/ice/base: support priority configuration of the exact node net/ice/base: support queue BW allocation configuration net/ice: support queue group bandwidth limit
net/ice: support queue priority configuration net/ice: support queue weight configuration
net/ice: support queue group priority configuration net/ice: add warning log for unsupported configuration

 doc/guides/rel_notes/release_22_07.rst |   4 +
 drivers/net/ice/base/ice_sched.c       |  89 ++-
 drivers/net/ice/base/ice_sched.h       |   6 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  55 ++
 drivers/net/ice/ice_tm.c               | 844 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 7 files changed, 1016 insertions(+), 2 deletions(-)  create mode 100644
Quoted 4 lines

drivers/net/ice/ice_tm.c

--
2.25.1

author
#72 lore raw

v8 cover Enable ETS-based TX QoS on PF

In reply to Wenjun Wu

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue.
v4: fix logical issue.
v5: fix CI testing issue. Add explicit cast.
v6: add release note.
v7: merge the release note with the previous patch.
v8: rework shared code patch.

Ting Xu (1):
net/ice: support queue bandwidth limit

Wenjun Wu (8):
net/ice/base: fix dead lock issue when getting node from ID type net/ice/base: support queue BW allocation configuration net/ice/base: support priority configuration of the exact node net/ice: support queue group bandwidth limit
net/ice: support queue priority configuration net/ice: support queue weight configuration
net/ice: support queue group priority configuration net/ice: add warning log for unsupported configuration

 doc/guides/rel_notes/release_22_07.rst |   4 +
 drivers/net/ice/base/ice_sched.c       |  90 ++-
 drivers/net/ice/base/ice_sched.h       |   6 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  55 ++
 drivers/net/ice/ice_tm.c               | 845 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 7 files changed, 1018 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#73 lore raw

PATCH v8 1/9 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#74 lore raw

PATCH v8 2/9 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index e697c579be..4ca15bf8f8 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..184ad09e6a 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#75 lore raw

PATCH v8 3/9 net/ice/base: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 22 ++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 25 insertions(+)
drivers/net/ice/base/ice_sched.c +22 -0
index 4ca15bf8f8..1b060d3567 100644
@@ -3613,6 +3613,28 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_cfg_sibl_node_prio_lock - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_sched_cfg_sibl_node_prio_lock(struct ice_port_info *pi,
+				  struct ice_sched_node *node,
+				  u8 priority)
+{
+	enum ice_status status;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
  * @q_ctx: queue context structure
drivers/net/ice/base/ice_sched.h +3 -0
index 184ad09e6a..c9f3f79eff 100644
@@ -169,6 +169,9 @@ enum ice_status
 ice_cfg_agg_bw_no_shared_lmt_per_tc(struct ice_port_info *pi, u32 agg_id,
 				    u8 tc);
 enum ice_status
+ice_sched_cfg_sibl_node_prio_lock(struct ice_port_info *pi,
+				  struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
Signature · 1 line
2.25.1
author
#76 lore raw

PATCH v8 4/9 net/ice: support queue bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues. Only TC0 is valid.

---
 doc/guides/rel_notes/release_22_07.rst |   4 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  48 ++
 drivers/net/ice/ice_tm.c               | 599 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 5 files changed, 671 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
doc/guides/rel_notes/release_22_07.rst +4 -0
index 42a5f2d990..2ce3c99fb8 100644
@@ -55,6 +55,10 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Intel ice driver.**
+
+  * Added Tx QoS rate limitation and priority configuration support for queue and queue group.
+  * Added TX QoS queue weight configuration support.
 
 Removed Items
 -------------
drivers/net/ice/ice_ethdev.c +19 -0
index 13adcf90ed..37897765c8 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2312,6 +2325,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2492,6 +2508,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +48 -0

76 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +599 -0

599 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#77 lore raw

PATCH v8 5/9 net/ice: support queue group bandwidth limit

In reply to Wenjun Wu

To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes.

This patch adds queue group configuration support and queue group bandwidth limit support.

---
 drivers/net/ice/ice_ethdev.h |   9 +-
 drivers/net/ice/ice_tm.c     | 239 ++++++++++++++++++++++++++++++++---
 2 files changed, 232 insertions(+), 16 deletions(-)
drivers/net/ice/ice_ethdev.h +8 -1
index 0841e1866c..6ddbcc9972 100644
@@ -474,6 +474,7 @@ struct ice_tm_node {
 	uint32_t weight;
 	uint32_t reference_count;
 	struct ice_tm_node *parent;
+	struct ice_tm_node **children;
 	struct ice_tm_shaper_profile *shaper_profile;
 	struct rte_tm_node_params params;
 };
@@ -482,6 +483,8 @@ struct ice_tm_node {
 enum ice_tm_node_type {
 	ICE_TM_NODE_TYPE_PORT,
 	ICE_TM_NODE_TYPE_TC,
+	ICE_TM_NODE_TYPE_VSI,
+	ICE_TM_NODE_TYPE_QGROUP,
 	ICE_TM_NODE_TYPE_QUEUE,
 	ICE_TM_NODE_TYPE_MAX,
 };
@@ -489,10 +492,14 @@ enum ice_tm_node_type {
 /* Struct to store all the Traffic Manager configuration. */
 struct ice_tm_conf {
 	struct ice_shaper_profile_list shaper_profile_list;
-	struct ice_tm_node *root; /* root node - vf vsi */
+	struct ice_tm_node *root; /* root node - port */
 	struct ice_tm_node_list tc_list; /* node list for all the TCs */
+	struct ice_tm_node_list vsi_list; /* node list for all the VSIs */
+	struct ice_tm_node_list qgroup_list; /* node list for all the queue groups */
 	struct ice_tm_node_list queue_list; /* node list for all the queues */
 	uint32_t nb_tc_node;
+	uint32_t nb_vsi_node;
+	uint32_t nb_qgroup_node;
 	uint32_t nb_queue_node;
 	bool committed;
 };
drivers/net/ice/ice_tm.c +224 -15

347 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#78 lore raw

PATCH v8 6/9 net/ice: support queue priority configuration

In reply to Wenjun Wu

This patch adds queue priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 drivers/net/ice/ice_tm.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +12 -2
index d70d077286..91e420d653 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -779,6 +780,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -794,6 +796,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#79 lore raw

PATCH v8 7/9 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 drivers/net/ice/ice_tm.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +11 -2
index 91e420d653..4d7bb9102c 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -804,6 +804,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#80 lore raw

PATCH v8 8/9 net/ice: support queue group priority configuration

In reply to Wenjun Wu

This patch adds queue group priority configuration support.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index 4d7bb9102c..f604523ead 100644
@@ -764,6 +764,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_sched_cfg_sibl_node_prio_lock(hw->port_info, qgroup_sched_node,
+							    priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
Signature · 1 line
2.25.1
author
#81 lore raw

PATCH v8 9/9 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index f604523ead..34a0bfcff8 100644
@@ -531,6 +531,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 			rte_calloc(NULL, 256, (sizeof(struct ice_tm_node *)), 0);
 	tm_node->parent->children[tm_node->parent->reference_count] = tm_node;
 
+	if (tm_node->priority != 0 && level_id != ICE_TM_NODE_TYPE_QUEUE &&
+	    level_id != ICE_TM_NODE_TYPE_QGROUP)
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	rte_memcpy(&tm_node->params, params,
 			 sizeof(struct rte_tm_node_params));
 	if (parent_node_type == ICE_TM_NODE_TYPE_PORT) {
Signature · 1 line
2.25.1
author
#82 lore raw

v9 cover Enable ETS-based TX QoS on PF

In reply to Wenjun Wu

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue.
v4: fix logical issue.
v5: fix CI testing issue. Add explicit cast.
v6: add release note.
v7: merge the release note with the previous patch.
v8: rework shared code patch.
v9: rebase the code.

Ting Xu (1):
net/ice: support queue bandwidth limit

Wenjun Wu (8):
net/ice/base: fix dead lock issue when getting node from ID type net/ice/base: support queue BW allocation configuration net/ice/base: support priority configuration of the exact node net/ice: support queue group bandwidth limit
net/ice: support queue priority configuration net/ice: support queue weight configuration
net/ice: support queue group priority configuration net/ice: add warning log for unsupported configuration

 doc/guides/rel_notes/release_22_07.rst |   5 +
 drivers/net/ice/base/ice_sched.c       |  90 ++-
 drivers/net/ice/base/ice_sched.h       |   6 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  55 ++
 drivers/net/ice/ice_tm.c               | 845 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 7 files changed, 1019 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#83 lore raw

PATCH v9 1/9 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#84 lore raw

PATCH v9 2/9 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index e697c579be..4ca15bf8f8 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..184ad09e6a 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#85 lore raw

PATCH v9 3/9 net/ice/base: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 22 ++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 25 insertions(+)
drivers/net/ice/base/ice_sched.c +22 -0
index 4ca15bf8f8..1b060d3567 100644
@@ -3613,6 +3613,28 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_cfg_sibl_node_prio_lock - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_sched_cfg_sibl_node_prio_lock(struct ice_port_info *pi,
+				  struct ice_sched_node *node,
+				  u8 priority)
+{
+	enum ice_status status;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
  * @q_ctx: queue context structure
drivers/net/ice/base/ice_sched.h +3 -0
index 184ad09e6a..c9f3f79eff 100644
@@ -169,6 +169,9 @@ enum ice_status
 ice_cfg_agg_bw_no_shared_lmt_per_tc(struct ice_port_info *pi, u32 agg_id,
 				    u8 tc);
 enum ice_status
+ice_sched_cfg_sibl_node_prio_lock(struct ice_port_info *pi,
+				  struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
Signature · 1 line
2.25.1
author
#86 lore raw

PATCH v9 4/9 net/ice: support queue bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues. Only TC0 is valid.

---
 doc/guides/rel_notes/release_22_07.rst |   5 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  48 ++
 drivers/net/ice/ice_tm.c               | 599 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 5 files changed, 672 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
doc/guides/rel_notes/release_22_07.rst +5 -0
index 90123bb807..4797da32da 100644
@@ -60,6 +60,11 @@ New Features
   * Added Tx QoS queue rate limitation support.
   * Added quanta size configuration support.
 
+* **Updated Intel ice driver.**
+
+  * Added Tx QoS rate limitation and priority configuration support for queue and queue group.
+  * Added TX QoS queue weight configuration support.
+
 
 Removed Items
 -------------
drivers/net/ice/ice_ethdev.c +19 -0
index 00ac2bb191..35ab542e61 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2328,6 +2341,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2508,6 +2524,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +48 -0

76 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +599 -0

599 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#87 lore raw

PATCH v9 5/9 net/ice: support queue group bandwidth limit

In reply to Wenjun Wu

To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes.

This patch adds queue group configuration support and queue group bandwidth limit support.

---
 drivers/net/ice/ice_ethdev.h |   9 +-
 drivers/net/ice/ice_tm.c     | 239 ++++++++++++++++++++++++++++++++---
 2 files changed, 232 insertions(+), 16 deletions(-)
drivers/net/ice/ice_ethdev.h +8 -1
index 4359c61624..f9f4a1c71b 100644
@@ -474,6 +474,7 @@ struct ice_tm_node {
 	uint32_t weight;
 	uint32_t reference_count;
 	struct ice_tm_node *parent;
+	struct ice_tm_node **children;
 	struct ice_tm_shaper_profile *shaper_profile;
 	struct rte_tm_node_params params;
 };
@@ -482,6 +483,8 @@ struct ice_tm_node {
 enum ice_tm_node_type {
 	ICE_TM_NODE_TYPE_PORT,
 	ICE_TM_NODE_TYPE_TC,
+	ICE_TM_NODE_TYPE_VSI,
+	ICE_TM_NODE_TYPE_QGROUP,
 	ICE_TM_NODE_TYPE_QUEUE,
 	ICE_TM_NODE_TYPE_MAX,
 };
@@ -489,10 +492,14 @@ enum ice_tm_node_type {
 /* Struct to store all the Traffic Manager configuration. */
 struct ice_tm_conf {
 	struct ice_shaper_profile_list shaper_profile_list;
-	struct ice_tm_node *root; /* root node - vf vsi */
+	struct ice_tm_node *root; /* root node - port */
 	struct ice_tm_node_list tc_list; /* node list for all the TCs */
+	struct ice_tm_node_list vsi_list; /* node list for all the VSIs */
+	struct ice_tm_node_list qgroup_list; /* node list for all the queue groups */
 	struct ice_tm_node_list queue_list; /* node list for all the queues */
 	uint32_t nb_tc_node;
+	uint32_t nb_vsi_node;
+	uint32_t nb_qgroup_node;
 	uint32_t nb_queue_node;
 	bool committed;
 };
drivers/net/ice/ice_tm.c +224 -15

347 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#88 lore raw

PATCH v9 6/9 net/ice: support queue priority configuration

In reply to Wenjun Wu

This patch adds queue priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 drivers/net/ice/ice_tm.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +12 -2
index d70d077286..91e420d653 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -779,6 +780,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -794,6 +796,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#89 lore raw

PATCH v9 7/9 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 drivers/net/ice/ice_tm.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/net/ice/ice_tm.c +11 -2
index 91e420d653..4d7bb9102c 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -804,6 +804,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#90 lore raw

PATCH v9 8/9 net/ice: support queue group priority configuration

In reply to Wenjun Wu

This patch adds queue group priority configuration support.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index 4d7bb9102c..f604523ead 100644
@@ -764,6 +764,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_sched_cfg_sibl_node_prio_lock(hw->port_info, qgroup_sched_node,
+							    priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
Signature · 1 line
2.25.1
author
#91 lore raw

PATCH v9 9/9 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index f604523ead..34a0bfcff8 100644
@@ -531,6 +531,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 			rte_calloc(NULL, 256, (sizeof(struct ice_tm_node *)), 0);
 	tm_node->parent->children[tm_node->parent->reference_count] = tm_node;
 
+	if (tm_node->priority != 0 && level_id != ICE_TM_NODE_TYPE_QUEUE &&
+	    level_id != ICE_TM_NODE_TYPE_QGROUP)
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	rte_memcpy(&tm_node->params, params,
 			 sizeof(struct rte_tm_node_params));
 	if (parent_node_type == ICE_TM_NODE_TYPE_PORT) {
Signature · 1 line
2.25.1
author
#92 lore raw

v10 cover Enable ETS-based TX QoS on PF

In reply to Wenjun Wu

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue.
v4: fix logical issue.
v5: fix CI testing issue. Add explicit cast.
v6: add release note.
v7: merge the release note with the previous patch.
v8: rework shared code patch.
v9: rebase the code.
v10: rebase the code and rework the release note.

Ting Xu (1):
net/ice: support queue and queue group bandwidth limit

Wenjun Wu (6):
net/ice/base: fix dead lock issue when getting node from ID type net/ice/base: support queue BW allocation configuration net/ice/base: support priority configuration of the exact node net/ice: support queue and queue group priority configuration net/ice: support queue weight configuration
net/ice: add warning log for unsupported configuration

 doc/guides/rel_notes/release_22_07.rst |   3 +
 drivers/net/ice/base/ice_sched.c       |  90 ++-
 drivers/net/ice/base/ice_sched.h       |   6 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  55 ++
 drivers/net/ice/ice_tm.c               | 845 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 7 files changed, 1017 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#93 lore raw

PATCH v10 1/7 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#94 lore raw

PATCH v10 2/7 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index e697c579be..4ca15bf8f8 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..184ad09e6a 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#95 lore raw

PATCH v10 3/7 net/ice/base: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 22 ++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 25 insertions(+)
drivers/net/ice/base/ice_sched.c +22 -0
index 4ca15bf8f8..1b060d3567 100644
@@ -3613,6 +3613,28 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_cfg_sibl_node_prio_lock - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_sched_cfg_sibl_node_prio_lock(struct ice_port_info *pi,
+				  struct ice_sched_node *node,
+				  u8 priority)
+{
+	enum ice_status status;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
  * @q_ctx: queue context structure
drivers/net/ice/base/ice_sched.h +3 -0
index 184ad09e6a..c9f3f79eff 100644
@@ -169,6 +169,9 @@ enum ice_status
 ice_cfg_agg_bw_no_shared_lmt_per_tc(struct ice_port_info *pi, u32 agg_id,
 				    u8 tc);
 enum ice_status
+ice_sched_cfg_sibl_node_prio_lock(struct ice_port_info *pi,
+				  struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
Signature · 1 line
2.25.1
author
#96 lore raw

PATCH v10 4/7 net/ice: support queue and queue group bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues and queue group. To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes. Only TC0 is valid.

---
 doc/guides/rel_notes/release_22_07.rst |   1 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  55 ++
 drivers/net/ice/ice_tm.c               | 808 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 5 files changed, 884 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
doc/guides/rel_notes/release_22_07.rst +1 -0
index a60a0d5f16..de29061809 100644
@@ -74,6 +74,7 @@ New Features
  * Added support for promisc configuration in DCF mode.
  * Added support for MAC configuration in DCF mode.
  * Added support for VLAN filter and offload configuration in DCF mode.
+ * Added Tx QoS queue / queue group rate limitation configure support.
 
 * **Updated Mellanox mlx5 driver.**
 
drivers/net/ice/ice_ethdev.c +19 -0
index 00ac2bb191..35ab542e61 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2328,6 +2341,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2508,6 +2524,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +55 -0

83 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +808 -0

808 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#97 lore raw

PATCH v10 5/7 net/ice: support queue and queue group priority configuration

In reply to Wenjun Wu

This patch adds queue and queue group priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 doc/guides/rel_notes/release_22_07.rst |  1 +
 drivers/net/ice/ice_tm.c               | 23 +++++++++++++++++++++--
 2 files changed, 22 insertions(+), 2 deletions(-)
doc/guides/rel_notes/release_22_07.rst +1 -0
index de29061809..c5bfc52368 100644
@@ -75,6 +75,7 @@ New Features
  * Added support for MAC configuration in DCF mode.
  * Added support for VLAN filter and offload configuration in DCF mode.
  * Added Tx QoS queue / queue group rate limitation configure support.
+ * Added Tx QoS queue / queue group priority configuration support.
 
 * **Updated Mellanox mlx5 driver.**
 
drivers/net/ice/ice_tm.c +21 -2
index d70d077286..105455f3cc 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -763,6 +764,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_sched_cfg_sibl_node_prio_lock(hw->port_info, qgroup_sched_node,
+							    priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
@@ -779,6 +789,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -794,6 +805,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#98 lore raw

PATCH v10 6/7 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 doc/guides/rel_notes/release_22_07.rst |  1 +
 drivers/net/ice/ice_tm.c               | 13 +++++++++++--
 2 files changed, 12 insertions(+), 2 deletions(-)
doc/guides/rel_notes/release_22_07.rst +1 -0
index c5bfc52368..a0eb6ab61b 100644
@@ -76,6 +76,7 @@ New Features
  * Added support for VLAN filter and offload configuration in DCF mode.
  * Added Tx QoS queue / queue group rate limitation configure support.
  * Added Tx QoS queue / queue group priority configuration support.
+ * Added Tx QoS queue weight configuration support.
 
 * **Updated Mellanox mlx5 driver.**
 
drivers/net/ice/ice_tm.c +11 -2
index 105455f3cc..f604523ead 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -813,6 +813,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#99 lore raw

PATCH v10 7/7 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index f604523ead..34a0bfcff8 100644
@@ -531,6 +531,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 			rte_calloc(NULL, 256, (sizeof(struct ice_tm_node *)), 0);
 	tm_node->parent->children[tm_node->parent->reference_count] = tm_node;
 
+	if (tm_node->priority != 0 && level_id != ICE_TM_NODE_TYPE_QUEUE &&
+	    level_id != ICE_TM_NODE_TYPE_QGROUP)
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	rte_memcpy(&tm_node->params, params,
 			 sizeof(struct rte_tm_node_params));
 	if (parent_node_type == ICE_TM_NODE_TYPE_PORT) {
Signature · 1 line
2.25.1
author
#100 lore raw

v11 cover Enable ETS-based TX QoS on PF

In reply to Wenjun Wu

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue.
v4: fix logical issue.
v5: fix CI testing issue. Add explicit cast.
v6: add release note.
v7: merge the release note with the previous patch.
v8: rework shared code patch.
v9: rebase the code.
v10: rebase the code and rework the release note.
v11: add fix information in commit log.

Ting Xu (1):
net/ice: support queue and queue group bandwidth limit

Wenjun Wu (6):
net/ice/base: fix dead lock issue when getting node from ID type net/ice/base: support queue BW allocation configuration net/ice/base: support priority configuration of the exact node net/ice: support queue and queue group priority configuration net/ice: support queue weight configuration
net/ice: add warning log for unsupported configuration

 doc/guides/rel_notes/release_22_07.rst |   3 +
 drivers/net/ice/base/ice_sched.c       |  90 ++-
 drivers/net/ice/base/ice_sched.h       |   6 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  55 ++
 drivers/net/ice/ice_tm.c               | 845 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 7 files changed, 1017 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_tm.c
Signature · 1 line
2.25.1
author
#101 lore raw

PATCH v11 1/7 net/ice/base: fix dead lock issue when getting node from ID type

The function ice_sched_get_node_by_id_type needs to be called with the scheduler lock held. However, the function ice_sched_get_node also requests the scheduler lock. It will cause the dead lock issue.

This patch replaces function ice_sched_get_node with function ice_sched_find_node_by_teid to solve this problem.

Fixes 93e84b1bfc92 ("net/ice/base: add basic Tx scheduler")
---
 drivers/net/ice/base/ice_sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ice/base/ice_sched.c +2 -2
index 2620892c9e..e697c579be 100644
@@ -4774,12 +4774,12 @@ ice_sched_get_node_by_id_type(struct ice_port_info *pi, u32 id,
 
 	case ICE_AGG_TYPE_Q:
 		/* The current implementation allows single queue to modify */
-		node = ice_sched_get_node(pi, id);
+		node = ice_sched_find_node_by_teid(pi->root, id);
 		break;
 
 	case ICE_AGG_TYPE_QG:
 		/* The current implementation allows single qg to modify */
-		child_node = ice_sched_get_node(pi, id);
+		child_node = ice_sched_find_node_by_teid(pi->root, id);
 		if (!child_node)
 			break;
 		node = child_node->parent;
Signature · 1 line
2.25.1
author
#102 lore raw

PATCH v11 2/7 net/ice/base: support queue BW allocation configuration

In reply to Wenjun Wu

This patch adds BW allocation support of queue scheduling node to support WFQ in queue level.

---
 drivers/net/ice/base/ice_sched.c | 64 ++++++++++++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 ++
 2 files changed, 67 insertions(+)
drivers/net/ice/base/ice_sched.c +64 -0
index e697c579be..4ca15bf8f8 100644
@@ -3613,6 +3613,70 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
+ * @q_ctx: queue context structure
+ * @rl_type: rate limit type min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * Save BW information of queue type node for post replay use.
+ */
+static enum ice_status
+ice_sched_save_q_bw_alloc(struct ice_q_ctx *q_ctx, enum ice_rl_type rl_type,
+			  u32 bw_alloc)
+{
+	switch (rl_type) {
+	case ICE_MIN_BW:
+		ice_set_clear_cir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	case ICE_MAX_BW:
+		ice_set_clear_eir_bw_alloc(&q_ctx->bw_t_info, bw_alloc);
+		break;
+	default:
+		return ICE_ERR_PARAM;
+	}
+	return ICE_SUCCESS;
+}
+
+/**
+ * ice_cfg_q_bw_alloc - configure queue BW weight/alloc params
+ * @pi: port information structure
+ * @vsi_handle: sw VSI handle
+ * @tc: traffic class
+ * @q_handle: software queue handle
+ * @rl_type: min, max, or shared
+ * @bw_alloc: BW weight/allocation
+ *
+ * This function configures BW allocation of queue scheduling node.
+ */
+enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc)
+{
+	enum ice_status status = ICE_ERR_PARAM;
+	struct ice_sched_node *node;
+	struct ice_q_ctx *q_ctx;
+
+	ice_acquire_lock(&pi->sched_lock);
+	q_ctx = ice_get_lan_q_ctx(pi->hw, vsi_handle, tc, q_handle);
+	if (!q_ctx)
+		goto exit_q_bw_alloc;
+
+	node = ice_sched_find_node_by_teid(pi->root, q_ctx->q_teid);
+	if (!node) {
+		ice_debug(pi->hw, ICE_DBG_SCHED, "Wrong q_teid\n");
+		goto exit_q_bw_alloc;
+	}
+
+	status = ice_sched_cfg_node_bw_alloc(pi->hw, node, rl_type, bw_alloc);
+	if (!status)
+		status = ice_sched_save_q_bw_alloc(q_ctx, rl_type, bw_alloc);
+
+exit_q_bw_alloc:
+	ice_release_lock(&pi->sched_lock);
+	return status;
+}
+
 /**
  * ice_cfg_agg_vsi_priority_per_tc - config aggregator's VSI priority per TC
  * @pi: port information structure
drivers/net/ice/base/ice_sched.h +3 -0
index 1441b5f191..184ad09e6a 100644
@@ -172,6 +172,9 @@ enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
+ice_cfg_q_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+		   u16 q_handle, enum ice_rl_type rl_type, u32 bw_alloc);
+enum ice_status
 ice_cfg_vsi_bw_alloc(struct ice_port_info *pi, u16 vsi_handle, u8 ena_tcmap,
 		     enum ice_rl_type rl_type, u8 *bw_alloc);
 enum ice_status
Signature · 1 line
2.25.1
author
#103 lore raw

PATCH v11 3/7 net/ice/base: support priority configuration of the exact node

In reply to Wenjun Wu

This patch adds priority configuration support of the exact node in the scheduler tree.
This function does not need additional calls to the scheduler lock.

---
 drivers/net/ice/base/ice_sched.c | 22 ++++++++++++++++++++++
 drivers/net/ice/base/ice_sched.h |  3 +++
 2 files changed, 25 insertions(+)
drivers/net/ice/base/ice_sched.c +22 -0
index 4ca15bf8f8..1b060d3567 100644
@@ -3613,6 +3613,28 @@ ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 	return status;
 }
 
+/**
+ * ice_sched_cfg_sibl_node_prio_lock - config priority of node
+ * @pi: port information structure
+ * @node: sched node to configure
+ * @priority: sibling priority
+ *
+ * This function configures node element's sibling priority only.
+ */
+enum ice_status
+ice_sched_cfg_sibl_node_prio_lock(struct ice_port_info *pi,
+				  struct ice_sched_node *node,
+				  u8 priority)
+{
+	enum ice_status status;
+
+	ice_acquire_lock(&pi->sched_lock);
+	status = ice_sched_cfg_sibl_node_prio(pi, node, priority);
+	ice_release_lock(&pi->sched_lock);
+
+	return status;
+}
+
 /**
  * ice_sched_save_q_bw_alloc - save queue node's BW allocation information
  * @q_ctx: queue context structure
drivers/net/ice/base/ice_sched.h +3 -0
index 184ad09e6a..c9f3f79eff 100644
@@ -169,6 +169,9 @@ enum ice_status
 ice_cfg_agg_bw_no_shared_lmt_per_tc(struct ice_port_info *pi, u32 agg_id,
 				    u8 tc);
 enum ice_status
+ice_sched_cfg_sibl_node_prio_lock(struct ice_port_info *pi,
+				  struct ice_sched_node *node, u8 priority);
+enum ice_status
 ice_cfg_vsi_q_priority(struct ice_port_info *pi, u16 num_qs, u32 *q_ids,
 		       u8 *q_prio);
 enum ice_status
Signature · 1 line
2.25.1
author
#104 lore raw

PATCH v11 4/7 net/ice: support queue and queue group bandwidth limit

In reply to Wenjun Wu

From: Ting Xu <[email protected]>

Enable basic TM API for PF only. Support for adding profiles and queue nodes. Only max bandwidth is supported in profiles. Profiles can be assigned to target queues and queue group. To set up the exact queue group, we need to reconfigure topology by delete and then recreate queue nodes. Only TC0 is valid.

---
 doc/guides/rel_notes/release_22_07.rst |   1 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  55 ++
 drivers/net/ice/ice_tm.c               | 808 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 5 files changed, 884 insertions(+)
 create mode 100644 drivers/net/ice/ice_tm.c
doc/guides/rel_notes/release_22_07.rst +1 -0
index a60a0d5f16..de29061809 100644
@@ -74,6 +74,7 @@ New Features
  * Added support for promisc configuration in DCF mode.
  * Added support for MAC configuration in DCF mode.
  * Added support for VLAN filter and offload configuration in DCF mode.
+ * Added Tx QoS queue / queue group rate limitation configure support.
 
 * **Updated Mellanox mlx5 driver.**
 
drivers/net/ice/ice_ethdev.c +19 -0
index 00ac2bb191..35ab542e61 100644
@@ -205,6 +205,18 @@ static const struct rte_pci_id pci_id_ice_map[] = {
 	{ .vendor_id = 0, /* sentinel */ },
 };
 
+static int
+ice_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_eth_dev_ops = {
 	.dev_configure                = ice_dev_configure,
 	.dev_start                    = ice_dev_start,
@@ -267,6 +279,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = {
 	.timesync_read_time           = ice_timesync_read_time,
 	.timesync_write_time          = ice_timesync_write_time,
 	.timesync_disable             = ice_timesync_disable,
+	.tm_ops_get                   = ice_tm_ops_get,
 };
 
 /* store statistics names and its offset in stats structure */
@@ -2328,6 +2341,9 @@ ice_dev_init(struct rte_eth_dev *dev)
 	/* Initialize RSS context for gtpu_eh */
 	ice_rss_ctx_init(pf);
 
+	/* Initialize TM configuration */
+	ice_tm_conf_init(dev);
+
 	if (!ad->is_safe_mode) {
 		ret = ice_flow_init(ad);
 		if (ret) {
@@ -2508,6 +2524,9 @@ ice_dev_close(struct rte_eth_dev *dev)
 	rte_free(pf->proto_xtr);
 	pf->proto_xtr = NULL;
 
+	/* Uninit TM configuration */
+	ice_tm_conf_uninit(dev);
+
 	if (ad->devargs.pps_out_ena) {
 		ICE_WRITE_REG(hw, GLTSYN_AUX_OUT(pin_idx, timer), 0);
 		ICE_WRITE_REG(hw, GLTSYN_CLKO(pin_idx, timer), 0);
drivers/net/ice/ice_ethdev.h +55 -0

83 lines not on this page · raw message

drivers/net/ice/ice_tm.c new file +808 -0

808 lines not on this page · raw message

drivers/net/ice/meson.build +1 -0
index d608da7765..de307c9e71 100644
@@ -12,6 +12,7 @@ sources = files(
         'ice_hash.c',
         'ice_rxtx.c',
         'ice_switch_filter.c',
+        'ice_tm.c',
 )
 
 deps += ['hash', 'net', 'common_iavf']
Signature · 1 line
2.25.1
author
#105 lore raw

PATCH v11 5/7 net/ice: support queue and queue group priority configuration

In reply to Wenjun Wu

This patch adds queue and queue group priority configuration support. The highest priority is 0, and the lowest priority is 7.

---
 doc/guides/rel_notes/release_22_07.rst |  1 +
 drivers/net/ice/ice_tm.c               | 23 +++++++++++++++++++++--
 2 files changed, 22 insertions(+), 2 deletions(-)
doc/guides/rel_notes/release_22_07.rst +1 -0
index de29061809..c5bfc52368 100644
@@ -75,6 +75,7 @@ New Features
  * Added support for MAC configuration in DCF mode.
  * Added support for VLAN filter and offload configuration in DCF mode.
  * Added Tx QoS queue / queue group rate limitation configure support.
+ * Added Tx QoS queue / queue group priority configuration support.
 
 * **Updated Mellanox mlx5 driver.**
 
drivers/net/ice/ice_tm.c +21 -2
index d70d077286..105455f3cc 100644
@@ -147,9 +147,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (priority) {
+	if (priority >= 8) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
-		error->message = "priority should be 0";
+		error->message = "priority should be less than 8";
 		return -EINVAL;
 	}
 
@@ -684,6 +684,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 	struct ice_vsi *vsi;
 	int ret_val = ICE_SUCCESS;
 	uint64_t peak = 0;
+	uint8_t priority;
 	uint32_t i;
 	uint32_t idx_vsi_child;
 	uint32_t idx_qg;
@@ -763,6 +764,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_sched_cfg_sibl_node_prio_lock(hw->port_info, qgroup_sched_node,
+							    priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue group %u priority failed",
+				    tm_node->priority);
+			goto fail_clear;
+		}
 		idx_qg++;
 		if (idx_qg >= nb_qg) {
 			idx_qg = 0;
@@ -779,6 +789,7 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 		qid = tm_node->id;
 		txq = dev->data->tx_queues[qid];
 		vsi = txq->vsi;
+		q_teid = txq->q_teid;
 		if (tm_node->shaper_profile) {
 			/* Transfer from Byte per seconds to Kbps */
 			peak = tm_node->shaper_profile->profile.peak.rate;
@@ -794,6 +805,14 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 				goto fail_clear;
 			}
 		}
+		priority = 7 - tm_node->priority;
+		ret_val = ice_cfg_vsi_q_priority(hw->port_info, 1,
+						 &q_teid, &priority);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_PRIORITY;
+			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#106 lore raw

PATCH v11 6/7 net/ice: support queue weight configuration

In reply to Wenjun Wu

This patch adds queue weight configuration support.

---
 doc/guides/rel_notes/release_22_07.rst |  1 +
 drivers/net/ice/ice_tm.c               | 13 +++++++++++--
 2 files changed, 12 insertions(+), 2 deletions(-)
doc/guides/rel_notes/release_22_07.rst +1 -0
index c5bfc52368..a0eb6ab61b 100644
@@ -76,6 +76,7 @@ New Features
  * Added support for VLAN filter and offload configuration in DCF mode.
  * Added Tx QoS queue / queue group rate limitation configure support.
  * Added Tx QoS queue / queue group priority configuration support.
+ * Added Tx QoS queue weight configuration support.
 
 * **Updated Mellanox mlx5 driver.**
 
drivers/net/ice/ice_tm.c +11 -2
index 105455f3cc..f604523ead 100644
@@ -153,9 +153,9 @@ ice_node_param_check(struct ice_pf *pf, uint32_t node_id,
 		return -EINVAL;
 	}
 
-	if (weight != 1) {
+	if (weight > 200 || weight < 1) {
 		error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
-		error->message = "weight must be 1";
+		error->message = "weight must be between 1 and 200";
 		return -EINVAL;
 	}
 
@@ -813,6 +813,15 @@ static int ice_hierarchy_commit(struct rte_eth_dev *dev,
 			PMD_DRV_LOG(ERR, "configure queue %u priority failed", tm_node->priority);
 			goto fail_clear;
 		}
+
+		ret_val = ice_cfg_q_bw_alloc(hw->port_info, vsi->idx,
+					     tm_node->tc, tm_node->id,
+					     ICE_MAX_BW, (u32)tm_node->weight);
+		if (ret_val) {
+			error->type = RTE_TM_ERROR_TYPE_NODE_WEIGHT;
+			PMD_DRV_LOG(ERR, "configure queue %u weight failed", tm_node->weight);
+			goto fail_clear;
+		}
 	}
 
 	return ret_val;
Signature · 1 line
2.25.1
author
#107 lore raw

PATCH v11 7/7 net/ice: add warning log for unsupported configuration

In reply to Wenjun Wu

Priority configuration is enabled in level 3 and level 4. Weight configuration is enabled in level 4.
This patch adds warning log for unsupported priority and weight configuration.

---
 drivers/net/ice/ice_tm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ice/ice_tm.c +9 -0
index f604523ead..34a0bfcff8 100644
@@ -531,6 +531,15 @@ ice_tm_node_add(struct rte_eth_dev *dev, uint32_t node_id,
 			rte_calloc(NULL, 256, (sizeof(struct ice_tm_node *)), 0);
 	tm_node->parent->children[tm_node->parent->reference_count] = tm_node;
 
+	if (tm_node->priority != 0 && level_id != ICE_TM_NODE_TYPE_QUEUE &&
+	    level_id != ICE_TM_NODE_TYPE_QGROUP)
+		PMD_DRV_LOG(WARNING, "priority != 0 not supported in level %d",
+			    level_id);
+
+	if (tm_node->weight != 1 && level_id != ICE_TM_NODE_TYPE_QUEUE)
+		PMD_DRV_LOG(WARNING, "weight != 1 not supported in level %d",
+			    level_id);
+
 	rte_memcpy(&tm_node->params, params,
 			 sizeof(struct rte_tm_node_params));
 	if (parent_node_type == ICE_TM_NODE_TYPE_PORT) {
Signature · 1 line
2.25.1
Acked-by on the v11 series Applied
#108 lore raw
In reply to Wenjun Wu
Quoted 43 lines

-----Original Message-----
From: Wu, Wenjun1 <[email protected]>
Sent: Tuesday, May 17, 2022 1:09 PM
To: [email protected]; Yang, Qiming <[email protected]>; Zhang, Qi Z <[email protected]>
Subject: [PATCH v11 0/7] Enable ETS-based TX QoS on PF

This patch set enables ETS-based TX QoS on PF. It is supported to configure bandwidth and priority in both queue and queue group level, and weight only in queue level.

v2: fix code style issue.
v3: fix uninitialization issue.
v4: fix logical issue.
v5: fix CI testing issue. Add explicit cast.
v6: add release note.
v7: merge the release note with the previous patch.
v8: rework shared code patch.
v9: rebase the code.
v10: rebase the code and rework the release note.
v11: add fix information in commit log.

Ting Xu (1):
net/ice: support queue and queue group bandwidth limit

Wenjun Wu (6):
net/ice/base: fix dead lock issue when getting node from ID type net/ice/base: support queue BW allocation configuration net/ice/base: support priority configuration of the exact node net/ice: support queue and queue group priority configuration net/ice: support queue weight configuration
net/ice: add warning log for unsupported configuration

 doc/guides/rel_notes/release_22_07.rst |   3 +
 drivers/net/ice/base/ice_sched.c       |  90 ++-
 drivers/net/ice/base/ice_sched.h       |   6 +
 drivers/net/ice/ice_ethdev.c           |  19 +
 drivers/net/ice/ice_ethdev.h           |  55 ++
 drivers/net/ice/ice_tm.c               | 845 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   1 +
 7 files changed, 1017 insertions(+), 2 deletions(-)  create mode 100644
Quoted 4 lines

drivers/net/ice/ice_tm.c

--
2.25.1

Applied to dpdk-next-net-intel.

Thanks
Qi