Skip to contentSkip to conversation

Enable ETS-based Tx QoS for VF in DCF

PATCH Superseded Ting Xu 3 participants 44 messages Jun 1, 2021 – Jul 7, 2021

Subsystems (2)
lore
Share & tools
lore

Ting Xu

author
#0 lore raw

cover Enable ETS-based Tx QoS for VF in DCF

This patch enables the ETS-based Tx QoS for IAVF. Kernel tool is used to configure ETS first. DCF is used to set bandwidth limit for VFs of each TC. IAVF is supported to query QoS capability and set queue TC mapping. Traffic Management API is utilized to configure the QoS hierarchy scheduler tree. The scheduler tree will be passed to hardware to enable all above functions.

Ting Xu (5):
common/iavf: add support for ETS-based Tx QoS net/ice/base: support DCF query port ETS adminq net/ice: support DCF link status event handling net/ice: support QoS config VF bandwidth in DCF net/iavf: query QoS cap and set queue TC mapping

 drivers/common/iavf/iavf_type.h  |   2 +
 drivers/common/iavf/virtchnl.h   | 117 ++++++
 drivers/net/iavf/iavf.h          |  45 +++
 drivers/net/iavf/iavf_ethdev.c   |  31 ++
 drivers/net/iavf/iavf_tm.c       | 675 +++++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c    |  56 ++-
 drivers/net/iavf/meson.build     |   1 +
 drivers/net/ice/base/ice_dcb.c   |   3 +-
 drivers/net/ice/ice_dcf.c        |   6 +-
 drivers/net/ice/ice_dcf.h        |  53 +++
 drivers/net/ice/ice_dcf_ethdev.c |  67 ++-
 drivers/net/ice/ice_dcf_ethdev.h |   3 +
 drivers/net/ice/ice_dcf_parent.c |  81 ++++
 drivers/net/ice/ice_dcf_sched.c  | 604 +++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   3 +-
 15 files changed, 1740 insertions(+), 7 deletions(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 1/5 common/iavf: add support for ETS-based Tx QoS

This patch adds support to configure ETS-based Tx QoS. Three parts of new virtchnl structures and opcodes are added to achieve:
1. Configure VF TC bandwidth limits.
2. VF queries current QoS configuration from PF.
3. Set up VF queue TC mapping.

---
 drivers/common/iavf/iavf_type.h |   2 +
 drivers/common/iavf/virtchnl.h  | 117 ++++++++++++++++++++++++++++++++
 2 files changed, 119 insertions(+)
drivers/common/iavf/iavf_type.h +2 -0
index f3815d523b..73dfb47e70 100644
@@ -141,6 +141,8 @@ enum iavf_debug_mask {
 #define IAVF_PHY_LED_MODE_MASK			0xFFFF
 #define IAVF_PHY_LED_MODE_ORIG			0x80000000
 
+#define IAVF_MAX_TRAFFIC_CLASS	8
+
 /* Memory types */
 enum iavf_memset_type {
 	IAVF_NONDMA_MEM = 0,
drivers/common/iavf/virtchnl.h +117 -0

153 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#2 lore raw

PATCH 2/5 net/ice/base: support DCF query port ETS adminq

In reply to Ting Xu

In the adminq command query port ETS function, the root node teid is needed. However, for DCF, the root node is not initialized, which will cause error when we refer to the variable. In this patch, we will check whether the root node is available or not first.

---
 drivers/net/ice/base/ice_dcb.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ice/base/ice_dcb.c +2 -1
index 0aaa5ae8c1..08c950cd9a 100644
@@ -1483,7 +1483,8 @@ ice_aq_query_port_ets(struct ice_port_info *pi,
 		return ICE_ERR_PARAM;
 	cmd = &desc.params.port_ets;
 	ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_query_port_ets);
-	cmd->port_teid = pi->root->info.node_teid;
+	if (pi->root)
+		cmd->port_teid = pi->root->info.node_teid;
 
 	status = ice_aq_send_cmd(pi->hw, &desc, buf, buf_size, cd);
 	return status;
Signature · 1 line
2.17.1
author
#3 lore raw

PATCH 3/5 net/ice: support DCF link status event handling

In reply to Ting Xu

When link status changes, DCF will receive virtchnl PF event message. Add support to handle this event, change link status and update link info.

---
 drivers/net/ice/ice_dcf.h        |  6 ++++
 drivers/net/ice/ice_dcf_ethdev.c | 54 ++++++++++++++++++++++++++++++--
 drivers/net/ice/ice_dcf_parent.c | 51 ++++++++++++++++++++++++++++++
 3 files changed, 108 insertions(+), 3 deletions(-)
drivers/net/ice/ice_dcf.h +6 -0
index 0cb90b5e9f..587093b909 100644
@@ -60,6 +60,10 @@ struct ice_dcf_hw {
 	uint16_t nb_msix;
 	uint16_t rxq_map[16];
 	struct virtchnl_eth_stats eth_stats_offset;
+
+	/* Link status */
+	bool link_up;
+	uint32_t link_speed;
 };
 
 int ice_dcf_execute_virtchnl_cmd(struct ice_dcf_hw *hw,
@@ -77,5 +81,7 @@ int ice_dcf_disable_queues(struct ice_dcf_hw *hw);
 int ice_dcf_query_stats(struct ice_dcf_hw *hw,
 			struct virtchnl_eth_stats *pstats);
 int ice_dcf_add_del_all_mac_addr(struct ice_dcf_hw *hw, bool add);
+int ice_dcf_link_update(struct rte_eth_dev *dev,
+		    __rte_unused int wait_to_complete);
 
 #endif /* _ICE_DCF_H_ */
drivers/net/ice/ice_dcf_ethdev.c +51 -3

62 lines not on this page · raw message

drivers/net/ice/ice_dcf_parent.c +51 -0

63 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#4 lore raw

PATCH 4/5 net/ice: support QoS config VF bandwidth in DCF

In reply to Ting Xu

This patch supports the ETS-based QoS configuration. It enables the DCF to configure bandwidth limits for each VF VSI of different TCs. A hierarchy scheduler tree is built with port, TC and VSI nodes.

---
 drivers/net/ice/ice_dcf.c        |   6 +-
 drivers/net/ice/ice_dcf.h        |  47 +++
 drivers/net/ice/ice_dcf_ethdev.c |  13 +
 drivers/net/ice/ice_dcf_ethdev.h |   3 +
 drivers/net/ice/ice_dcf_parent.c |  30 ++
 drivers/net/ice/ice_dcf_sched.c  | 604 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   3 +-
 7 files changed, 704 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
drivers/net/ice/ice_dcf.c +5 -1
index d72a6f357e..f8b4e07d86 100644
@@ -235,7 +235,8 @@ ice_dcf_get_vf_resource(struct ice_dcf_hw *hw)
 	caps = VIRTCHNL_VF_OFFLOAD_WB_ON_ITR | VIRTCHNL_VF_OFFLOAD_RX_POLLING |
 	       VIRTCHNL_VF_CAP_ADV_LINK_SPEED | VIRTCHNL_VF_CAP_DCF |
 	       VIRTCHNL_VF_OFFLOAD_VLAN_V2 |
-	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC;
+	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC |
+	       VIRTCHNL_VF_OFFLOAD_TC;
 
 	err = ice_dcf_send_cmd_req_no_irq(hw, VIRTCHNL_OP_GET_VF_RESOURCES,
 					  (uint8_t *)&caps, sizeof(caps));
@@ -668,6 +669,9 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 		}
 	}
 
+	if (hw->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_TC)
+		ice_dcf_tm_conf_init(eth_dev);
+
 	hw->eth_dev = eth_dev;
 	rte_intr_callback_register(&pci_dev->intr_handle,
 				   ice_dcf_dev_interrupt_handler, hw);
drivers/net/ice/ice_dcf.h +47 -0

70 lines not on this page · raw message

drivers/net/ice/ice_dcf_ethdev.c +13 -0
index 332ce340cf..91c4486260 100644
@@ -993,6 +993,18 @@ ice_dcf_dev_udp_tunnel_port_del(struct rte_eth_dev *dev,
 	return ret;
 }
 
+static int
+ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_dcf_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.dev_start               = ice_dcf_dev_start,
 	.dev_stop                = ice_dcf_dev_stop,
@@ -1017,6 +1029,7 @@ static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.flow_ops_get            = ice_dcf_dev_flow_ops_get,
 	.udp_tunnel_port_add	 = ice_dcf_dev_udp_tunnel_port_add,
 	.udp_tunnel_port_del	 = ice_dcf_dev_udp_tunnel_port_del,
+	.tm_ops_get              = ice_dcf_tm_ops_get,
 };
 
 static int
drivers/net/ice/ice_dcf_ethdev.h +3 -0
index e7c9d7fe41..8510e37119 100644
@@ -7,6 +7,8 @@
 
 #include "base/ice_common.h"
 #include "base/ice_adminq_cmd.h"
+#include "base/ice_dcb.h"
+#include "base/ice_sched.h"
 
 #include "ice_ethdev.h"
 #include "ice_dcf.h"
@@ -52,6 +54,7 @@ struct ice_dcf_vf_repr {
 	struct ice_dcf_vlan outer_vlan_info; /* DCF always handle outer VLAN */
 };
 
+extern const struct rte_tm_ops ice_dcf_tm_ops;
 void ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 				 uint8_t *msg, uint16_t msglen);
 int ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev);
drivers/net/ice/ice_dcf_parent.c +30 -0
index 0c0706316d..2403d9c259 100644
@@ -264,6 +264,29 @@ ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 	}
 }
 
+static int
+ice_dcf_query_port_ets(struct ice_hw *parent_hw, struct ice_dcf_hw *real_hw)
+{
+	int ret;
+
+	real_hw->ets_config = (struct ice_aqc_port_ets_elem *)
+			ice_malloc(real_hw, sizeof(*real_hw->ets_config));
+	if (!real_hw->ets_config)
+		return ICE_ERR_NO_MEMORY;
+
+	ret = ice_aq_query_port_ets(parent_hw->port_info,
+			real_hw->ets_config, sizeof(*real_hw->ets_config),
+			NULL);
+	if (ret) {
+		PMD_DRV_LOG(ERR, "DCF Query Port ETS failed");
+		rte_free(real_hw->ets_config);
+		real_hw->ets_config = NULL;
+		return ret;
+	}
+
+	return ICE_SUCCESS;
+}
+
 static int
 ice_dcf_init_parent_hw(struct ice_hw *hw)
 {
@@ -487,6 +510,13 @@ ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev)
 		return err;
 	}
 
+	err = ice_dcf_query_port_ets(parent_hw, hw);
+	if (err) {
+		PMD_INIT_LOG(ERR, "failed to query port ets with error %d",
+			     err);
+		goto uninit_hw;
+	}
+
 	err = ice_dcf_load_pkg(parent_hw);
 	if (err) {
 		PMD_INIT_LOG(ERR, "failed to load package with error %d",
drivers/net/ice/ice_dcf_sched.c new file +604 -0

604 lines not on this page · raw message

drivers/net/ice/meson.build +2 -1
index 65750d3501..0b86d74a49 100644
@@ -70,6 +70,7 @@ endif
 sources += files('ice_dcf.c',
          'ice_dcf_vf_representor.c',
          'ice_dcf_ethdev.c',
-         'ice_dcf_parent.c')
+         'ice_dcf_parent.c',
+	 'ice_dcf_sched.c')
 
 headers = files('rte_pmd_ice.h')
Signature · 1 line
2.17.1
author
#5 lore raw

PATCH 5/5 net/iavf: query QoS cap and set queue TC mapping

In reply to Ting Xu

This patch added the support for VF to config the ETS-based Tx QoS, including querying current QoS configuration from PF and config queue TC mapping. PF QoS is configured in advance and the queried info is provided to the user for future usage. VF queues are mapped to different TCs in PF through virtchnl.

---
 drivers/net/iavf/iavf.h        |  45 +++
 drivers/net/iavf/iavf_ethdev.c |  31 ++
 drivers/net/iavf/iavf_tm.c     | 675 +++++++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c  |  56 ++-
 drivers/net/iavf/meson.build   |   1 +
 5 files changed, 807 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
drivers/net/iavf/iavf.h +45 -0
index 4f5811ae87..77ddf15f42 100644
@@ -6,6 +6,8 @@
 #define _IAVF_ETHDEV_H_
 
 #include <rte_kvargs.h>
+#include <rte_tm_driver.h>
+
 #include <iavf_prototype.h>
 #include <iavf_adminq_cmd.h>
 #include <iavf_type.h>
@@ -82,6 +84,8 @@
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_MASK  0x03
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_FD_ID 0x01
 
+#define IAVF_BITS_PER_BYTE 8
+
 struct iavf_adapter;
 struct iavf_rx_queue;
 struct iavf_tx_queue;
@@ -129,6 +133,38 @@ enum iavf_aq_result {
 	IAVF_MSG_CMD,      /* Read async command result */
 };
 
+/* Struct to store Traffic Manager node configuration. */
+struct iavf_tm_node {
+	TAILQ_ENTRY(iavf_tm_node) node;
+	uint32_t id;
+	uint32_t tc;
+	uint32_t priority;
+	uint32_t weight;
+	uint32_t reference_count;
+	struct iavf_tm_node *parent;
+	struct rte_tm_node_params params;
+};
+
+TAILQ_HEAD(iavf_tm_node_list, iavf_tm_node);
+
+/* node type of Traffic Manager */
+enum iavf_tm_node_type {
+	IAVF_TM_NODE_TYPE_PORT,
+	IAVF_TM_NODE_TYPE_TC,
+	IAVF_TM_NODE_TYPE_QUEUE,
+	IAVF_TM_NODE_TYPE_MAX,
+};
+
+/* Struct to store all the Traffic Manager configuration. */
+struct iavf_tm_conf {
+	struct iavf_tm_node *root; /* root node - vf vsi */
+	struct iavf_tm_node_list tc_list; /* node list for all the TCs */
+	struct iavf_tm_node_list queue_list; /* node list for all the queues */
+	uint32_t nb_tc_node;
+	uint32_t nb_queue_node;
+	bool committed;
+};
+
 /* Structure to store private data specific for VF instance. */
 struct iavf_info {
 	uint16_t num_queue_pairs;
@@ -175,6 +211,9 @@ struct iavf_info {
 	struct iavf_fdir_info fdir; /* flow director info */
 	/* indicate large VF support enabled or not */
 	bool lv_enabled;
+
+	struct virtchnl_qos_cap_list *qos_cap;
+	struct iavf_tm_conf tm_conf;
 };
 
 #define IAVF_MAX_PKT_TYPE 1024
@@ -344,4 +383,10 @@ int iavf_add_del_mc_addr_list(struct iavf_adapter *adapter,
 			uint32_t mc_addrs_num, bool add);
 int iavf_request_queues(struct iavf_adapter *adapter, uint16_t num);
 int iavf_get_max_rss_queue_region(struct iavf_adapter *adapter);
+int iavf_get_qos_cap(struct iavf_adapter *adapter);
+int iavf_set_q_tc_map(struct rte_eth_dev *dev,
+			struct virtchnl_queue_tc_mapping *q_tc_mapping,
+			uint16_t size);
+void iavf_tm_conf_init(struct rte_eth_dev *dev);
+extern const struct rte_tm_ops iavf_tm_ops;
 #endif /* _IAVF_ETHDEV_H_ */
drivers/net/iavf/iavf_ethdev.c +31 -0
index cb38fe81e1..e0a03a0bee 100644
@@ -122,6 +122,7 @@ static int iavf_dev_flow_ops_get(struct rte_eth_dev *dev,
 static int iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
 			uint32_t mc_addrs_num);
+static int iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused, void *arg);
 
 static const struct rte_pci_id pci_id_iavf_map[] = {
 	{ RTE_PCI_DEVICE(IAVF_INTEL_VENDOR_ID, IAVF_DEV_ID_ADAPTIVE_VF) },
@@ -200,8 +201,21 @@ static const struct eth_dev_ops iavf_eth_dev_ops = {
 	.flow_ops_get               = iavf_dev_flow_ops_get,
 	.tx_done_cleanup	    = iavf_dev_tx_done_cleanup,
 	.get_monitor_addr           = iavf_get_monitor_addr,
+	.tm_ops_get                 = iavf_tm_ops_get,
 };
 
+static int
+iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+			void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &iavf_tm_ops;
+
+	return 0;
+}
+
 static int
 iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
@@ -806,6 +820,11 @@ iavf_dev_start(struct rte_eth_dev *dev)
 				      dev->data->nb_tx_queues);
 	num_queue_pairs = vf->num_queue_pairs;
 
+	if (iavf_get_qos_cap(adapter)) {
+		PMD_INIT_LOG(ERR, "Failed to get qos capability");
+		return -1;
+	}
+
 	if (iavf_init_queues(dev) != 0) {
 		PMD_DRV_LOG(ERR, "failed to do Queue init");
 		return -1;
@@ -2090,6 +2109,15 @@ iavf_init_vf(struct rte_eth_dev *dev)
 		PMD_INIT_LOG(ERR, "unable to allocate vf_res memory");
 		goto err_api;
 	}
+
+	bufsz = sizeof(struct virtchnl_qos_cap_list) +
+		IAVF_MAX_TRAFFIC_CLASS * sizeof(struct virtchnl_qos_cap_elem);
+	vf->qos_cap = rte_zmalloc("qos_cap", bufsz, 0);
+	if (!vf->qos_cap) {
+		PMD_INIT_LOG(ERR, "unable to allocate qos_cap memory");
+		goto err_api;
+	}
+
 	if (iavf_get_vf_resource(adapter) != 0) {
 		PMD_INIT_LOG(ERR, "iavf_get_vf_config failed");
 		goto err_alloc;
@@ -2131,6 +2159,7 @@ iavf_init_vf(struct rte_eth_dev *dev)
 	rte_free(vf->rss_key);
 	rte_free(vf->rss_lut);
 err_alloc:
+	rte_free(vf->qos_cap);
 	rte_free(vf->vf_res);
 	vf->vsi_res = NULL;
 err_api:
@@ -2299,6 +2328,8 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 
 	iavf_default_rss_disable(adapter);
 
+	iavf_tm_conf_init(eth_dev);
+
 	return 0;
 }
 
drivers/net/iavf/iavf_tm.c new file +675 -0

675 lines not on this page · raw message

drivers/net/iavf/iavf_vchnl.c +55 -1

68 lines not on this page · raw message

drivers/net/iavf/meson.build +1 -0
index 6f222a9e87..f2010a8337 100644
@@ -19,6 +19,7 @@ sources = files(
         'iavf_generic_flow.c',
         'iavf_fdir.c',
         'iavf_hash.c',
+        'iavf_tm.c',
 )
 
 if arch_subdir == 'x86'
Signature · 1 line
2.17.1
author
#6 lore raw

v2 cover Enable ETS-based Tx QoS for VF in DCF

In reply to Ting Xu

This patch enables the ETS-based Tx QoS for IAVF. Kernel tool is used to configure ETS first. DCF is used to set bandwidth limit for VFs of each TC. IAVF is supported to query QoS capability and set queue TC mapping. Traffic Management API is utilized to configure the QoS hierarchy scheduler tree. The scheduler tree will be passed to hardware to enable all above functions.

Ting Xu (5):
common/iavf: support ETS-based QoS offload configuration net/ice/base: support DCF query port ETS adminq net/ice: support DCF link status event handling net/ice: support QoS config VF bandwidth in DCF net/iavf: query QoS cap and set queue TC mapping

 drivers/common/iavf/iavf_type.h  |   2 +
 drivers/common/iavf/virtchnl.h   | 125 ++++++
 drivers/net/iavf/iavf.h          |  45 ++
 drivers/net/iavf/iavf_ethdev.c   |  31 ++
 drivers/net/iavf/iavf_tm.c       | 663 +++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c    |  56 ++-
 drivers/net/iavf/meson.build     |   1 +
 drivers/net/ice/base/ice_dcb.c   |   3 +-
 drivers/net/ice/ice_dcf.c        |   6 +-
 drivers/net/ice/ice_dcf.h        |  53 +++
 drivers/net/ice/ice_dcf_ethdev.c |  67 ++-
 drivers/net/ice/ice_dcf_ethdev.h |   3 +
 drivers/net/ice/ice_dcf_parent.c |  81 ++++
 drivers/net/ice/ice_dcf_sched.c  | 688 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   3 +-
 15 files changed, 1820 insertions(+), 7 deletions(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
Signature · 1 line
2.25.1
author
#7 lore raw

PATCH v2 1/5 common/iavf: support ETS-based QoS offload configuration

This patch adds new virtchnl opcodes and structures for QoS configuration, which includes:
1. VIRTCHNL_VF_OFFLOAD_TC, to negotiate the capability supporting QoS configuration. If VF and PF both have this flag, then the ETS-based QoS offload function is supported.
2. VIRTCHNL_OP_DCF_CONFIG_BW, DCF is supposed to configure min and max bandwidth for each VF per enabled TCs. To make the VSI node bandwidth configuration work, DCF also needs to configure TC node bandwidth directly.
3. VIRTCHNL_OP_GET_QOS_CAPS, VF queries current QoS configuration, such as enabled TCs, arbiter type, up2tc and bandwidth of VSI node. The configuration is previously set by DCB and DCF, and now is the potential QoS capability of VF. VF can take it as reference to configure queue TC mapping.
4. VIRTCHNL_OP_CONFIG_TC_MAP, set VF queues to TC mapping for all Tx and Rx queues. Queues mapping to one TC should be continuous and all allocated queues should be mapped.

---
 drivers/common/iavf/iavf_type.h |   2 +
 drivers/common/iavf/virtchnl.h  | 125 ++++++++++++++++++++++++++++++++
 2 files changed, 127 insertions(+)
drivers/common/iavf/iavf_type.h +2 -0
index f3815d523b..73dfb47e70 100644
@@ -141,6 +141,8 @@ enum iavf_debug_mask {
 #define IAVF_PHY_LED_MODE_MASK			0xFFFF
 #define IAVF_PHY_LED_MODE_ORIG			0x80000000
 
+#define IAVF_MAX_TRAFFIC_CLASS	8
+
 /* Memory types */
 enum iavf_memset_type {
 	IAVF_NONDMA_MEM = 0,
drivers/common/iavf/virtchnl.h +125 -0

179 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#8 lore raw

PATCH v2 2/5 net/ice/base: support DCF query port ETS adminq

In reply to Ting Xu

In the adminq command query port ETS function, the root node teid is needed. However, for DCF, the root node is not initialized, which will cause error when we refer to the variable. In this patch, we will check whether the root node is available or not first.

---
 drivers/net/ice/base/ice_dcb.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ice/base/ice_dcb.c +2 -1
index 0aaa5ae8c1..08c950cd9a 100644
@@ -1483,7 +1483,8 @@ ice_aq_query_port_ets(struct ice_port_info *pi,
 		return ICE_ERR_PARAM;
 	cmd = &desc.params.port_ets;
 	ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_query_port_ets);
-	cmd->port_teid = pi->root->info.node_teid;
+	if (pi->root)
+		cmd->port_teid = pi->root->info.node_teid;
 
 	status = ice_aq_send_cmd(pi->hw, &desc, buf, buf_size, cd);
 	return status;
Signature · 1 line
2.25.1
author
#9 lore raw

PATCH v2 3/5 net/ice: support DCF link status event handling

In reply to Ting Xu

When link status changes, DCF will receive virtchnl PF event message. Add support to handle this event, change link status and update link info.

---
 drivers/net/ice/ice_dcf.h        |  6 ++++
 drivers/net/ice/ice_dcf_ethdev.c | 54 ++++++++++++++++++++++++++++++--
 drivers/net/ice/ice_dcf_parent.c | 51 ++++++++++++++++++++++++++++++
 3 files changed, 108 insertions(+), 3 deletions(-)
drivers/net/ice/ice_dcf.h +6 -0
index 0cb90b5e9f..587093b909 100644
@@ -60,6 +60,10 @@ struct ice_dcf_hw {
 	uint16_t nb_msix;
 	uint16_t rxq_map[16];
 	struct virtchnl_eth_stats eth_stats_offset;
+
+	/* Link status */
+	bool link_up;
+	uint32_t link_speed;
 };
 
 int ice_dcf_execute_virtchnl_cmd(struct ice_dcf_hw *hw,
@@ -77,5 +81,7 @@ int ice_dcf_disable_queues(struct ice_dcf_hw *hw);
 int ice_dcf_query_stats(struct ice_dcf_hw *hw,
 			struct virtchnl_eth_stats *pstats);
 int ice_dcf_add_del_all_mac_addr(struct ice_dcf_hw *hw, bool add);
+int ice_dcf_link_update(struct rte_eth_dev *dev,
+		    __rte_unused int wait_to_complete);
 
 #endif /* _ICE_DCF_H_ */
drivers/net/ice/ice_dcf_ethdev.c +51 -3

62 lines not on this page · raw message

drivers/net/ice/ice_dcf_parent.c +51 -0

63 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#10 lore raw

PATCH v2 4/5 net/ice: support QoS config VF bandwidth in DCF

In reply to Ting Xu

This patch supports the ETS-based QoS configuration. It enables the DCF to configure bandwidth limits for each VF VSI of different TCs. A hierarchy scheduler tree is built with port, TC and VSI nodes.

---
 drivers/net/ice/ice_dcf.c        |   6 +-
 drivers/net/ice/ice_dcf.h        |  47 +++
 drivers/net/ice/ice_dcf_ethdev.c |  13 +
 drivers/net/ice/ice_dcf_ethdev.h |   3 +
 drivers/net/ice/ice_dcf_parent.c |  30 ++
 drivers/net/ice/ice_dcf_sched.c  | 688 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   3 +-
 7 files changed, 788 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
drivers/net/ice/ice_dcf.c +5 -1
index d72a6f357e..f8b4e07d86 100644
@@ -235,7 +235,8 @@ ice_dcf_get_vf_resource(struct ice_dcf_hw *hw)
 	caps = VIRTCHNL_VF_OFFLOAD_WB_ON_ITR | VIRTCHNL_VF_OFFLOAD_RX_POLLING |
 	       VIRTCHNL_VF_CAP_ADV_LINK_SPEED | VIRTCHNL_VF_CAP_DCF |
 	       VIRTCHNL_VF_OFFLOAD_VLAN_V2 |
-	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC;
+	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC |
+	       VIRTCHNL_VF_OFFLOAD_TC;
 
 	err = ice_dcf_send_cmd_req_no_irq(hw, VIRTCHNL_OP_GET_VF_RESOURCES,
 					  (uint8_t *)&caps, sizeof(caps));
@@ -668,6 +669,9 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 		}
 	}
 
+	if (hw->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_TC)
+		ice_dcf_tm_conf_init(eth_dev);
+
 	hw->eth_dev = eth_dev;
 	rte_intr_callback_register(&pci_dev->intr_handle,
 				   ice_dcf_dev_interrupt_handler, hw);
drivers/net/ice/ice_dcf.h +47 -0

70 lines not on this page · raw message

drivers/net/ice/ice_dcf_ethdev.c +13 -0
index 819c671c2d..219ff7cd86 100644
@@ -993,6 +993,18 @@ ice_dcf_dev_udp_tunnel_port_del(struct rte_eth_dev *dev,
 	return ret;
 }
 
+static int
+ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_dcf_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.dev_start               = ice_dcf_dev_start,
 	.dev_stop                = ice_dcf_dev_stop,
@@ -1017,6 +1029,7 @@ static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.flow_ops_get            = ice_dcf_dev_flow_ops_get,
 	.udp_tunnel_port_add	 = ice_dcf_dev_udp_tunnel_port_add,
 	.udp_tunnel_port_del	 = ice_dcf_dev_udp_tunnel_port_del,
+	.tm_ops_get              = ice_dcf_tm_ops_get,
 };
 
 static int
drivers/net/ice/ice_dcf_ethdev.h +3 -0
index e7c9d7fe41..8510e37119 100644
@@ -7,6 +7,8 @@
 
 #include "base/ice_common.h"
 #include "base/ice_adminq_cmd.h"
+#include "base/ice_dcb.h"
+#include "base/ice_sched.h"
 
 #include "ice_ethdev.h"
 #include "ice_dcf.h"
@@ -52,6 +54,7 @@ struct ice_dcf_vf_repr {
 	struct ice_dcf_vlan outer_vlan_info; /* DCF always handle outer VLAN */
 };
 
+extern const struct rte_tm_ops ice_dcf_tm_ops;
 void ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 				 uint8_t *msg, uint16_t msglen);
 int ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev);
drivers/net/ice/ice_dcf_parent.c +30 -0
index 0c0706316d..2403d9c259 100644
@@ -264,6 +264,29 @@ ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 	}
 }
 
+static int
+ice_dcf_query_port_ets(struct ice_hw *parent_hw, struct ice_dcf_hw *real_hw)
+{
+	int ret;
+
+	real_hw->ets_config = (struct ice_aqc_port_ets_elem *)
+			ice_malloc(real_hw, sizeof(*real_hw->ets_config));
+	if (!real_hw->ets_config)
+		return ICE_ERR_NO_MEMORY;
+
+	ret = ice_aq_query_port_ets(parent_hw->port_info,
+			real_hw->ets_config, sizeof(*real_hw->ets_config),
+			NULL);
+	if (ret) {
+		PMD_DRV_LOG(ERR, "DCF Query Port ETS failed");
+		rte_free(real_hw->ets_config);
+		real_hw->ets_config = NULL;
+		return ret;
+	}
+
+	return ICE_SUCCESS;
+}
+
 static int
 ice_dcf_init_parent_hw(struct ice_hw *hw)
 {
@@ -487,6 +510,13 @@ ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev)
 		return err;
 	}
 
+	err = ice_dcf_query_port_ets(parent_hw, hw);
+	if (err) {
+		PMD_INIT_LOG(ERR, "failed to query port ets with error %d",
+			     err);
+		goto uninit_hw;
+	}
+
 	err = ice_dcf_load_pkg(parent_hw);
 	if (err) {
 		PMD_INIT_LOG(ERR, "failed to load package with error %d",
drivers/net/ice/ice_dcf_sched.c new file +688 -0

688 lines not on this page · raw message

drivers/net/ice/meson.build +2 -1
index 65750d3501..0b86d74a49 100644
@@ -70,6 +70,7 @@ endif
 sources += files('ice_dcf.c',
          'ice_dcf_vf_representor.c',
          'ice_dcf_ethdev.c',
-         'ice_dcf_parent.c')
+         'ice_dcf_parent.c',
+	 'ice_dcf_sched.c')
 
 headers = files('rte_pmd_ice.h')
Signature · 1 line
2.25.1
author
#11 lore raw

PATCH v2 5/5 net/iavf: query QoS cap and set queue TC mapping

In reply to Ting Xu

This patch added the support for VF to config the ETS-based Tx QoS, including querying current QoS configuration from PF and config queue TC mapping. PF QoS is configured in advance and the queried info is provided to the user for future usage. VF queues are mapped to different TCs in PF through virtchnl.

---
 drivers/net/iavf/iavf.h        |  45 +++
 drivers/net/iavf/iavf_ethdev.c |  31 ++
 drivers/net/iavf/iavf_tm.c     | 663 +++++++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c  |  56 ++-
 drivers/net/iavf/meson.build   |   1 +
 5 files changed, 795 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
drivers/net/iavf/iavf.h +45 -0
index 4f5811ae87..77ddf15f42 100644
@@ -6,6 +6,8 @@
 #define _IAVF_ETHDEV_H_
 
 #include <rte_kvargs.h>
+#include <rte_tm_driver.h>
+
 #include <iavf_prototype.h>
 #include <iavf_adminq_cmd.h>
 #include <iavf_type.h>
@@ -82,6 +84,8 @@
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_MASK  0x03
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_FD_ID 0x01
 
+#define IAVF_BITS_PER_BYTE 8
+
 struct iavf_adapter;
 struct iavf_rx_queue;
 struct iavf_tx_queue;
@@ -129,6 +133,38 @@ enum iavf_aq_result {
 	IAVF_MSG_CMD,      /* Read async command result */
 };
 
+/* Struct to store Traffic Manager node configuration. */
+struct iavf_tm_node {
+	TAILQ_ENTRY(iavf_tm_node) node;
+	uint32_t id;
+	uint32_t tc;
+	uint32_t priority;
+	uint32_t weight;
+	uint32_t reference_count;
+	struct iavf_tm_node *parent;
+	struct rte_tm_node_params params;
+};
+
+TAILQ_HEAD(iavf_tm_node_list, iavf_tm_node);
+
+/* node type of Traffic Manager */
+enum iavf_tm_node_type {
+	IAVF_TM_NODE_TYPE_PORT,
+	IAVF_TM_NODE_TYPE_TC,
+	IAVF_TM_NODE_TYPE_QUEUE,
+	IAVF_TM_NODE_TYPE_MAX,
+};
+
+/* Struct to store all the Traffic Manager configuration. */
+struct iavf_tm_conf {
+	struct iavf_tm_node *root; /* root node - vf vsi */
+	struct iavf_tm_node_list tc_list; /* node list for all the TCs */
+	struct iavf_tm_node_list queue_list; /* node list for all the queues */
+	uint32_t nb_tc_node;
+	uint32_t nb_queue_node;
+	bool committed;
+};
+
 /* Structure to store private data specific for VF instance. */
 struct iavf_info {
 	uint16_t num_queue_pairs;
@@ -175,6 +211,9 @@ struct iavf_info {
 	struct iavf_fdir_info fdir; /* flow director info */
 	/* indicate large VF support enabled or not */
 	bool lv_enabled;
+
+	struct virtchnl_qos_cap_list *qos_cap;
+	struct iavf_tm_conf tm_conf;
 };
 
 #define IAVF_MAX_PKT_TYPE 1024
@@ -344,4 +383,10 @@ int iavf_add_del_mc_addr_list(struct iavf_adapter *adapter,
 			uint32_t mc_addrs_num, bool add);
 int iavf_request_queues(struct iavf_adapter *adapter, uint16_t num);
 int iavf_get_max_rss_queue_region(struct iavf_adapter *adapter);
+int iavf_get_qos_cap(struct iavf_adapter *adapter);
+int iavf_set_q_tc_map(struct rte_eth_dev *dev,
+			struct virtchnl_queue_tc_mapping *q_tc_mapping,
+			uint16_t size);
+void iavf_tm_conf_init(struct rte_eth_dev *dev);
+extern const struct rte_tm_ops iavf_tm_ops;
 #endif /* _IAVF_ETHDEV_H_ */
drivers/net/iavf/iavf_ethdev.c +31 -0
index cb38fe81e1..e0a03a0bee 100644
@@ -122,6 +122,7 @@ static int iavf_dev_flow_ops_get(struct rte_eth_dev *dev,
 static int iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
 			uint32_t mc_addrs_num);
+static int iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused, void *arg);
 
 static const struct rte_pci_id pci_id_iavf_map[] = {
 	{ RTE_PCI_DEVICE(IAVF_INTEL_VENDOR_ID, IAVF_DEV_ID_ADAPTIVE_VF) },
@@ -200,8 +201,21 @@ static const struct eth_dev_ops iavf_eth_dev_ops = {
 	.flow_ops_get               = iavf_dev_flow_ops_get,
 	.tx_done_cleanup	    = iavf_dev_tx_done_cleanup,
 	.get_monitor_addr           = iavf_get_monitor_addr,
+	.tm_ops_get                 = iavf_tm_ops_get,
 };
 
+static int
+iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+			void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &iavf_tm_ops;
+
+	return 0;
+}
+
 static int
 iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
@@ -806,6 +820,11 @@ iavf_dev_start(struct rte_eth_dev *dev)
 				      dev->data->nb_tx_queues);
 	num_queue_pairs = vf->num_queue_pairs;
 
+	if (iavf_get_qos_cap(adapter)) {
+		PMD_INIT_LOG(ERR, "Failed to get qos capability");
+		return -1;
+	}
+
 	if (iavf_init_queues(dev) != 0) {
 		PMD_DRV_LOG(ERR, "failed to do Queue init");
 		return -1;
@@ -2090,6 +2109,15 @@ iavf_init_vf(struct rte_eth_dev *dev)
 		PMD_INIT_LOG(ERR, "unable to allocate vf_res memory");
 		goto err_api;
 	}
+
+	bufsz = sizeof(struct virtchnl_qos_cap_list) +
+		IAVF_MAX_TRAFFIC_CLASS * sizeof(struct virtchnl_qos_cap_elem);
+	vf->qos_cap = rte_zmalloc("qos_cap", bufsz, 0);
+	if (!vf->qos_cap) {
+		PMD_INIT_LOG(ERR, "unable to allocate qos_cap memory");
+		goto err_api;
+	}
+
 	if (iavf_get_vf_resource(adapter) != 0) {
 		PMD_INIT_LOG(ERR, "iavf_get_vf_config failed");
 		goto err_alloc;
@@ -2131,6 +2159,7 @@ iavf_init_vf(struct rte_eth_dev *dev)
 	rte_free(vf->rss_key);
 	rte_free(vf->rss_lut);
 err_alloc:
+	rte_free(vf->qos_cap);
 	rte_free(vf->vf_res);
 	vf->vsi_res = NULL;
 err_api:
@@ -2299,6 +2328,8 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 
 	iavf_default_rss_disable(adapter);
 
+	iavf_tm_conf_init(eth_dev);
+
 	return 0;
 }
 
drivers/net/iavf/iavf_tm.c new file +663 -0

663 lines not on this page · raw message

drivers/net/iavf/iavf_vchnl.c +55 -1

68 lines not on this page · raw message

drivers/net/iavf/meson.build +1 -0
index 6f222a9e87..f2010a8337 100644
@@ -19,6 +19,7 @@ sources = files(
         'iavf_generic_flow.c',
         'iavf_fdir.c',
         'iavf_hash.c',
+        'iavf_tm.c',
 )
 
 if arch_subdir == 'x86'
Signature · 1 line
2.25.1
author
#12 lore raw

v3 cover Enable ETS-based Tx QoS for VF in DCF

In reply to Ting Xu

This patch enables the ETS-based Tx QoS for IAVF. Kernel tool is used to configure ETS first. DCF is used to set bandwidth limit for VFs of each TC. IAVF is supported to query QoS capability and set queue TC mapping. Traffic Management API is utilized to configure the QoS hierarchy scheduler tree. The scheduler tree will be passed to hardware to enable all above functions.

Ting Xu (5):
common/iavf: support ETS-based QoS offload configuration net/ice/base: support DCF query port ETS adminq net/ice: support DCF link status event handling net/ice: support QoS config VF bandwidth in DCF net/iavf: query QoS cap and set queue TC mapping

 drivers/common/iavf/iavf_type.h  |   2 +
 drivers/common/iavf/virtchnl.h   | 131 ++++++
 drivers/net/iavf/iavf.h          |  45 ++
 drivers/net/iavf/iavf_ethdev.c   |  31 ++
 drivers/net/iavf/iavf_tm.c       | 667 +++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c    |  56 ++-
 drivers/net/iavf/meson.build     |   1 +
 drivers/net/ice/base/ice_dcb.c   |   3 +-
 drivers/net/ice/ice_dcf.c        |   6 +-
 drivers/net/ice/ice_dcf.h        |  53 +++
 drivers/net/ice/ice_dcf_ethdev.c |  67 ++-
 drivers/net/ice/ice_dcf_ethdev.h |   3 +
 drivers/net/ice/ice_dcf_parent.c |  81 ++++
 drivers/net/ice/ice_dcf_sched.c  | 697 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   3 +-
 15 files changed, 1839 insertions(+), 7 deletions(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
Signature · 1 line
2.25.1
author
#13 lore raw

PATCH v3 1/5 common/iavf: support ETS-based QoS offload configuration

This patch adds new virtchnl opcodes and structures for QoS configuration, which includes:
1. VIRTCHNL_VF_OFFLOAD_TC, to negotiate the capability supporting QoS configuration. If VF and PF both have this flag, then the ETS-based QoS offload function is supported.
2. VIRTCHNL_OP_DCF_CONFIG_BW, DCF is supposed to configure min and max bandwidth for each VF per enabled TCs. To make the VSI node bandwidth configuration work, DCF also needs to configure TC node bandwidth directly.
3. VIRTCHNL_OP_GET_QOS_CAPS, VF queries current QoS configuration, such as enabled TCs, arbiter type, up2tc and bandwidth of VSI node. The configuration is previously set by DCB and DCF, and now is the potential QoS capability of VF. VF can take it as reference to configure queue TC mapping.
4. VIRTCHNL_OP_CONFIG_TC_MAP, set VF queues to TC mapping for all Tx and Rx queues. Queues mapping to one TC should be continuous and all allocated queues should be mapped.

---
 drivers/common/iavf/iavf_type.h |   2 +
 drivers/common/iavf/virtchnl.h  | 131 ++++++++++++++++++++++++++++++++
 2 files changed, 133 insertions(+)
drivers/common/iavf/iavf_type.h +2 -0
index f3815d523b..73dfb47e70 100644
@@ -141,6 +141,8 @@ enum iavf_debug_mask {
 #define IAVF_PHY_LED_MODE_MASK			0xFFFF
 #define IAVF_PHY_LED_MODE_ORIG			0x80000000
 
+#define IAVF_MAX_TRAFFIC_CLASS	8
+
 /* Memory types */
 enum iavf_memset_type {
 	IAVF_NONDMA_MEM = 0,
drivers/common/iavf/virtchnl.h +131 -0

185 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#14 lore raw

PATCH v3 2/5 net/ice/base: support DCF query port ETS adminq

In reply to Ting Xu

In the adminq command query port ETS function, the root node teid is needed. However, for DCF, the root node is not initialized, which will cause error when we refer to the variable. In this patch, we will check whether the root node is available or not first.

---
 drivers/net/ice/base/ice_dcb.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ice/base/ice_dcb.c +2 -1
index c73fc095ff..9c9675f6ef 100644
@@ -1524,7 +1524,8 @@ ice_aq_query_port_ets(struct ice_port_info *pi,
 		return ICE_ERR_PARAM;
 	cmd = &desc.params.port_ets;
 	ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_query_port_ets);
-	cmd->port_teid = pi->root->info.node_teid;
+	if (pi->root)
+		cmd->port_teid = pi->root->info.node_teid;
 
 	status = ice_aq_send_cmd(pi->hw, &desc, buf, buf_size, cd);
 	return status;
Signature · 1 line
2.25.1
author
#15 lore raw

PATCH v3 3/5 net/ice: support DCF link status event handling

In reply to Ting Xu

When link status changes, DCF will receive virtchnl PF event message. Add support to handle this event, change link status and update link info.

---
 drivers/net/ice/ice_dcf.h        |  6 ++++
 drivers/net/ice/ice_dcf_ethdev.c | 54 ++++++++++++++++++++++++++++++--
 drivers/net/ice/ice_dcf_parent.c | 51 ++++++++++++++++++++++++++++++
 3 files changed, 108 insertions(+), 3 deletions(-)
drivers/net/ice/ice_dcf.h +6 -0
index 0cb90b5e9f..587093b909 100644
@@ -60,6 +60,10 @@ struct ice_dcf_hw {
 	uint16_t nb_msix;
 	uint16_t rxq_map[16];
 	struct virtchnl_eth_stats eth_stats_offset;
+
+	/* Link status */
+	bool link_up;
+	uint32_t link_speed;
 };
 
 int ice_dcf_execute_virtchnl_cmd(struct ice_dcf_hw *hw,
@@ -77,5 +81,7 @@ int ice_dcf_disable_queues(struct ice_dcf_hw *hw);
 int ice_dcf_query_stats(struct ice_dcf_hw *hw,
 			struct virtchnl_eth_stats *pstats);
 int ice_dcf_add_del_all_mac_addr(struct ice_dcf_hw *hw, bool add);
+int ice_dcf_link_update(struct rte_eth_dev *dev,
+		    __rte_unused int wait_to_complete);
 
 #endif /* _ICE_DCF_H_ */
drivers/net/ice/ice_dcf_ethdev.c +51 -3

62 lines not on this page · raw message

drivers/net/ice/ice_dcf_parent.c +51 -0

63 lines not on this page · raw message

Signature · 1 line
2.25.1
author
#16 lore raw

PATCH v3 4/5 net/ice: support QoS config VF bandwidth in DCF

In reply to Ting Xu

This patch supports the ETS-based QoS configuration. It enables the DCF to configure bandwidth limits for each VF VSI of different TCs. A hierarchy scheduler tree is built with port, TC and VSI nodes.

---
 drivers/net/ice/ice_dcf.c        |   6 +-
 drivers/net/ice/ice_dcf.h        |  47 +++
 drivers/net/ice/ice_dcf_ethdev.c |  13 +
 drivers/net/ice/ice_dcf_ethdev.h |   3 +
 drivers/net/ice/ice_dcf_parent.c |  30 ++
 drivers/net/ice/ice_dcf_sched.c  | 697 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   3 +-
 7 files changed, 797 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
drivers/net/ice/ice_dcf.c +5 -1
index d72a6f357e..4ff2216a5c 100644
@@ -235,7 +235,8 @@ ice_dcf_get_vf_resource(struct ice_dcf_hw *hw)
 	caps = VIRTCHNL_VF_OFFLOAD_WB_ON_ITR | VIRTCHNL_VF_OFFLOAD_RX_POLLING |
 	       VIRTCHNL_VF_CAP_ADV_LINK_SPEED | VIRTCHNL_VF_CAP_DCF |
 	       VIRTCHNL_VF_OFFLOAD_VLAN_V2 |
-	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC;
+	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC |
+	       VIRTCHNL_VF_OFFLOAD_QOS;
 
 	err = ice_dcf_send_cmd_req_no_irq(hw, VIRTCHNL_OP_GET_VF_RESOURCES,
 					  (uint8_t *)&caps, sizeof(caps));
@@ -668,6 +669,9 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 		}
 	}
 
+	if (hw->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS)
+		ice_dcf_tm_conf_init(eth_dev);
+
 	hw->eth_dev = eth_dev;
 	rte_intr_callback_register(&pci_dev->intr_handle,
 				   ice_dcf_dev_interrupt_handler, hw);
drivers/net/ice/ice_dcf.h +47 -0

70 lines not on this page · raw message

drivers/net/ice/ice_dcf_ethdev.c +13 -0
index 0b40ebbec6..cab7c4da87 100644
@@ -994,6 +994,18 @@ ice_dcf_dev_udp_tunnel_port_del(struct rte_eth_dev *dev,
 	return ret;
 }
 
+static int
+ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_dcf_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.dev_start               = ice_dcf_dev_start,
 	.dev_stop                = ice_dcf_dev_stop,
@@ -1018,6 +1030,7 @@ static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.flow_ops_get            = ice_dcf_dev_flow_ops_get,
 	.udp_tunnel_port_add	 = ice_dcf_dev_udp_tunnel_port_add,
 	.udp_tunnel_port_del	 = ice_dcf_dev_udp_tunnel_port_del,
+	.tm_ops_get              = ice_dcf_tm_ops_get,
 };
 
 static int
drivers/net/ice/ice_dcf_ethdev.h +3 -0
index e7c9d7fe41..8510e37119 100644
@@ -7,6 +7,8 @@
 
 #include "base/ice_common.h"
 #include "base/ice_adminq_cmd.h"
+#include "base/ice_dcb.h"
+#include "base/ice_sched.h"
 
 #include "ice_ethdev.h"
 #include "ice_dcf.h"
@@ -52,6 +54,7 @@ struct ice_dcf_vf_repr {
 	struct ice_dcf_vlan outer_vlan_info; /* DCF always handle outer VLAN */
 };
 
+extern const struct rte_tm_ops ice_dcf_tm_ops;
 void ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 				 uint8_t *msg, uint16_t msglen);
 int ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev);
drivers/net/ice/ice_dcf_parent.c +30 -0
index 788f6dd2a0..0ea32cf8e9 100644
@@ -264,6 +264,29 @@ ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 	}
 }
 
+static int
+ice_dcf_query_port_ets(struct ice_hw *parent_hw, struct ice_dcf_hw *real_hw)
+{
+	int ret;
+
+	real_hw->ets_config = (struct ice_aqc_port_ets_elem *)
+			ice_malloc(real_hw, sizeof(*real_hw->ets_config));
+	if (!real_hw->ets_config)
+		return ICE_ERR_NO_MEMORY;
+
+	ret = ice_aq_query_port_ets(parent_hw->port_info,
+			real_hw->ets_config, sizeof(*real_hw->ets_config),
+			NULL);
+	if (ret) {
+		PMD_DRV_LOG(ERR, "DCF Query Port ETS failed");
+		rte_free(real_hw->ets_config);
+		real_hw->ets_config = NULL;
+		return ret;
+	}
+
+	return ICE_SUCCESS;
+}
+
 static int
 ice_dcf_init_parent_hw(struct ice_hw *hw)
 {
@@ -486,6 +509,13 @@ ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev)
 		return err;
 	}
 
+	err = ice_dcf_query_port_ets(parent_hw, hw);
+	if (err) {
+		PMD_INIT_LOG(ERR, "failed to query port ets with error %d",
+			     err);
+		goto uninit_hw;
+	}
+
 	err = ice_dcf_load_pkg(parent_hw);
 	if (err) {
 		PMD_INIT_LOG(ERR, "failed to load package with error %d",
drivers/net/ice/ice_dcf_sched.c new file +697 -0

697 lines not on this page · raw message

drivers/net/ice/meson.build +2 -1
index 65750d3501..0b86d74a49 100644
@@ -70,6 +70,7 @@ endif
 sources += files('ice_dcf.c',
          'ice_dcf_vf_representor.c',
          'ice_dcf_ethdev.c',
-         'ice_dcf_parent.c')
+         'ice_dcf_parent.c',
+	 'ice_dcf_sched.c')
 
 headers = files('rte_pmd_ice.h')
Signature · 1 line
2.25.1
author
#17 lore raw

PATCH v3 5/5 net/iavf: query QoS cap and set queue TC mapping

In reply to Ting Xu

This patch added the support for VF to config the ETS-based Tx QoS, including querying current QoS configuration from PF and config queue TC mapping. PF QoS is configured in advance and the queried info is provided to the user for future usage. VF queues are mapped to different TCs in PF through virtchnl.

---
 drivers/net/iavf/iavf.h        |  45 +++
 drivers/net/iavf/iavf_ethdev.c |  31 ++
 drivers/net/iavf/iavf_tm.c     | 667 +++++++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c  |  56 ++-
 drivers/net/iavf/meson.build   |   1 +
 5 files changed, 799 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
drivers/net/iavf/iavf.h +45 -0
index 4f5811ae87..77ddf15f42 100644
@@ -6,6 +6,8 @@
 #define _IAVF_ETHDEV_H_
 
 #include <rte_kvargs.h>
+#include <rte_tm_driver.h>
+
 #include <iavf_prototype.h>
 #include <iavf_adminq_cmd.h>
 #include <iavf_type.h>
@@ -82,6 +84,8 @@
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_MASK  0x03
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_FD_ID 0x01
 
+#define IAVF_BITS_PER_BYTE 8
+
 struct iavf_adapter;
 struct iavf_rx_queue;
 struct iavf_tx_queue;
@@ -129,6 +133,38 @@ enum iavf_aq_result {
 	IAVF_MSG_CMD,      /* Read async command result */
 };
 
+/* Struct to store Traffic Manager node configuration. */
+struct iavf_tm_node {
+	TAILQ_ENTRY(iavf_tm_node) node;
+	uint32_t id;
+	uint32_t tc;
+	uint32_t priority;
+	uint32_t weight;
+	uint32_t reference_count;
+	struct iavf_tm_node *parent;
+	struct rte_tm_node_params params;
+};
+
+TAILQ_HEAD(iavf_tm_node_list, iavf_tm_node);
+
+/* node type of Traffic Manager */
+enum iavf_tm_node_type {
+	IAVF_TM_NODE_TYPE_PORT,
+	IAVF_TM_NODE_TYPE_TC,
+	IAVF_TM_NODE_TYPE_QUEUE,
+	IAVF_TM_NODE_TYPE_MAX,
+};
+
+/* Struct to store all the Traffic Manager configuration. */
+struct iavf_tm_conf {
+	struct iavf_tm_node *root; /* root node - vf vsi */
+	struct iavf_tm_node_list tc_list; /* node list for all the TCs */
+	struct iavf_tm_node_list queue_list; /* node list for all the queues */
+	uint32_t nb_tc_node;
+	uint32_t nb_queue_node;
+	bool committed;
+};
+
 /* Structure to store private data specific for VF instance. */
 struct iavf_info {
 	uint16_t num_queue_pairs;
@@ -175,6 +211,9 @@ struct iavf_info {
 	struct iavf_fdir_info fdir; /* flow director info */
 	/* indicate large VF support enabled or not */
 	bool lv_enabled;
+
+	struct virtchnl_qos_cap_list *qos_cap;
+	struct iavf_tm_conf tm_conf;
 };
 
 #define IAVF_MAX_PKT_TYPE 1024
@@ -344,4 +383,10 @@ int iavf_add_del_mc_addr_list(struct iavf_adapter *adapter,
 			uint32_t mc_addrs_num, bool add);
 int iavf_request_queues(struct iavf_adapter *adapter, uint16_t num);
 int iavf_get_max_rss_queue_region(struct iavf_adapter *adapter);
+int iavf_get_qos_cap(struct iavf_adapter *adapter);
+int iavf_set_q_tc_map(struct rte_eth_dev *dev,
+			struct virtchnl_queue_tc_mapping *q_tc_mapping,
+			uint16_t size);
+void iavf_tm_conf_init(struct rte_eth_dev *dev);
+extern const struct rte_tm_ops iavf_tm_ops;
 #endif /* _IAVF_ETHDEV_H_ */
drivers/net/iavf/iavf_ethdev.c +31 -0
index 5290588b17..c8ee1a834b 100644
@@ -122,6 +122,7 @@ static int iavf_dev_flow_ops_get(struct rte_eth_dev *dev,
 static int iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
 			uint32_t mc_addrs_num);
+static int iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused, void *arg);
 
 static const struct rte_pci_id pci_id_iavf_map[] = {
 	{ RTE_PCI_DEVICE(IAVF_INTEL_VENDOR_ID, IAVF_DEV_ID_ADAPTIVE_VF) },
@@ -200,8 +201,21 @@ static const struct eth_dev_ops iavf_eth_dev_ops = {
 	.flow_ops_get               = iavf_dev_flow_ops_get,
 	.tx_done_cleanup	    = iavf_dev_tx_done_cleanup,
 	.get_monitor_addr           = iavf_get_monitor_addr,
+	.tm_ops_get                 = iavf_tm_ops_get,
 };
 
+static int
+iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+			void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &iavf_tm_ops;
+
+	return 0;
+}
+
 static int
 iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
@@ -806,6 +820,11 @@ iavf_dev_start(struct rte_eth_dev *dev)
 				      dev->data->nb_tx_queues);
 	num_queue_pairs = vf->num_queue_pairs;
 
+	if (iavf_get_qos_cap(adapter)) {
+		PMD_INIT_LOG(ERR, "Failed to get qos capability");
+		return -1;
+	}
+
 	if (iavf_init_queues(dev) != 0) {
 		PMD_DRV_LOG(ERR, "failed to do Queue init");
 		return -1;
@@ -2090,6 +2109,15 @@ iavf_init_vf(struct rte_eth_dev *dev)
 		PMD_INIT_LOG(ERR, "unable to allocate vf_res memory");
 		goto err_api;
 	}
+
+	bufsz = sizeof(struct virtchnl_qos_cap_list) +
+		IAVF_MAX_TRAFFIC_CLASS * sizeof(struct virtchnl_qos_cap_elem);
+	vf->qos_cap = rte_zmalloc("qos_cap", bufsz, 0);
+	if (!vf->qos_cap) {
+		PMD_INIT_LOG(ERR, "unable to allocate qos_cap memory");
+		goto err_api;
+	}
+
 	if (iavf_get_vf_resource(adapter) != 0) {
 		PMD_INIT_LOG(ERR, "iavf_get_vf_config failed");
 		goto err_alloc;
@@ -2131,6 +2159,7 @@ iavf_init_vf(struct rte_eth_dev *dev)
 	rte_free(vf->rss_key);
 	rte_free(vf->rss_lut);
 err_alloc:
+	rte_free(vf->qos_cap);
 	rte_free(vf->vf_res);
 	vf->vsi_res = NULL;
 err_api:
@@ -2299,6 +2328,8 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 
 	iavf_default_rss_disable(adapter);
 
+	iavf_tm_conf_init(eth_dev);
+
 	return 0;
 }
 
drivers/net/iavf/iavf_tm.c new file +667 -0

667 lines not on this page · raw message

drivers/net/iavf/iavf_vchnl.c +55 -1

68 lines not on this page · raw message

drivers/net/iavf/meson.build +1 -0
index 6f222a9e87..f2010a8337 100644
@@ -19,6 +19,7 @@ sources = files(
         'iavf_generic_flow.c',
         'iavf_fdir.c',
         'iavf_hash.c',
+        'iavf_tm.c',
 )
 
 if arch_subdir == 'x86'
Signature · 1 line
2.25.1
author
#18 lore raw

v4 cover Enable ETS-based Tx QoS for VF in DCF

In reply to Ting Xu

This patch enables the ETS-based Tx QoS for IAVF. Kernel tool is used to configure ETS first. DCF is used to set bandwidth limit for VFs of each TC. IAVF is supported to query QoS capability and set queue TC mapping. Traffic Management API is utilized to configure the QoS hierarchy scheduler tree. The scheduler tree will be passed to hardware to enable all above functions.

Ting Xu (7):
common/iavf: support ETS-based QoS offload configuration net/ice/base: support DCF query port ETS adminq net/ice: support DCF link status event handling net/ice: support QoS config VF bandwidth in DCF net/iavf: query QoS cap and set queue TC mapping net/iavf: check Tx packet with correct UP and queue
doc: release note for ETS-based Tx QoS

 doc/guides/rel_notes/release_21_08.rst |   7 +
 drivers/common/iavf/iavf_type.h        |   2 +
 drivers/common/iavf/virtchnl.h         | 131 +++++
 drivers/net/iavf/iavf.h                |  56 ++
 drivers/net/iavf/iavf_ethdev.c         |  34 ++
 drivers/net/iavf/iavf_rxtx.c           |  43 ++
 drivers/net/iavf/iavf_tm.c             | 727 +++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c          |  56 +-
 drivers/net/iavf/meson.build           |   1 +
 drivers/net/ice/base/ice_dcb.c         |   3 +-
 drivers/net/ice/ice_dcf.c              |   9 +-
 drivers/net/ice/ice_dcf.h              |  54 ++
 drivers/net/ice/ice_dcf_ethdev.c       |  68 ++-
 drivers/net/ice/ice_dcf_ethdev.h       |   3 +
 drivers/net/ice/ice_dcf_parent.c       |  81 +++
 drivers/net/ice/ice_dcf_sched.c        | 759 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   3 +-
 17 files changed, 2030 insertions(+), 7 deletions(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
Signature · 1 line
2.17.1
author
#19 lore raw

PATCH v4 1/7 common/iavf: support ETS-based QoS offload configuration

This patch adds new virtchnl opcodes and structures for QoS configuration, which includes:
1. VIRTCHNL_VF_OFFLOAD_TC, to negotiate the capability supporting QoS configuration. If VF and PF both have this flag, then the ETS-based QoS offload function is supported.
2. VIRTCHNL_OP_DCF_CONFIG_BW, DCF is supposed to configure min and max bandwidth for each VF per enabled TCs. To make the VSI node bandwidth configuration work, DCF also needs to configure TC node bandwidth directly.
3. VIRTCHNL_OP_GET_QOS_CAPS, VF queries current QoS configuration, such as enabled TCs, arbiter type, up2tc and bandwidth of VSI node. The configuration is previously set by DCB and DCF, and now is the potential QoS capability of VF. VF can take it as reference to configure queue TC mapping.
4. VIRTCHNL_OP_CONFIG_TC_MAP, set VF queues to TC mapping for all Tx and Rx queues. Queues mapping to one TC should be continuous and all allocated queues should be mapped.

---
 drivers/common/iavf/iavf_type.h |   2 +
 drivers/common/iavf/virtchnl.h  | 131 ++++++++++++++++++++++++++++++++
 2 files changed, 133 insertions(+)
drivers/common/iavf/iavf_type.h +2 -0
index f3815d523b..73dfb47e70 100644
@@ -141,6 +141,8 @@ enum iavf_debug_mask {
 #define IAVF_PHY_LED_MODE_MASK			0xFFFF
 #define IAVF_PHY_LED_MODE_ORIG			0x80000000
 
+#define IAVF_MAX_TRAFFIC_CLASS	8
+
 /* Memory types */
 enum iavf_memset_type {
 	IAVF_NONDMA_MEM = 0,
drivers/common/iavf/virtchnl.h +131 -0

185 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#20 lore raw

PATCH v4 2/7 net/ice/base: support DCF query port ETS adminq

In reply to Ting Xu

In the adminq command query port ETS function, the root node teid is needed. However, for DCF, the root node is not initialized, which will cause error when we refer to the variable. In this patch, we will check whether the root node is available or not first.

---
 drivers/net/ice/base/ice_dcb.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ice/base/ice_dcb.c +2 -1
index c73fc095ff..9c9675f6ef 100644
@@ -1524,7 +1524,8 @@ ice_aq_query_port_ets(struct ice_port_info *pi,
 		return ICE_ERR_PARAM;
 	cmd = &desc.params.port_ets;
 	ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_query_port_ets);
-	cmd->port_teid = pi->root->info.node_teid;
+	if (pi->root)
+		cmd->port_teid = pi->root->info.node_teid;
 
 	status = ice_aq_send_cmd(pi->hw, &desc, buf, buf_size, cd);
 	return status;
Signature · 1 line
2.17.1
author
#21 lore raw

PATCH v4 3/7 net/ice: support DCF link status event handling

In reply to Ting Xu

When link status changes, DCF will receive virtchnl PF event message. Add support to handle this event, change link status and update link info.

---
 drivers/net/ice/ice_dcf.h        |  6 ++++
 drivers/net/ice/ice_dcf_ethdev.c | 54 ++++++++++++++++++++++++++++++--
 drivers/net/ice/ice_dcf_parent.c | 51 ++++++++++++++++++++++++++++++
 3 files changed, 108 insertions(+), 3 deletions(-)
drivers/net/ice/ice_dcf.h +6 -0
index 0cb90b5e9f..587093b909 100644
@@ -60,6 +60,10 @@ struct ice_dcf_hw {
 	uint16_t nb_msix;
 	uint16_t rxq_map[16];
 	struct virtchnl_eth_stats eth_stats_offset;
+
+	/* Link status */
+	bool link_up;
+	uint32_t link_speed;
 };
 
 int ice_dcf_execute_virtchnl_cmd(struct ice_dcf_hw *hw,
@@ -77,5 +81,7 @@ int ice_dcf_disable_queues(struct ice_dcf_hw *hw);
 int ice_dcf_query_stats(struct ice_dcf_hw *hw,
 			struct virtchnl_eth_stats *pstats);
 int ice_dcf_add_del_all_mac_addr(struct ice_dcf_hw *hw, bool add);
+int ice_dcf_link_update(struct rte_eth_dev *dev,
+		    __rte_unused int wait_to_complete);
 
 #endif /* _ICE_DCF_H_ */
drivers/net/ice/ice_dcf_ethdev.c +51 -3

62 lines not on this page · raw message

drivers/net/ice/ice_dcf_parent.c +51 -0

63 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#22 lore raw

PATCH v4 4/7 net/ice: support QoS config VF bandwidth in DCF

In reply to Ting Xu

This patch supports the ETS-based QoS configuration. It enables the DCF to configure bandwidth limits for each VF VSI of different TCs. A hierarchy scheduler tree is built with port, TC and VSI nodes.

---
 drivers/net/ice/ice_dcf.c        |   9 +-
 drivers/net/ice/ice_dcf.h        |  48 ++
 drivers/net/ice/ice_dcf_ethdev.c |  14 +
 drivers/net/ice/ice_dcf_ethdev.h |   3 +
 drivers/net/ice/ice_dcf_parent.c |  30 ++
 drivers/net/ice/ice_dcf_sched.c  | 759 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   3 +-
 7 files changed, 864 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
drivers/net/ice/ice_dcf.c +8 -1
index d72a6f357e..349d23ee4f 100644
@@ -235,7 +235,8 @@ ice_dcf_get_vf_resource(struct ice_dcf_hw *hw)
 	caps = VIRTCHNL_VF_OFFLOAD_WB_ON_ITR | VIRTCHNL_VF_OFFLOAD_RX_POLLING |
 	       VIRTCHNL_VF_CAP_ADV_LINK_SPEED | VIRTCHNL_VF_CAP_DCF |
 	       VIRTCHNL_VF_OFFLOAD_VLAN_V2 |
-	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC;
+	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC |
+	       VIRTCHNL_VF_OFFLOAD_QOS;
 
 	err = ice_dcf_send_cmd_req_no_irq(hw, VIRTCHNL_OP_GET_VF_RESOURCES,
 					  (uint8_t *)&caps, sizeof(caps));
@@ -668,6 +669,9 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 		}
 	}
 
+	if (hw->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS)
+		ice_dcf_tm_conf_init(eth_dev);
+
 	hw->eth_dev = eth_dev;
 	rte_intr_callback_register(&pci_dev->intr_handle,
 				   ice_dcf_dev_interrupt_handler, hw);
@@ -703,6 +707,9 @@ ice_dcf_uninit_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 	ice_dcf_mode_disable(hw);
 	iavf_shutdown_adminq(&hw->avf);
 
+	if (hw->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS)
+		ice_dcf_tm_conf_uninit(eth_dev);
+
 	rte_free(hw->arq_buf);
 	rte_free(hw->vf_vsi_map);
 	rte_free(hw->vf_res);
drivers/net/ice/ice_dcf.h +48 -0

71 lines not on this page · raw message

drivers/net/ice/ice_dcf_ethdev.c +14 -0
index 0b40ebbec6..69fe6e63d1 100644
@@ -622,6 +622,7 @@ ice_dcf_dev_stop(struct rte_eth_dev *dev)
 	ice_dcf_add_del_all_mac_addr(&dcf_ad->real_hw, false);
 	dev->data->dev_link.link_status = ETH_LINK_DOWN;
 	ad->pf.adapter_stopped = 1;
+	dcf_ad->real_hw.tm_conf.committed = false;
 
 	return 0;
 }
@@ -994,6 +995,18 @@ ice_dcf_dev_udp_tunnel_port_del(struct rte_eth_dev *dev,
 	return ret;
 }
 
+static int
+ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_dcf_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.dev_start               = ice_dcf_dev_start,
 	.dev_stop                = ice_dcf_dev_stop,
@@ -1018,6 +1031,7 @@ static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.flow_ops_get            = ice_dcf_dev_flow_ops_get,
 	.udp_tunnel_port_add	 = ice_dcf_dev_udp_tunnel_port_add,
 	.udp_tunnel_port_del	 = ice_dcf_dev_udp_tunnel_port_del,
+	.tm_ops_get              = ice_dcf_tm_ops_get,
 };
 
 static int
drivers/net/ice/ice_dcf_ethdev.h +3 -0
index e7c9d7fe41..8510e37119 100644
@@ -7,6 +7,8 @@
 
 #include "base/ice_common.h"
 #include "base/ice_adminq_cmd.h"
+#include "base/ice_dcb.h"
+#include "base/ice_sched.h"
 
 #include "ice_ethdev.h"
 #include "ice_dcf.h"
@@ -52,6 +54,7 @@ struct ice_dcf_vf_repr {
 	struct ice_dcf_vlan outer_vlan_info; /* DCF always handle outer VLAN */
 };
 
+extern const struct rte_tm_ops ice_dcf_tm_ops;
 void ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 				 uint8_t *msg, uint16_t msglen);
 int ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev);
drivers/net/ice/ice_dcf_parent.c +30 -0
index 788f6dd2a0..0ea32cf8e9 100644
@@ -264,6 +264,29 @@ ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 	}
 }
 
+static int
+ice_dcf_query_port_ets(struct ice_hw *parent_hw, struct ice_dcf_hw *real_hw)
+{
+	int ret;
+
+	real_hw->ets_config = (struct ice_aqc_port_ets_elem *)
+			ice_malloc(real_hw, sizeof(*real_hw->ets_config));
+	if (!real_hw->ets_config)
+		return ICE_ERR_NO_MEMORY;
+
+	ret = ice_aq_query_port_ets(parent_hw->port_info,
+			real_hw->ets_config, sizeof(*real_hw->ets_config),
+			NULL);
+	if (ret) {
+		PMD_DRV_LOG(ERR, "DCF Query Port ETS failed");
+		rte_free(real_hw->ets_config);
+		real_hw->ets_config = NULL;
+		return ret;
+	}
+
+	return ICE_SUCCESS;
+}
+
 static int
 ice_dcf_init_parent_hw(struct ice_hw *hw)
 {
@@ -486,6 +509,13 @@ ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev)
 		return err;
 	}
 
+	err = ice_dcf_query_port_ets(parent_hw, hw);
+	if (err) {
+		PMD_INIT_LOG(ERR, "failed to query port ets with error %d",
+			     err);
+		goto uninit_hw;
+	}
+
 	err = ice_dcf_load_pkg(parent_hw);
 	if (err) {
 		PMD_INIT_LOG(ERR, "failed to load package with error %d",
drivers/net/ice/ice_dcf_sched.c new file +759 -0

759 lines not on this page · raw message

drivers/net/ice/meson.build +2 -1
index 65750d3501..0b86d74a49 100644
@@ -70,6 +70,7 @@ endif
 sources += files('ice_dcf.c',
          'ice_dcf_vf_representor.c',
          'ice_dcf_ethdev.c',
-         'ice_dcf_parent.c')
+         'ice_dcf_parent.c',
+	 'ice_dcf_sched.c')
 
 headers = files('rte_pmd_ice.h')
Signature · 1 line
2.17.1
author
#23 lore raw

PATCH v4 5/7 net/iavf: query QoS cap and set queue TC mapping

In reply to Ting Xu

This patch added the support for VF to config the ETS-based Tx QoS, including querying current QoS configuration from PF and config queue TC mapping. PF QoS is configured in advance and the queried info is provided to the user for future usage. VF queues are mapped to different TCs in PF through virtchnl.

---
 drivers/net/iavf/iavf.h        |  46 +++
 drivers/net/iavf/iavf_ethdev.c |  34 ++
 drivers/net/iavf/iavf_tm.c     | 714 +++++++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c  |  56 ++-
 drivers/net/iavf/meson.build   |   1 +
 5 files changed, 850 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
drivers/net/iavf/iavf.h +46 -0
index 4f5811ae87..feb8337b55 100644
@@ -6,6 +6,8 @@
 #define _IAVF_ETHDEV_H_
 
 #include <rte_kvargs.h>
+#include <rte_tm_driver.h>
+
 #include <iavf_prototype.h>
 #include <iavf_adminq_cmd.h>
 #include <iavf_type.h>
@@ -82,6 +84,8 @@
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_MASK  0x03
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_FD_ID 0x01
 
+#define IAVF_BITS_PER_BYTE 8
+
 struct iavf_adapter;
 struct iavf_rx_queue;
 struct iavf_tx_queue;
@@ -129,6 +133,38 @@ enum iavf_aq_result {
 	IAVF_MSG_CMD,      /* Read async command result */
 };
 
+/* Struct to store Traffic Manager node configuration. */
+struct iavf_tm_node {
+	TAILQ_ENTRY(iavf_tm_node) node;
+	uint32_t id;
+	uint32_t tc;
+	uint32_t priority;
+	uint32_t weight;
+	uint32_t reference_count;
+	struct iavf_tm_node *parent;
+	struct rte_tm_node_params params;
+};
+
+TAILQ_HEAD(iavf_tm_node_list, iavf_tm_node);
+
+/* node type of Traffic Manager */
+enum iavf_tm_node_type {
+	IAVF_TM_NODE_TYPE_PORT,
+	IAVF_TM_NODE_TYPE_TC,
+	IAVF_TM_NODE_TYPE_QUEUE,
+	IAVF_TM_NODE_TYPE_MAX,
+};
+
+/* Struct to store all the Traffic Manager configuration. */
+struct iavf_tm_conf {
+	struct iavf_tm_node *root; /* root node - vf vsi */
+	struct iavf_tm_node_list tc_list; /* node list for all the TCs */
+	struct iavf_tm_node_list queue_list; /* node list for all the queues */
+	uint32_t nb_tc_node;
+	uint32_t nb_queue_node;
+	bool committed;
+};
+
 /* Structure to store private data specific for VF instance. */
 struct iavf_info {
 	uint16_t num_queue_pairs;
@@ -175,6 +211,9 @@ struct iavf_info {
 	struct iavf_fdir_info fdir; /* flow director info */
 	/* indicate large VF support enabled or not */
 	bool lv_enabled;
+
+	struct virtchnl_qos_cap_list *qos_cap;
+	struct iavf_tm_conf tm_conf;
 };
 
 #define IAVF_MAX_PKT_TYPE 1024
@@ -344,4 +383,11 @@ int iavf_add_del_mc_addr_list(struct iavf_adapter *adapter,
 			uint32_t mc_addrs_num, bool add);
 int iavf_request_queues(struct iavf_adapter *adapter, uint16_t num);
 int iavf_get_max_rss_queue_region(struct iavf_adapter *adapter);
+int iavf_get_qos_cap(struct iavf_adapter *adapter);
+int iavf_set_q_tc_map(struct rte_eth_dev *dev,
+			struct virtchnl_queue_tc_mapping *q_tc_mapping,
+			uint16_t size);
+void iavf_tm_conf_init(struct rte_eth_dev *dev);
+void iavf_tm_conf_uninit(struct rte_eth_dev *dev);
+extern const struct rte_tm_ops iavf_tm_ops;
 #endif /* _IAVF_ETHDEV_H_ */
drivers/net/iavf/iavf_ethdev.c +34 -0
index 5290588b17..93f82773e2 100644
@@ -122,6 +122,7 @@ static int iavf_dev_flow_ops_get(struct rte_eth_dev *dev,
 static int iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
 			uint32_t mc_addrs_num);
+static int iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused, void *arg);
 
 static const struct rte_pci_id pci_id_iavf_map[] = {
 	{ RTE_PCI_DEVICE(IAVF_INTEL_VENDOR_ID, IAVF_DEV_ID_ADAPTIVE_VF) },
@@ -200,8 +201,21 @@ static const struct eth_dev_ops iavf_eth_dev_ops = {
 	.flow_ops_get               = iavf_dev_flow_ops_get,
 	.tx_done_cleanup	    = iavf_dev_tx_done_cleanup,
 	.get_monitor_addr           = iavf_get_monitor_addr,
+	.tm_ops_get                 = iavf_tm_ops_get,
 };
 
+static int
+iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+			void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &iavf_tm_ops;
+
+	return 0;
+}
+
 static int
 iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
@@ -806,6 +820,11 @@ iavf_dev_start(struct rte_eth_dev *dev)
 				      dev->data->nb_tx_queues);
 	num_queue_pairs = vf->num_queue_pairs;
 
+	if (iavf_get_qos_cap(adapter)) {
+		PMD_INIT_LOG(ERR, "Failed to get qos capability");
+		return -1;
+	}
+
 	if (iavf_init_queues(dev) != 0) {
 		PMD_DRV_LOG(ERR, "failed to do Queue init");
 		return -1;
@@ -891,6 +910,7 @@ iavf_dev_stop(struct rte_eth_dev *dev)
 
 	adapter->stopped = 1;
 	dev->data->dev_started = 0;
+	vf->tm_conf.committed = false;
 
 	return 0;
 }
@@ -2090,6 +2110,15 @@ iavf_init_vf(struct rte_eth_dev *dev)
 		PMD_INIT_LOG(ERR, "unable to allocate vf_res memory");
 		goto err_api;
 	}
+
+	bufsz = sizeof(struct virtchnl_qos_cap_list) +
+		IAVF_MAX_TRAFFIC_CLASS * sizeof(struct virtchnl_qos_cap_elem);
+	vf->qos_cap = rte_zmalloc("qos_cap", bufsz, 0);
+	if (!vf->qos_cap) {
+		PMD_INIT_LOG(ERR, "unable to allocate qos_cap memory");
+		goto err_api;
+	}
+
 	if (iavf_get_vf_resource(adapter) != 0) {
 		PMD_INIT_LOG(ERR, "iavf_get_vf_config failed");
 		goto err_alloc;
@@ -2131,6 +2160,7 @@ iavf_init_vf(struct rte_eth_dev *dev)
 	rte_free(vf->rss_key);
 	rte_free(vf->rss_lut);
 err_alloc:
+	rte_free(vf->qos_cap);
 	rte_free(vf->vf_res);
 	vf->vsi_res = NULL;
 err_api:
@@ -2299,6 +2329,8 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 
 	iavf_default_rss_disable(adapter);
 
+	iavf_tm_conf_init(eth_dev);
+
 	return 0;
 }
 
@@ -2338,6 +2370,8 @@ iavf_dev_close(struct rte_eth_dev *dev)
 				     iavf_dev_interrupt_handler, dev);
 	iavf_disable_irq0(hw);
 
+	iavf_tm_conf_uninit(dev);
+
 	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
 		if (vf->rss_lut) {
 			rte_free(vf->rss_lut);
drivers/net/iavf/iavf_tm.c new file +714 -0

714 lines not on this page · raw message

drivers/net/iavf/iavf_vchnl.c +55 -1

68 lines not on this page · raw message

drivers/net/iavf/meson.build +1 -0
index 6f222a9e87..f2010a8337 100644
@@ -19,6 +19,7 @@ sources = files(
         'iavf_generic_flow.c',
         'iavf_fdir.c',
         'iavf_hash.c',
+        'iavf_tm.c',
 )
 
 if arch_subdir == 'x86'
Signature · 1 line
2.17.1
author
#24 lore raw

PATCH v4 6/7 net/iavf: check Tx packet with correct UP and queue

In reply to Ting Xu

Add check in the Tx packet preparation function, to guarantee that the packet with specific user priority is distributed to the correct Tx queue according to the configured Tx queue TC mapping.

---
 drivers/net/iavf/iavf.h      | 10 +++++++++
 drivers/net/iavf/iavf_rxtx.c | 43 ++++++++++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_tm.c   | 13 +++++++++++
 3 files changed, 66 insertions(+)
drivers/net/iavf/iavf.h +10 -0
index feb8337b55..b3bd078111 100644
@@ -86,6 +86,8 @@
 
 #define IAVF_BITS_PER_BYTE 8
 
+#define IAVF_VLAN_TAG_PCP_OFFSET 13
+
 struct iavf_adapter;
 struct iavf_rx_queue;
 struct iavf_tx_queue;
@@ -165,6 +167,13 @@ struct iavf_tm_conf {
 	bool committed;
 };
 
+/* Struct to store queue TC mapping. Queue is continuous in one TC */
+struct iavf_qtc_map {
+	uint8_t	tc;
+	uint16_t start_queue_id;
+	uint16_t queue_count;
+};
+
 /* Structure to store private data specific for VF instance. */
 struct iavf_info {
 	uint16_t num_queue_pairs;
@@ -213,6 +222,7 @@ struct iavf_info {
 	bool lv_enabled;
 
 	struct virtchnl_qos_cap_list *qos_cap;
+	struct iavf_qtc_map *qtc_map;
 	struct iavf_tm_conf tm_conf;
 };
 
drivers/net/iavf/iavf_rxtx.c +43 -0

63 lines not on this page · raw message

drivers/net/iavf/iavf_tm.c +13 -0
index a8fc142c89..185b37b970 100644
@@ -653,6 +653,7 @@ static int iavf_hierarchy_commit(struct rte_eth_dev *dev,
 	struct virtchnl_queue_tc_mapping *q_tc_mapping;
 	struct iavf_tm_node_list *queue_list = &vf->tm_conf.queue_list;
 	struct iavf_tm_node *tm_node;
+	struct iavf_qtc_map *qtc_map;
 	uint16_t size;
 	int index = 0, node_committed = 0;
 	int i, ret_val = IAVF_SUCCESS;
@@ -675,6 +676,7 @@ static int iavf_hierarchy_commit(struct rte_eth_dev *dev,
 	q_tc_mapping->vsi_id = vf->vsi.vsi_id;
 	q_tc_mapping->num_tc = vf->qos_cap->num_elem;
 	q_tc_mapping->num_queue_pairs = vf->num_queue_pairs;
+
 	TAILQ_FOREACH(tm_node, queue_list, node) {
 		if (tm_node->tc >= q_tc_mapping->num_tc) {
 			PMD_DRV_LOG(ERR, "TC%d is not enabled", tm_node->tc);
@@ -692,15 +694,26 @@ static int iavf_hierarchy_commit(struct rte_eth_dev *dev,
 		goto fail_clear;
 	}
 
+	/* store the queue TC mapping info */
+	qtc_map = rte_zmalloc("qtc_map",
+		sizeof(struct iavf_qtc_map) * q_tc_mapping->num_tc, 0);
+	if (!qtc_map)
+		return IAVF_ERR_NO_MEMORY;
+
 	for (i = 0; i < q_tc_mapping->num_tc; i++) {
 		q_tc_mapping->tc[i].req.start_queue_id = index;
 		index += q_tc_mapping->tc[i].req.queue_count;
+		qtc_map[i].tc = i;
+		qtc_map[i].start_queue_id =
+			q_tc_mapping->tc[i].req.start_queue_id;
+		qtc_map[i].queue_count = q_tc_mapping->tc[i].req.queue_count;
 	}
 
 	ret_val = iavf_set_q_tc_map(dev, q_tc_mapping, size);
 	if (ret_val)
 		goto fail_clear;
 
+	vf->qtc_map = qtc_map;
 	vf->tm_conf.committed = true;
 	return ret_val;
 
Signature · 1 line
2.17.1
author
#25 lore raw

PATCH v4 7/7 doc: release note for ETS-based Tx QoS

In reply to Ting Xu

Add 21.08 release note for ETS-based Tx QoS

---
 doc/guides/rel_notes/release_21_08.rst | 7 +++++++
 1 file changed, 7 insertions(+)
doc/guides/rel_notes/release_21_08.rst +7 -0
index a6ecfdf3ce..35aa76a270 100644
@@ -55,6 +55,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Intel iavf driver.**
+
+  * Added Tx QoS VF queue TC mapping.
+
+* **Updated Intel ice driver.**
+
+  * Added Tx QoS TC bandwidth configuration in DCF.
 
 Removed Items
 -------------
Signature · 1 line
2.17.1
author
#26 lore raw

v5 cover Enable ETS-based Tx QoS for VF in DCF

In reply to Ting Xu

This patch enables the ETS-based Tx QoS for IAVF. Kernel tool is used to configure ETS first. DCF is used to set bandwidth limit for VFs of each TC. IAVF is supported to query QoS capability and set queue TC mapping. Traffic Management API is utilized to configure the QoS hierarchy scheduler tree. The scheduler tree will be passed to hardware to enable all above functions.

Ting Xu (7):
common/iavf: support ETS-based QoS offload configuration net/ice/base: support DCF query port ETS adminq net/ice: support DCF link status event handling net/ice: support QoS config VF bandwidth in DCF net/iavf: query QoS cap and set queue TC mapping net/iavf: check Tx packet with correct UP and queue
doc: release note for ETS-based Tx QoS

 doc/guides/rel_notes/release_21_08.rst |   7 +
 drivers/common/iavf/iavf_type.h        |   2 +
 drivers/common/iavf/virtchnl.h         | 131 +++++
 drivers/net/iavf/iavf.h                |  56 ++
 drivers/net/iavf/iavf_ethdev.c         |  34 ++
 drivers/net/iavf/iavf_rxtx.c           |  43 ++
 drivers/net/iavf/iavf_tm.c             | 737 ++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c          |  56 +-
 drivers/net/iavf/meson.build           |   1 +
 drivers/net/ice/base/ice_dcb.c         |   3 +-
 drivers/net/ice/ice_dcf.c              |   9 +-
 drivers/net/ice/ice_dcf.h              |  54 ++
 drivers/net/ice/ice_dcf_ethdev.c       |  68 ++-
 drivers/net/ice/ice_dcf_ethdev.h       |   3 +
 drivers/net/ice/ice_dcf_parent.c       |  81 +++
 drivers/net/ice/ice_dcf_sched.c        | 759 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   3 +-
 17 files changed, 2040 insertions(+), 7 deletions(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
Signature · 1 line
2.17.1
author
#27 lore raw

PATCH v5 1/7 common/iavf: support ETS-based QoS offload configuration

This patch adds new virtchnl opcodes and structures for QoS configuration, which includes:
1. VIRTCHNL_VF_OFFLOAD_TC, to negotiate the capability supporting QoS configuration. If VF and PF both have this flag, then the ETS-based QoS offload function is supported.
2. VIRTCHNL_OP_DCF_CONFIG_BW, DCF is supposed to configure min and max bandwidth for each VF per enabled TCs. To make the VSI node bandwidth configuration work, DCF also needs to configure TC node bandwidth directly.
3. VIRTCHNL_OP_GET_QOS_CAPS, VF queries current QoS configuration, such as enabled TCs, arbiter type, up2tc and bandwidth of VSI node. The configuration is previously set by DCB and DCF, and now is the potential QoS capability of VF. VF can take it as reference to configure queue TC mapping.
4. VIRTCHNL_OP_CONFIG_TC_MAP, set VF queues to TC mapping for all Tx and Rx queues. Queues mapping to one TC should be continuous and all allocated queues should be mapped.

---
 drivers/common/iavf/iavf_type.h |   2 +
 drivers/common/iavf/virtchnl.h  | 131 ++++++++++++++++++++++++++++++++
 2 files changed, 133 insertions(+)
drivers/common/iavf/iavf_type.h +2 -0
index f3815d523b..73dfb47e70 100644
@@ -141,6 +141,8 @@ enum iavf_debug_mask {
 #define IAVF_PHY_LED_MODE_MASK			0xFFFF
 #define IAVF_PHY_LED_MODE_ORIG			0x80000000
 
+#define IAVF_MAX_TRAFFIC_CLASS	8
+
 /* Memory types */
 enum iavf_memset_type {
 	IAVF_NONDMA_MEM = 0,
drivers/common/iavf/virtchnl.h +131 -0

185 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#28 lore raw

PATCH v5 2/7 net/ice/base: support DCF query port ETS adminq

In reply to Ting Xu

In the adminq command query port ETS function, the root node teid is needed. However, for DCF, the root node is not initialized, which will cause error when we refer to the variable. In this patch, we will check whether the root node is available or not first.

---
 drivers/net/ice/base/ice_dcb.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ice/base/ice_dcb.c +2 -1
index c73fc095ff..9c9675f6ef 100644
@@ -1524,7 +1524,8 @@ ice_aq_query_port_ets(struct ice_port_info *pi,
 		return ICE_ERR_PARAM;
 	cmd = &desc.params.port_ets;
 	ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_query_port_ets);
-	cmd->port_teid = pi->root->info.node_teid;
+	if (pi->root)
+		cmd->port_teid = pi->root->info.node_teid;
 
 	status = ice_aq_send_cmd(pi->hw, &desc, buf, buf_size, cd);
 	return status;
Signature · 1 line
2.17.1
author
#29 lore raw

PATCH v5 3/7 net/ice: support DCF link status event handling

In reply to Ting Xu

When link status changes, DCF will receive virtchnl PF event message. Add support to handle this event, change link status and update link info.

---
 drivers/net/ice/ice_dcf.h        |  6 ++++
 drivers/net/ice/ice_dcf_ethdev.c | 54 ++++++++++++++++++++++++++++++--
 drivers/net/ice/ice_dcf_parent.c | 51 ++++++++++++++++++++++++++++++
 3 files changed, 108 insertions(+), 3 deletions(-)
drivers/net/ice/ice_dcf.h +6 -0
index 0cb90b5e9f..587093b909 100644
@@ -60,6 +60,10 @@ struct ice_dcf_hw {
 	uint16_t nb_msix;
 	uint16_t rxq_map[16];
 	struct virtchnl_eth_stats eth_stats_offset;
+
+	/* Link status */
+	bool link_up;
+	uint32_t link_speed;
 };
 
 int ice_dcf_execute_virtchnl_cmd(struct ice_dcf_hw *hw,
@@ -77,5 +81,7 @@ int ice_dcf_disable_queues(struct ice_dcf_hw *hw);
 int ice_dcf_query_stats(struct ice_dcf_hw *hw,
 			struct virtchnl_eth_stats *pstats);
 int ice_dcf_add_del_all_mac_addr(struct ice_dcf_hw *hw, bool add);
+int ice_dcf_link_update(struct rte_eth_dev *dev,
+		    __rte_unused int wait_to_complete);
 
 #endif /* _ICE_DCF_H_ */
drivers/net/ice/ice_dcf_ethdev.c +51 -3

62 lines not on this page · raw message

drivers/net/ice/ice_dcf_parent.c +51 -0

63 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#30 lore raw

PATCH v5 4/7 net/ice: support QoS config VF bandwidth in DCF

In reply to Ting Xu

This patch supports the ETS-based QoS configuration. It enables the DCF to configure bandwidth limits for each VF VSI of different TCs. A hierarchy scheduler tree is built with port, TC and VSI nodes.

---
 drivers/net/ice/ice_dcf.c        |   9 +-
 drivers/net/ice/ice_dcf.h        |  48 ++
 drivers/net/ice/ice_dcf_ethdev.c |  14 +
 drivers/net/ice/ice_dcf_ethdev.h |   3 +
 drivers/net/ice/ice_dcf_parent.c |  30 ++
 drivers/net/ice/ice_dcf_sched.c  | 759 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   3 +-
 7 files changed, 864 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
drivers/net/ice/ice_dcf.c +8 -1
index d72a6f357e..349d23ee4f 100644
@@ -235,7 +235,8 @@ ice_dcf_get_vf_resource(struct ice_dcf_hw *hw)
 	caps = VIRTCHNL_VF_OFFLOAD_WB_ON_ITR | VIRTCHNL_VF_OFFLOAD_RX_POLLING |
 	       VIRTCHNL_VF_CAP_ADV_LINK_SPEED | VIRTCHNL_VF_CAP_DCF |
 	       VIRTCHNL_VF_OFFLOAD_VLAN_V2 |
-	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC;
+	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC |
+	       VIRTCHNL_VF_OFFLOAD_QOS;
 
 	err = ice_dcf_send_cmd_req_no_irq(hw, VIRTCHNL_OP_GET_VF_RESOURCES,
 					  (uint8_t *)&caps, sizeof(caps));
@@ -668,6 +669,9 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 		}
 	}
 
+	if (hw->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS)
+		ice_dcf_tm_conf_init(eth_dev);
+
 	hw->eth_dev = eth_dev;
 	rte_intr_callback_register(&pci_dev->intr_handle,
 				   ice_dcf_dev_interrupt_handler, hw);
@@ -703,6 +707,9 @@ ice_dcf_uninit_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 	ice_dcf_mode_disable(hw);
 	iavf_shutdown_adminq(&hw->avf);
 
+	if (hw->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS)
+		ice_dcf_tm_conf_uninit(eth_dev);
+
 	rte_free(hw->arq_buf);
 	rte_free(hw->vf_vsi_map);
 	rte_free(hw->vf_res);
drivers/net/ice/ice_dcf.h +48 -0

71 lines not on this page · raw message

drivers/net/ice/ice_dcf_ethdev.c +14 -0
index 0b40ebbec6..69fe6e63d1 100644
@@ -622,6 +622,7 @@ ice_dcf_dev_stop(struct rte_eth_dev *dev)
 	ice_dcf_add_del_all_mac_addr(&dcf_ad->real_hw, false);
 	dev->data->dev_link.link_status = ETH_LINK_DOWN;
 	ad->pf.adapter_stopped = 1;
+	dcf_ad->real_hw.tm_conf.committed = false;
 
 	return 0;
 }
@@ -994,6 +995,18 @@ ice_dcf_dev_udp_tunnel_port_del(struct rte_eth_dev *dev,
 	return ret;
 }
 
+static int
+ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_dcf_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.dev_start               = ice_dcf_dev_start,
 	.dev_stop                = ice_dcf_dev_stop,
@@ -1018,6 +1031,7 @@ static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.flow_ops_get            = ice_dcf_dev_flow_ops_get,
 	.udp_tunnel_port_add	 = ice_dcf_dev_udp_tunnel_port_add,
 	.udp_tunnel_port_del	 = ice_dcf_dev_udp_tunnel_port_del,
+	.tm_ops_get              = ice_dcf_tm_ops_get,
 };
 
 static int
drivers/net/ice/ice_dcf_ethdev.h +3 -0
index e7c9d7fe41..8510e37119 100644
@@ -7,6 +7,8 @@
 
 #include "base/ice_common.h"
 #include "base/ice_adminq_cmd.h"
+#include "base/ice_dcb.h"
+#include "base/ice_sched.h"
 
 #include "ice_ethdev.h"
 #include "ice_dcf.h"
@@ -52,6 +54,7 @@ struct ice_dcf_vf_repr {
 	struct ice_dcf_vlan outer_vlan_info; /* DCF always handle outer VLAN */
 };
 
+extern const struct rte_tm_ops ice_dcf_tm_ops;
 void ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 				 uint8_t *msg, uint16_t msglen);
 int ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev);
drivers/net/ice/ice_dcf_parent.c +30 -0
index 788f6dd2a0..0ea32cf8e9 100644
@@ -264,6 +264,29 @@ ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 	}
 }
 
+static int
+ice_dcf_query_port_ets(struct ice_hw *parent_hw, struct ice_dcf_hw *real_hw)
+{
+	int ret;
+
+	real_hw->ets_config = (struct ice_aqc_port_ets_elem *)
+			ice_malloc(real_hw, sizeof(*real_hw->ets_config));
+	if (!real_hw->ets_config)
+		return ICE_ERR_NO_MEMORY;
+
+	ret = ice_aq_query_port_ets(parent_hw->port_info,
+			real_hw->ets_config, sizeof(*real_hw->ets_config),
+			NULL);
+	if (ret) {
+		PMD_DRV_LOG(ERR, "DCF Query Port ETS failed");
+		rte_free(real_hw->ets_config);
+		real_hw->ets_config = NULL;
+		return ret;
+	}
+
+	return ICE_SUCCESS;
+}
+
 static int
 ice_dcf_init_parent_hw(struct ice_hw *hw)
 {
@@ -486,6 +509,13 @@ ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev)
 		return err;
 	}
 
+	err = ice_dcf_query_port_ets(parent_hw, hw);
+	if (err) {
+		PMD_INIT_LOG(ERR, "failed to query port ets with error %d",
+			     err);
+		goto uninit_hw;
+	}
+
 	err = ice_dcf_load_pkg(parent_hw);
 	if (err) {
 		PMD_INIT_LOG(ERR, "failed to load package with error %d",
drivers/net/ice/ice_dcf_sched.c new file +759 -0

759 lines not on this page · raw message

drivers/net/ice/meson.build +2 -1
index 65750d3501..0b86d74a49 100644
@@ -70,6 +70,7 @@ endif
 sources += files('ice_dcf.c',
          'ice_dcf_vf_representor.c',
          'ice_dcf_ethdev.c',
-         'ice_dcf_parent.c')
+         'ice_dcf_parent.c',
+	 'ice_dcf_sched.c')
 
 headers = files('rte_pmd_ice.h')
Signature · 1 line
2.17.1
#31 lore raw

Re: [dpdk-dev] [PATCH v5 4/7] net/ice: support QoS config VF bandwidth in DCF

01/07/2021 12:20, Ting Xu:

Quoted 6 lines

This patch supports the ETS-based QoS configuration. It enables the DCF to configure bandwidth limits for each VF VSI of different TCs. A hierarchy scheduler tree is built with port, TC and VSI nodes.

This patch has been merged without checking the CI report:
http://mails.dpdk.org/archives/test-report/2021-July/201905.html

As a result, the compilation is now failing in the main branch on RHEL. Please fix it today, it is blocking the -rc1.

author
#32 lore raw

PATCH v5 5/7 net/iavf: query QoS cap and set queue TC mapping

In reply to Ting Xu

This patch added the support for VF to config the ETS-based Tx QoS, including querying current QoS configuration from PF and config queue TC mapping. PF QoS is configured in advance and the queried info is provided to the user for future usage. VF queues are mapped to different TCs in PF through virtchnl.

---
 drivers/net/iavf/iavf.h        |  46 +++
 drivers/net/iavf/iavf_ethdev.c |  34 ++
 drivers/net/iavf/iavf_tm.c     | 724 +++++++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c  |  56 ++-
 drivers/net/iavf/meson.build   |   1 +
 5 files changed, 860 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
drivers/net/iavf/iavf.h +46 -0
index 4f5811ae87..feb8337b55 100644
@@ -6,6 +6,8 @@
 #define _IAVF_ETHDEV_H_
 
 #include <rte_kvargs.h>
+#include <rte_tm_driver.h>
+
 #include <iavf_prototype.h>
 #include <iavf_adminq_cmd.h>
 #include <iavf_type.h>
@@ -82,6 +84,8 @@
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_MASK  0x03
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_FD_ID 0x01
 
+#define IAVF_BITS_PER_BYTE 8
+
 struct iavf_adapter;
 struct iavf_rx_queue;
 struct iavf_tx_queue;
@@ -129,6 +133,38 @@ enum iavf_aq_result {
 	IAVF_MSG_CMD,      /* Read async command result */
 };
 
+/* Struct to store Traffic Manager node configuration. */
+struct iavf_tm_node {
+	TAILQ_ENTRY(iavf_tm_node) node;
+	uint32_t id;
+	uint32_t tc;
+	uint32_t priority;
+	uint32_t weight;
+	uint32_t reference_count;
+	struct iavf_tm_node *parent;
+	struct rte_tm_node_params params;
+};
+
+TAILQ_HEAD(iavf_tm_node_list, iavf_tm_node);
+
+/* node type of Traffic Manager */
+enum iavf_tm_node_type {
+	IAVF_TM_NODE_TYPE_PORT,
+	IAVF_TM_NODE_TYPE_TC,
+	IAVF_TM_NODE_TYPE_QUEUE,
+	IAVF_TM_NODE_TYPE_MAX,
+};
+
+/* Struct to store all the Traffic Manager configuration. */
+struct iavf_tm_conf {
+	struct iavf_tm_node *root; /* root node - vf vsi */
+	struct iavf_tm_node_list tc_list; /* node list for all the TCs */
+	struct iavf_tm_node_list queue_list; /* node list for all the queues */
+	uint32_t nb_tc_node;
+	uint32_t nb_queue_node;
+	bool committed;
+};
+
 /* Structure to store private data specific for VF instance. */
 struct iavf_info {
 	uint16_t num_queue_pairs;
@@ -175,6 +211,9 @@ struct iavf_info {
 	struct iavf_fdir_info fdir; /* flow director info */
 	/* indicate large VF support enabled or not */
 	bool lv_enabled;
+
+	struct virtchnl_qos_cap_list *qos_cap;
+	struct iavf_tm_conf tm_conf;
 };
 
 #define IAVF_MAX_PKT_TYPE 1024
@@ -344,4 +383,11 @@ int iavf_add_del_mc_addr_list(struct iavf_adapter *adapter,
 			uint32_t mc_addrs_num, bool add);
 int iavf_request_queues(struct iavf_adapter *adapter, uint16_t num);
 int iavf_get_max_rss_queue_region(struct iavf_adapter *adapter);
+int iavf_get_qos_cap(struct iavf_adapter *adapter);
+int iavf_set_q_tc_map(struct rte_eth_dev *dev,
+			struct virtchnl_queue_tc_mapping *q_tc_mapping,
+			uint16_t size);
+void iavf_tm_conf_init(struct rte_eth_dev *dev);
+void iavf_tm_conf_uninit(struct rte_eth_dev *dev);
+extern const struct rte_tm_ops iavf_tm_ops;
 #endif /* _IAVF_ETHDEV_H_ */
drivers/net/iavf/iavf_ethdev.c +34 -0
index 5290588b17..93f82773e2 100644
@@ -122,6 +122,7 @@ static int iavf_dev_flow_ops_get(struct rte_eth_dev *dev,
 static int iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
 			uint32_t mc_addrs_num);
+static int iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused, void *arg);
 
 static const struct rte_pci_id pci_id_iavf_map[] = {
 	{ RTE_PCI_DEVICE(IAVF_INTEL_VENDOR_ID, IAVF_DEV_ID_ADAPTIVE_VF) },
@@ -200,8 +201,21 @@ static const struct eth_dev_ops iavf_eth_dev_ops = {
 	.flow_ops_get               = iavf_dev_flow_ops_get,
 	.tx_done_cleanup	    = iavf_dev_tx_done_cleanup,
 	.get_monitor_addr           = iavf_get_monitor_addr,
+	.tm_ops_get                 = iavf_tm_ops_get,
 };
 
+static int
+iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+			void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &iavf_tm_ops;
+
+	return 0;
+}
+
 static int
 iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
@@ -806,6 +820,11 @@ iavf_dev_start(struct rte_eth_dev *dev)
 				      dev->data->nb_tx_queues);
 	num_queue_pairs = vf->num_queue_pairs;
 
+	if (iavf_get_qos_cap(adapter)) {
+		PMD_INIT_LOG(ERR, "Failed to get qos capability");
+		return -1;
+	}
+
 	if (iavf_init_queues(dev) != 0) {
 		PMD_DRV_LOG(ERR, "failed to do Queue init");
 		return -1;
@@ -891,6 +910,7 @@ iavf_dev_stop(struct rte_eth_dev *dev)
 
 	adapter->stopped = 1;
 	dev->data->dev_started = 0;
+	vf->tm_conf.committed = false;
 
 	return 0;
 }
@@ -2090,6 +2110,15 @@ iavf_init_vf(struct rte_eth_dev *dev)
 		PMD_INIT_LOG(ERR, "unable to allocate vf_res memory");
 		goto err_api;
 	}
+
+	bufsz = sizeof(struct virtchnl_qos_cap_list) +
+		IAVF_MAX_TRAFFIC_CLASS * sizeof(struct virtchnl_qos_cap_elem);
+	vf->qos_cap = rte_zmalloc("qos_cap", bufsz, 0);
+	if (!vf->qos_cap) {
+		PMD_INIT_LOG(ERR, "unable to allocate qos_cap memory");
+		goto err_api;
+	}
+
 	if (iavf_get_vf_resource(adapter) != 0) {
 		PMD_INIT_LOG(ERR, "iavf_get_vf_config failed");
 		goto err_alloc;
@@ -2131,6 +2160,7 @@ iavf_init_vf(struct rte_eth_dev *dev)
 	rte_free(vf->rss_key);
 	rte_free(vf->rss_lut);
 err_alloc:
+	rte_free(vf->qos_cap);
 	rte_free(vf->vf_res);
 	vf->vsi_res = NULL;
 err_api:
@@ -2299,6 +2329,8 @@ iavf_dev_init(struct rte_eth_dev *eth_dev)
 
 	iavf_default_rss_disable(adapter);
 
+	iavf_tm_conf_init(eth_dev);
+
 	return 0;
 }
 
@@ -2338,6 +2370,8 @@ iavf_dev_close(struct rte_eth_dev *dev)
 				     iavf_dev_interrupt_handler, dev);
 	iavf_disable_irq0(hw);
 
+	iavf_tm_conf_uninit(dev);
+
 	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
 		if (vf->rss_lut) {
 			rte_free(vf->rss_lut);
drivers/net/iavf/iavf_tm.c new file +724 -0

724 lines not on this page · raw message

drivers/net/iavf/iavf_vchnl.c +55 -1

68 lines not on this page · raw message

drivers/net/iavf/meson.build +1 -0
index 6f222a9e87..f2010a8337 100644
@@ -19,6 +19,7 @@ sources = files(
         'iavf_generic_flow.c',
         'iavf_fdir.c',
         'iavf_hash.c',
+        'iavf_tm.c',
 )
 
 if arch_subdir == 'x86'
Signature · 1 line
2.17.1
author
#33 lore raw

PATCH v5 6/7 net/iavf: check Tx packet with correct UP and queue

In reply to Ting Xu

Add check in the Tx packet preparation function, to guarantee that the packet with specific user priority is distributed to the correct Tx queue according to the configured Tx queue TC mapping.

---
 drivers/net/iavf/iavf.h      | 10 +++++++++
 drivers/net/iavf/iavf_rxtx.c | 43 ++++++++++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_tm.c   | 13 +++++++++++
 3 files changed, 66 insertions(+)
drivers/net/iavf/iavf.h +10 -0
index feb8337b55..b3bd078111 100644
@@ -86,6 +86,8 @@
 
 #define IAVF_BITS_PER_BYTE 8
 
+#define IAVF_VLAN_TAG_PCP_OFFSET 13
+
 struct iavf_adapter;
 struct iavf_rx_queue;
 struct iavf_tx_queue;
@@ -165,6 +167,13 @@ struct iavf_tm_conf {
 	bool committed;
 };
 
+/* Struct to store queue TC mapping. Queue is continuous in one TC */
+struct iavf_qtc_map {
+	uint8_t	tc;
+	uint16_t start_queue_id;
+	uint16_t queue_count;
+};
+
 /* Structure to store private data specific for VF instance. */
 struct iavf_info {
 	uint16_t num_queue_pairs;
@@ -213,6 +222,7 @@ struct iavf_info {
 	bool lv_enabled;
 
 	struct virtchnl_qos_cap_list *qos_cap;
+	struct iavf_qtc_map *qtc_map;
 	struct iavf_tm_conf tm_conf;
 };
 
drivers/net/iavf/iavf_rxtx.c +43 -0

63 lines not on this page · raw message

drivers/net/iavf/iavf_tm.c +13 -0
index 79b6bcb79d..e36c4a5528 100644
@@ -655,6 +655,7 @@ static int iavf_hierarchy_commit(struct rte_eth_dev *dev,
 	struct virtchnl_queue_tc_mapping *q_tc_mapping;
 	struct iavf_tm_node_list *queue_list = &vf->tm_conf.queue_list;
 	struct iavf_tm_node *tm_node;
+	struct iavf_qtc_map *qtc_map;
 	uint16_t size;
 	int index = 0, node_committed = 0;
 	int i, ret_val = IAVF_SUCCESS;
@@ -684,6 +685,7 @@ static int iavf_hierarchy_commit(struct rte_eth_dev *dev,
 	q_tc_mapping->vsi_id = vf->vsi.vsi_id;
 	q_tc_mapping->num_tc = vf->qos_cap->num_elem;
 	q_tc_mapping->num_queue_pairs = vf->num_queue_pairs;
+
 	TAILQ_FOREACH(tm_node, queue_list, node) {
 		if (tm_node->tc >= q_tc_mapping->num_tc) {
 			PMD_DRV_LOG(ERR, "TC%d is not enabled", tm_node->tc);
@@ -701,15 +703,26 @@ static int iavf_hierarchy_commit(struct rte_eth_dev *dev,
 		goto fail_clear;
 	}
 
+	/* store the queue TC mapping info */
+	qtc_map = rte_zmalloc("qtc_map",
+		sizeof(struct iavf_qtc_map) * q_tc_mapping->num_tc, 0);
+	if (!qtc_map)
+		return IAVF_ERR_NO_MEMORY;
+
 	for (i = 0; i < q_tc_mapping->num_tc; i++) {
 		q_tc_mapping->tc[i].req.start_queue_id = index;
 		index += q_tc_mapping->tc[i].req.queue_count;
+		qtc_map[i].tc = i;
+		qtc_map[i].start_queue_id =
+			q_tc_mapping->tc[i].req.start_queue_id;
+		qtc_map[i].queue_count = q_tc_mapping->tc[i].req.queue_count;
 	}
 
 	ret_val = iavf_set_q_tc_map(dev, q_tc_mapping, size);
 	if (ret_val)
 		goto fail_clear;
 
+	vf->qtc_map = qtc_map;
 	vf->tm_conf.committed = true;
 	return ret_val;
 
Signature · 1 line
2.17.1
author
#34 lore raw

PATCH v5 7/7 doc: release note for ETS-based Tx QoS

In reply to Ting Xu

Add 21.08 release note for ETS-based Tx QoS

---
 doc/guides/rel_notes/release_21_08.rst | 7 +++++++
 1 file changed, 7 insertions(+)
doc/guides/rel_notes/release_21_08.rst +7 -0
index a6ecfdf3ce..35aa76a270 100644
@@ -55,6 +55,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Intel iavf driver.**
+
+  * Added Tx QoS VF queue TC mapping.
+
+* **Updated Intel ice driver.**
+
+  * Added Tx QoS TC bandwidth configuration in DCF.
 
 Removed Items
 -------------
Signature · 1 line
2.17.1
Acked-by on the v5 series Applied
#35 lore raw
In reply to Ting Xu
Quoted 46 lines

-----Original Message-----
From: Xu, Ting <[email protected]>
Sent: Thursday, July 1, 2021 6:20 PM
To: [email protected]
Cc: Zhang, Qi Z <[email protected]>; Wu, Jingjing <[email protected]>; Xing, Beilei <[email protected]>; Yang, Qiming <[email protected]>; Xu, Ting <[email protected]>
Subject: [PATCH v5 0/7] Enable ETS-based Tx QoS for VF in DCF

This patch enables the ETS-based Tx QoS for IAVF. Kernel tool is used to configure ETS first. DCF is used to set bandwidth limit for VFs of each TC. IAVF is supported to query QoS capability and set queue TC mapping. Traffic Management API is utilized to configure the QoS hierarchy scheduler tree. The scheduler tree will be passed to hardware to enable all above functions.

Ting Xu (7):
common/iavf: support ETS-based QoS offload configuration net/ice/base: support DCF query port ETS adminq net/ice: support DCF link status event handling net/ice: support QoS config VF bandwidth in DCF net/iavf: query QoS cap and set queue TC mapping net/iavf: check Tx packet with correct UP and queue
doc: release note for ETS-based Tx QoS

 doc/guides/rel_notes/release_21_08.rst |   7 +
 drivers/common/iavf/iavf_type.h        |   2 +
 drivers/common/iavf/virtchnl.h         | 131 +++++
 drivers/net/iavf/iavf.h                |  56 ++
 drivers/net/iavf/iavf_ethdev.c         |  34 ++
 drivers/net/iavf/iavf_rxtx.c           |  43 ++
 drivers/net/iavf/iavf_tm.c             | 737 ++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c          |  56 +-
 drivers/net/iavf/meson.build           |   1 +
 drivers/net/ice/base/ice_dcb.c         |   3 +-
 drivers/net/ice/ice_dcf.c              |   9 +-
 drivers/net/ice/ice_dcf.h              |  54 ++
 drivers/net/ice/ice_dcf_ethdev.c       |  68 ++-
 drivers/net/ice/ice_dcf_ethdev.h       |   3 +
 drivers/net/ice/ice_dcf_parent.c       |  81 +++
 drivers/net/ice/ice_dcf_sched.c        | 759
+++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   3 +-
 17 files changed, 2040 insertions(+), 7 deletions(-)  create mode 100644
Quoted 5 lines

drivers/net/iavf/iavf_tm.c create mode 100644
drivers/net/ice/ice_dcf_sched.c

--
2.17.1

Applied to dpdk-next-net-intel.

Thanks
Qi

author
#36 lore raw

v5 cover Enable ETS-based Tx QoS for VF in DCF

In reply to Ting Xu

This patch enables the ETS-based Tx QoS for IAVF. Kernel tool is used to configure ETS first. DCF is used to set bandwidth limit for VFs of each TC. IAVF is supported to query QoS capability and set queue TC mapping. Traffic Management API is utilized to configure the QoS hierarchy scheduler tree. The scheduler tree will be passed to hardware to enable all above functions.

Ting Xu (7):
common/iavf: support ETS-based QoS offload configuration net/ice/base: support DCF query port ETS adminq net/ice: support DCF link status event handling net/ice: support QoS config VF bandwidth in DCF net/iavf: query QoS cap and set queue TC mapping net/iavf: check Tx packet with correct UP and queue
doc: release note for ETS-based Tx QoS

 doc/guides/rel_notes/release_21_08.rst |   7 +
 drivers/common/iavf/iavf_type.h        |   2 +
 drivers/common/iavf/virtchnl.h         | 131 +++++
 drivers/net/iavf/iavf.h                |  56 ++
 drivers/net/iavf/iavf_ethdev.c         |  37 ++
 drivers/net/iavf/iavf_rxtx.c           |  49 ++
 drivers/net/iavf/iavf_rxtx.h           |   1 +
 drivers/net/iavf/iavf_tm.c             | 743 ++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c          |  56 +-
 drivers/net/iavf/meson.build           |   1 +
 drivers/net/ice/base/ice_dcb.c         |   3 +-
 drivers/net/ice/ice_dcf.c              |   9 +-
 drivers/net/ice/ice_dcf.h              |  54 ++
 drivers/net/ice/ice_dcf_ethdev.c       |  68 ++-
 drivers/net/ice/ice_dcf_ethdev.h       |   3 +
 drivers/net/ice/ice_dcf_parent.c       |  83 +++
 drivers/net/ice/ice_dcf_sched.c        | 765 +++++++++++++++++++++++++
 drivers/net/ice/meson.build            |   3 +-
 18 files changed, 2064 insertions(+), 7 deletions(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
Signature · 1 line
2.17.1
author
#37 lore raw

PATCH v5 1/7 common/iavf: support ETS-based QoS offload configuration

This patch adds new virtchnl opcodes and structures for QoS configuration, which includes:
1. VIRTCHNL_VF_OFFLOAD_TC, to negotiate the capability supporting QoS configuration. If VF and PF both have this flag, then the ETS-based QoS offload function is supported.
2. VIRTCHNL_OP_DCF_CONFIG_BW, DCF is supposed to configure min and max bandwidth for each VF per enabled TCs. To make the VSI node bandwidth configuration work, DCF also needs to configure TC node bandwidth directly.
3. VIRTCHNL_OP_GET_QOS_CAPS, VF queries current QoS configuration, such as enabled TCs, arbiter type, up2tc and bandwidth of VSI node. The configuration is previously set by DCB and DCF, and now is the potential QoS capability of VF. VF can take it as reference to configure queue TC mapping.
4. VIRTCHNL_OP_CONFIG_TC_MAP, set VF queues to TC mapping for all Tx and Rx queues. Queues mapping to one TC should be continuous and all allocated queues should be mapped.

---
 drivers/common/iavf/iavf_type.h |   2 +
 drivers/common/iavf/virtchnl.h  | 131 ++++++++++++++++++++++++++++++++
 2 files changed, 133 insertions(+)
drivers/common/iavf/iavf_type.h +2 -0
index f3815d523b..73dfb47e70 100644
@@ -141,6 +141,8 @@ enum iavf_debug_mask {
 #define IAVF_PHY_LED_MODE_MASK			0xFFFF
 #define IAVF_PHY_LED_MODE_ORIG			0x80000000
 
+#define IAVF_MAX_TRAFFIC_CLASS	8
+
 /* Memory types */
 enum iavf_memset_type {
 	IAVF_NONDMA_MEM = 0,
drivers/common/iavf/virtchnl.h +131 -0

185 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#38 lore raw

PATCH v5 2/7 net/ice/base: support DCF query port ETS adminq

In reply to Ting Xu

In the adminq command query port ETS function, the root node teid is needed. However, for DCF, the root node is not initialized, which will cause error when we refer to the variable. In this patch, we will check whether the root node is available or not first.

---
 drivers/net/ice/base/ice_dcb.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ice/base/ice_dcb.c +2 -1
index c73fc095ff..9c9675f6ef 100644
@@ -1524,7 +1524,8 @@ ice_aq_query_port_ets(struct ice_port_info *pi,
 		return ICE_ERR_PARAM;
 	cmd = &desc.params.port_ets;
 	ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_query_port_ets);
-	cmd->port_teid = pi->root->info.node_teid;
+	if (pi->root)
+		cmd->port_teid = pi->root->info.node_teid;
 
 	status = ice_aq_send_cmd(pi->hw, &desc, buf, buf_size, cd);
 	return status;
Signature · 1 line
2.17.1
author
#39 lore raw

PATCH v5 3/7 net/ice: support DCF link status event handling

In reply to Ting Xu

When link status changes, DCF will receive virtchnl PF event message. Add support to handle this event, change link status and update link info.

---
 drivers/net/ice/ice_dcf.h        |  6 ++++
 drivers/net/ice/ice_dcf_ethdev.c | 54 ++++++++++++++++++++++++++++++--
 drivers/net/ice/ice_dcf_parent.c | 51 ++++++++++++++++++++++++++++++
 3 files changed, 108 insertions(+), 3 deletions(-)
drivers/net/ice/ice_dcf.h +6 -0
index 0cb90b5e9f..587093b909 100644
@@ -60,6 +60,10 @@ struct ice_dcf_hw {
 	uint16_t nb_msix;
 	uint16_t rxq_map[16];
 	struct virtchnl_eth_stats eth_stats_offset;
+
+	/* Link status */
+	bool link_up;
+	uint32_t link_speed;
 };
 
 int ice_dcf_execute_virtchnl_cmd(struct ice_dcf_hw *hw,
@@ -77,5 +81,7 @@ int ice_dcf_disable_queues(struct ice_dcf_hw *hw);
 int ice_dcf_query_stats(struct ice_dcf_hw *hw,
 			struct virtchnl_eth_stats *pstats);
 int ice_dcf_add_del_all_mac_addr(struct ice_dcf_hw *hw, bool add);
+int ice_dcf_link_update(struct rte_eth_dev *dev,
+		    __rte_unused int wait_to_complete);
 
 #endif /* _ICE_DCF_H_ */
drivers/net/ice/ice_dcf_ethdev.c +51 -3

62 lines not on this page · raw message

drivers/net/ice/ice_dcf_parent.c +51 -0

63 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#40 lore raw

PATCH v5 4/7 net/ice: support QoS config VF bandwidth in DCF

In reply to Ting Xu

This patch supports the ETS-based QoS configuration. It enables the DCF to configure bandwidth limits for each VF VSI of different TCs. A hierarchy scheduler tree is built with port, TC and VSI nodes.

---
 drivers/net/ice/ice_dcf.c        |   9 +-
 drivers/net/ice/ice_dcf.h        |  48 ++
 drivers/net/ice/ice_dcf_ethdev.c |  14 +
 drivers/net/ice/ice_dcf_ethdev.h |   3 +
 drivers/net/ice/ice_dcf_parent.c |  32 ++
 drivers/net/ice/ice_dcf_sched.c  | 765 +++++++++++++++++++++++++++++++
 drivers/net/ice/meson.build      |   3 +-
 7 files changed, 872 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ice/ice_dcf_sched.c
drivers/net/ice/ice_dcf.c +8 -1
index d72a6f357e..349d23ee4f 100644
@@ -235,7 +235,8 @@ ice_dcf_get_vf_resource(struct ice_dcf_hw *hw)
 	caps = VIRTCHNL_VF_OFFLOAD_WB_ON_ITR | VIRTCHNL_VF_OFFLOAD_RX_POLLING |
 	       VIRTCHNL_VF_CAP_ADV_LINK_SPEED | VIRTCHNL_VF_CAP_DCF |
 	       VIRTCHNL_VF_OFFLOAD_VLAN_V2 |
-	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC;
+	       VF_BASE_MODE_OFFLOADS | VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC |
+	       VIRTCHNL_VF_OFFLOAD_QOS;
 
 	err = ice_dcf_send_cmd_req_no_irq(hw, VIRTCHNL_OP_GET_VF_RESOURCES,
 					  (uint8_t *)&caps, sizeof(caps));
@@ -668,6 +669,9 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 		}
 	}
 
+	if (hw->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS)
+		ice_dcf_tm_conf_init(eth_dev);
+
 	hw->eth_dev = eth_dev;
 	rte_intr_callback_register(&pci_dev->intr_handle,
 				   ice_dcf_dev_interrupt_handler, hw);
@@ -703,6 +707,9 @@ ice_dcf_uninit_hw(struct rte_eth_dev *eth_dev, struct ice_dcf_hw *hw)
 	ice_dcf_mode_disable(hw);
 	iavf_shutdown_adminq(&hw->avf);
 
+	if (hw->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS)
+		ice_dcf_tm_conf_uninit(eth_dev);
+
 	rte_free(hw->arq_buf);
 	rte_free(hw->vf_vsi_map);
 	rte_free(hw->vf_res);
drivers/net/ice/ice_dcf.h +48 -0

71 lines not on this page · raw message

drivers/net/ice/ice_dcf_ethdev.c +14 -0
index 0b40ebbec6..69fe6e63d1 100644
@@ -622,6 +622,7 @@ ice_dcf_dev_stop(struct rte_eth_dev *dev)
 	ice_dcf_add_del_all_mac_addr(&dcf_ad->real_hw, false);
 	dev->data->dev_link.link_status = ETH_LINK_DOWN;
 	ad->pf.adapter_stopped = 1;
+	dcf_ad->real_hw.tm_conf.committed = false;
 
 	return 0;
 }
@@ -994,6 +995,18 @@ ice_dcf_dev_udp_tunnel_port_del(struct rte_eth_dev *dev,
 	return ret;
 }
 
+static int
+ice_dcf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+		void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &ice_dcf_tm_ops;
+
+	return 0;
+}
+
 static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.dev_start               = ice_dcf_dev_start,
 	.dev_stop                = ice_dcf_dev_stop,
@@ -1018,6 +1031,7 @@ static const struct eth_dev_ops ice_dcf_eth_dev_ops = {
 	.flow_ops_get            = ice_dcf_dev_flow_ops_get,
 	.udp_tunnel_port_add	 = ice_dcf_dev_udp_tunnel_port_add,
 	.udp_tunnel_port_del	 = ice_dcf_dev_udp_tunnel_port_del,
+	.tm_ops_get              = ice_dcf_tm_ops_get,
 };
 
 static int
drivers/net/ice/ice_dcf_ethdev.h +3 -0
index e7c9d7fe41..8510e37119 100644
@@ -7,6 +7,8 @@
 
 #include "base/ice_common.h"
 #include "base/ice_adminq_cmd.h"
+#include "base/ice_dcb.h"
+#include "base/ice_sched.h"
 
 #include "ice_ethdev.h"
 #include "ice_dcf.h"
@@ -52,6 +54,7 @@ struct ice_dcf_vf_repr {
 	struct ice_dcf_vlan outer_vlan_info; /* DCF always handle outer VLAN */
 };
 
+extern const struct rte_tm_ops ice_dcf_tm_ops;
 void ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 				 uint8_t *msg, uint16_t msglen);
 int ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev);
drivers/net/ice/ice_dcf_parent.c +32 -0
index 788f6dd2a0..c59cd0bef9 100644
@@ -264,6 +264,29 @@ ice_dcf_handle_pf_event_msg(struct ice_dcf_hw *dcf_hw,
 	}
 }
 
+static int
+ice_dcf_query_port_ets(struct ice_hw *parent_hw, struct ice_dcf_hw *real_hw)
+{
+	int ret;
+
+	real_hw->ets_config = (struct ice_aqc_port_ets_elem *)
+			ice_malloc(real_hw, sizeof(*real_hw->ets_config));
+	if (!real_hw->ets_config)
+		return ICE_ERR_NO_MEMORY;
+
+	ret = ice_aq_query_port_ets(parent_hw->port_info,
+			real_hw->ets_config, sizeof(*real_hw->ets_config),
+			NULL);
+	if (ret) {
+		PMD_DRV_LOG(ERR, "DCF Query Port ETS failed");
+		rte_free(real_hw->ets_config);
+		real_hw->ets_config = NULL;
+		return ret;
+	}
+
+	return ICE_SUCCESS;
+}
+
 static int
 ice_dcf_init_parent_hw(struct ice_hw *hw)
 {
@@ -486,6 +509,15 @@ ice_dcf_init_parent_adapter(struct rte_eth_dev *eth_dev)
 		return err;
 	}
 
+	if (hw->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS) {
+		err = ice_dcf_query_port_ets(parent_hw, hw);
+		if (err) {
+			PMD_INIT_LOG(ERR, "failed to query port ets with error %d",
+				     err);
+			goto uninit_hw;
+		}
+	}
+
 	err = ice_dcf_load_pkg(parent_hw);
 	if (err) {
 		PMD_INIT_LOG(ERR, "failed to load package with error %d",
drivers/net/ice/ice_dcf_sched.c new file +765 -0

765 lines not on this page · raw message

drivers/net/ice/meson.build +2 -1
index 65750d3501..0b86d74a49 100644
@@ -70,6 +70,7 @@ endif
 sources += files('ice_dcf.c',
          'ice_dcf_vf_representor.c',
          'ice_dcf_ethdev.c',
-         'ice_dcf_parent.c')
+         'ice_dcf_parent.c',
+	 'ice_dcf_sched.c')
 
 headers = files('rte_pmd_ice.h')
Signature · 1 line
2.17.1
author
#41 lore raw

PATCH v5 5/7 net/iavf: query QoS cap and set queue TC mapping

In reply to Ting Xu

This patch added the support for VF to config the ETS-based Tx QoS, including querying current QoS configuration from PF and config queue TC mapping. PF QoS is configured in advance and the queried info is provided to the user for future usage. VF queues are mapped to different TCs in PF through virtchnl.

---
 drivers/net/iavf/iavf.h        |  46 +++
 drivers/net/iavf/iavf_ethdev.c |  37 ++
 drivers/net/iavf/iavf_tm.c     | 730 +++++++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_vchnl.c  |  56 ++-
 drivers/net/iavf/meson.build   |   1 +
 5 files changed, 869 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/iavf/iavf_tm.c
drivers/net/iavf/iavf.h +46 -0
index 4f5811ae87..feb8337b55 100644
@@ -6,6 +6,8 @@
 #define _IAVF_ETHDEV_H_
 
 #include <rte_kvargs.h>
+#include <rte_tm_driver.h>
+
 #include <iavf_prototype.h>
 #include <iavf_adminq_cmd.h>
 #include <iavf_type.h>
@@ -82,6 +84,8 @@
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_MASK  0x03
 #define IAVF_RX_DESC_EXT_STATUS_FLEXBH_FD_ID 0x01
 
+#define IAVF_BITS_PER_BYTE 8
+
 struct iavf_adapter;
 struct iavf_rx_queue;
 struct iavf_tx_queue;
@@ -129,6 +133,38 @@ enum iavf_aq_result {
 	IAVF_MSG_CMD,      /* Read async command result */
 };
 
+/* Struct to store Traffic Manager node configuration. */
+struct iavf_tm_node {
+	TAILQ_ENTRY(iavf_tm_node) node;
+	uint32_t id;
+	uint32_t tc;
+	uint32_t priority;
+	uint32_t weight;
+	uint32_t reference_count;
+	struct iavf_tm_node *parent;
+	struct rte_tm_node_params params;
+};
+
+TAILQ_HEAD(iavf_tm_node_list, iavf_tm_node);
+
+/* node type of Traffic Manager */
+enum iavf_tm_node_type {
+	IAVF_TM_NODE_TYPE_PORT,
+	IAVF_TM_NODE_TYPE_TC,
+	IAVF_TM_NODE_TYPE_QUEUE,
+	IAVF_TM_NODE_TYPE_MAX,
+};
+
+/* Struct to store all the Traffic Manager configuration. */
+struct iavf_tm_conf {
+	struct iavf_tm_node *root; /* root node - vf vsi */
+	struct iavf_tm_node_list tc_list; /* node list for all the TCs */
+	struct iavf_tm_node_list queue_list; /* node list for all the queues */
+	uint32_t nb_tc_node;
+	uint32_t nb_queue_node;
+	bool committed;
+};
+
 /* Structure to store private data specific for VF instance. */
 struct iavf_info {
 	uint16_t num_queue_pairs;
@@ -175,6 +211,9 @@ struct iavf_info {
 	struct iavf_fdir_info fdir; /* flow director info */
 	/* indicate large VF support enabled or not */
 	bool lv_enabled;
+
+	struct virtchnl_qos_cap_list *qos_cap;
+	struct iavf_tm_conf tm_conf;
 };
 
 #define IAVF_MAX_PKT_TYPE 1024
@@ -344,4 +383,11 @@ int iavf_add_del_mc_addr_list(struct iavf_adapter *adapter,
 			uint32_t mc_addrs_num, bool add);
 int iavf_request_queues(struct iavf_adapter *adapter, uint16_t num);
 int iavf_get_max_rss_queue_region(struct iavf_adapter *adapter);
+int iavf_get_qos_cap(struct iavf_adapter *adapter);
+int iavf_set_q_tc_map(struct rte_eth_dev *dev,
+			struct virtchnl_queue_tc_mapping *q_tc_mapping,
+			uint16_t size);
+void iavf_tm_conf_init(struct rte_eth_dev *dev);
+void iavf_tm_conf_uninit(struct rte_eth_dev *dev);
+extern const struct rte_tm_ops iavf_tm_ops;
 #endif /* _IAVF_ETHDEV_H_ */
drivers/net/iavf/iavf_ethdev.c +37 -0
index 5290588b17..7272f69a06 100644
@@ -122,6 +122,7 @@ static int iavf_dev_flow_ops_get(struct rte_eth_dev *dev,
 static int iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
 			uint32_t mc_addrs_num);
+static int iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused, void *arg);
 
 static const struct rte_pci_id pci_id_iavf_map[] = {
 	{ RTE_PCI_DEVICE(IAVF_INTEL_VENDOR_ID, IAVF_DEV_ID_ADAPTIVE_VF) },
@@ -200,8 +201,21 @@ static const struct eth_dev_ops iavf_eth_dev_ops = {
 	.flow_ops_get               = iavf_dev_flow_ops_get,
 	.tx_done_cleanup	    = iavf_dev_tx_done_cleanup,
 	.get_monitor_addr           = iavf_get_monitor_addr,
+	.tm_ops_get                 = iavf_tm_ops_get,
 };
 
+static int
+iavf_tm_ops_get(struct rte_eth_dev *dev __rte_unused,
+			void *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	*(const void **)arg = &iavf_tm_ops;
+
+	return 0;
+}
+
 static int
 iavf_set_mc_addr_list(struct rte_eth_dev *dev,
 			struct rte_ether_addr *mc_addrs,
@@ -806,6 +820,12 @@ iavf_dev_start(struct rte_eth_dev *dev)
 				      dev->data->nb_tx_queues);
 	num_queue_pairs = vf->num_queue_pairs;
 
+	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS)
+		if (iavf_get_qos_cap(adapter)) {
+			PMD_INIT_LOG(ERR, "Failed to get qos capability");
+			return -1;
+		}
+
 	if (iavf_init_queues(dev) != 0) {
 		PMD_DRV_LOG(ERR, "failed to do Queue init");
 		return -1;
@@ -2090,6 +2110,7 @@ iavf_init_vf(struct rte_eth_dev *dev)
 		PMD_INIT_LOG(ERR, "unable to allocate vf_res memory");
 		goto err_api;
 	}
+
 	if (iavf_get_vf_resource(adapter) != 0) {
 		PMD_INIT_LOG(ERR, "iavf_get_vf_config failed");
 		goto err_alloc;
@@ -2124,6 +2145,18 @@ iavf_init_vf(struct rte_eth_dev *dev)
 		}
 	}
 
+	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS) {
+		bufsz = sizeof(struct virtchnl_qos_cap_list) +
+			IAVF_MAX_TRAFFIC_CLASS *
+			sizeof(struct virtchnl_qos_cap_elem);
+		vf->qos_cap = rte_zmalloc("qos_cap", bufsz, 0);
+		if (!vf->qos_cap) {
+			PMD_INIT_LOG(ERR, "unable to allocate qos_cap memory");
+			goto err_rss;
+		}
+		iavf_tm_conf_init(dev);
+	}
+
 	iavf_init_proto_xtr(dev);
 
 	return 0;
@@ -2131,6 +2164,7 @@ iavf_init_vf(struct rte_eth_dev *dev)
 	rte_free(vf->rss_key);
 	rte_free(vf->rss_lut);
 err_alloc:
+	rte_free(vf->qos_cap);
 	rte_free(vf->vf_res);
 	vf->vsi_res = NULL;
 err_api:
@@ -2338,6 +2372,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
 				     iavf_dev_interrupt_handler, dev);
 	iavf_disable_irq0(hw);
 
+	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_QOS)
+		iavf_tm_conf_uninit(dev);
+
 	if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
 		if (vf->rss_lut) {
 			rte_free(vf->rss_lut);
drivers/net/iavf/iavf_tm.c new file +730 -0

730 lines not on this page · raw message

drivers/net/iavf/iavf_vchnl.c +55 -1

68 lines not on this page · raw message

drivers/net/iavf/meson.build +1 -0
index 6f222a9e87..f2010a8337 100644
@@ -19,6 +19,7 @@ sources = files(
         'iavf_generic_flow.c',
         'iavf_fdir.c',
         'iavf_hash.c',
+        'iavf_tm.c',
 )
 
 if arch_subdir == 'x86'
Signature · 1 line
2.17.1
author
#42 lore raw

PATCH v5 6/7 net/iavf: check Tx packet with correct UP and queue

In reply to Ting Xu

Add check in the Tx packet preparation function, to guarantee that the packet with specific user priority is distributed to the correct Tx queue according to the configured Tx queue TC mapping.

---
 drivers/net/iavf/iavf.h      | 10 ++++++++
 drivers/net/iavf/iavf_rxtx.c | 49 ++++++++++++++++++++++++++++++++++++
 drivers/net/iavf/iavf_rxtx.h |  1 +
 drivers/net/iavf/iavf_tm.c   | 13 ++++++++++
 4 files changed, 73 insertions(+)
drivers/net/iavf/iavf.h +10 -0
index feb8337b55..b3bd078111 100644
@@ -86,6 +86,8 @@
 
 #define IAVF_BITS_PER_BYTE 8
 
+#define IAVF_VLAN_TAG_PCP_OFFSET 13
+
 struct iavf_adapter;
 struct iavf_rx_queue;
 struct iavf_tx_queue;
@@ -165,6 +167,13 @@ struct iavf_tm_conf {
 	bool committed;
 };
 
+/* Struct to store queue TC mapping. Queue is continuous in one TC */
+struct iavf_qtc_map {
+	uint8_t	tc;
+	uint16_t start_queue_id;
+	uint16_t queue_count;
+};
+
 /* Structure to store private data specific for VF instance. */
 struct iavf_info {
 	uint16_t num_queue_pairs;
@@ -213,6 +222,7 @@ struct iavf_info {
 	bool lv_enabled;
 
 	struct virtchnl_qos_cap_list *qos_cap;
+	struct iavf_qtc_map *qtc_map;
 	struct iavf_tm_conf tm_conf;
 };
 
drivers/net/iavf/iavf_rxtx.c +49 -0

73 lines not on this page · raw message

drivers/net/iavf/iavf_rxtx.h +1 -0
index 19b6028a15..e210b913d6 100644
@@ -252,6 +252,7 @@ struct iavf_tx_queue {
 #define IAVF_TX_FLAGS_VLAN_TAG_LOC_L2TAG1	BIT(0)
 #define IAVF_TX_FLAGS_VLAN_TAG_LOC_L2TAG2	BIT(1)
 	uint8_t vlan_flag;
+	uint8_t tc;
 };
 
 /* Offload features */
drivers/net/iavf/iavf_tm.c +13 -0
index 03349619d1..558021014e 100644
@@ -655,6 +655,7 @@ static int iavf_hierarchy_commit(struct rte_eth_dev *dev,
 	struct virtchnl_queue_tc_mapping *q_tc_mapping;
 	struct iavf_tm_node_list *queue_list = &vf->tm_conf.queue_list;
 	struct iavf_tm_node *tm_node;
+	struct iavf_qtc_map *qtc_map;
 	uint16_t size;
 	int index = 0, node_committed = 0;
 	int i, ret_val = IAVF_SUCCESS;
@@ -690,6 +691,7 @@ static int iavf_hierarchy_commit(struct rte_eth_dev *dev,
 	q_tc_mapping->vsi_id = vf->vsi.vsi_id;
 	q_tc_mapping->num_tc = vf->qos_cap->num_elem;
 	q_tc_mapping->num_queue_pairs = vf->num_queue_pairs;
+
 	TAILQ_FOREACH(tm_node, queue_list, node) {
 		if (tm_node->tc >= q_tc_mapping->num_tc) {
 			PMD_DRV_LOG(ERR, "TC%d is not enabled", tm_node->tc);
@@ -707,15 +709,26 @@ static int iavf_hierarchy_commit(struct rte_eth_dev *dev,
 		goto fail_clear;
 	}
 
+	/* store the queue TC mapping info */
+	qtc_map = rte_zmalloc("qtc_map",
+		sizeof(struct iavf_qtc_map) * q_tc_mapping->num_tc, 0);
+	if (!qtc_map)
+		return IAVF_ERR_NO_MEMORY;
+
 	for (i = 0; i < q_tc_mapping->num_tc; i++) {
 		q_tc_mapping->tc[i].req.start_queue_id = index;
 		index += q_tc_mapping->tc[i].req.queue_count;
+		qtc_map[i].tc = i;
+		qtc_map[i].start_queue_id =
+			q_tc_mapping->tc[i].req.start_queue_id;
+		qtc_map[i].queue_count = q_tc_mapping->tc[i].req.queue_count;
 	}
 
 	ret_val = iavf_set_q_tc_map(dev, q_tc_mapping, size);
 	if (ret_val)
 		goto fail_clear;
 
+	vf->qtc_map = qtc_map;
 	vf->tm_conf.committed = true;
 	return ret_val;
 
Signature · 1 line
2.17.1
author
#43 lore raw

PATCH v5 7/7 doc: release note for ETS-based Tx QoS

In reply to Ting Xu

Add 21.08 release note for ETS-based Tx QoS

---
 doc/guides/rel_notes/release_21_08.rst | 7 +++++++
 1 file changed, 7 insertions(+)
doc/guides/rel_notes/release_21_08.rst +7 -0
index a6ecfdf3ce..35aa76a270 100644
@@ -55,6 +55,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated Intel iavf driver.**
+
+  * Added Tx QoS VF queue TC mapping.
+
+* **Updated Intel ice driver.**
+
+  * Added Tx QoS TC bandwidth configuration in DCF.
 
 Removed Items
 -------------
Signature · 1 line
2.17.1