net/gve: RSS Support for GVE Driver
Subsystems (2)
cover net/gve: RSS Support for GVE Driver
This patch series introduces RSS support for the GVE poll-mode driver. This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set with a key before the redriection table is set up. When the RSS hash is configured, a default redirection table is generated based on the number of queues. When the device is re-configured, the redirection table is reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for RSS redirection table entries, while the RTE API uses 16 bit integers. However, this is unlikely to be an issue, as these values represent receive queues, and the gVNIC device does not support anywhere near 64K queues.
This series also updates the corresponding feature matrix ertries and documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches fixes.
Note: There is a warning about complex macros being parenthesized that does not seem to be well-founded.
Joshua Washington (7):
net/gve: fully expose RSS offload support in dev_info net/gve: RSS adminq command changes
net/gve: add gve_rss library for handling RSS-related behaviors net/gve: RSS configuration update support
net/gve: RSS redirection table update support net/gve: update gve.ini with RSS capabilities net/gve: update GVE documentation with RSS support
doc/guides/nics/features/gve.ini | 3 + doc/guides/nics/gve.rst | 16 ++- drivers/net/gve/base/gve.h | 15 ++ drivers/net/gve/base/gve_adminq.c | 59 ++++++++ drivers/net/gve/base/gve_adminq.h | 21 +++ drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 17 +++ drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++ drivers/net/gve/meson.build | 1 + 10 files changed, 667 insertions(+), 9 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v2 1/7 net/gve: fully expose RSS offload support in dev_info
This patch communicates that the GVE driver supports RSS, along with the RSS offloads supported by the driver.
--- drivers/net/gve/gve_ethdev.c | 4 +++- drivers/net/gve/gve_ethdev.h | 8 ++++++++ 2 files changed, 11 insertions(+), 1 deletion(-)
drivers/net/gve/gve_ethdev.c +3 -1
@@ -405,7 +405,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_mtu = priv->max_mtu; dev_info->min_mtu =RTE_ETHER_MIN_MTU ; - dev_info->rx_offload_capa =0 ; + dev_info->rx_offload_capa =RTE_ETH_RX_OFFLOAD_RSS_HASH ; dev_info->tx_offload_capa =RTE_ETH_TX_OFFLOAD_MULTI_SEGS | RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
@@ -442,6 +442,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .nb_align = 1, }; + dev_info->flow_type_rss_offloads =GVE_RSS_OFFLOAD_ALL ; +return 0 ; }
drivers/net/gve/gve_ethdev.h +8 -0
@@ -33,6 +33,14 @@ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG) +#define GVE_RSS_OFFLOAD_ALL ( \ + RTE_ETH_RSS_IPV4 | \ + RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ + RTE_ETH_RSS_IPV6 | \ + RTE_ETH_RSS_IPV6_EX | \ + RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ + RTE_ETH_RSS_IPV6_TCP_EX) +/* A list of pages registered with the device during setup and used by a queue * as buffers */
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v2 2/7 net/gve: RSS adminq command changes
In reply to Joshua WashingtonThis change introduces admin queue changes that enable the configuration of RSS parameters for the GVE driver.
--- drivers/net/gve/base/gve.h | 15 ++++++++ drivers/net/gve/base/gve_adminq.c | 59 +++++++++++++++++++++++++++++++ drivers/net/gve/base/gve_adminq.h | 29 +++++++++++++++ drivers/net/gve/gve_ethdev.h | 6 +++- 4 files changed, 108 insertions(+), 1 deletion(-)
drivers/net/gve/base/gve.h +15 -0
@@ -51,4 +51,19 @@ enum gve_state_flags_bit { GVE_PRIV_FLAGS_NAPI_ENABLED = 4, }; +enum gve_rss_hash_algorithm { +GVE_RSS_HASH_UNDEFINED =0 , +GVE_RSS_HASH_TOEPLITZ =1 , +}; + +struct gve_rss_config { +uint16_t hash_types; +enum gve_rss_hash_algorithm alg; +uint16_t key_size; +uint16_t indir_size; +uint8_t *key; +uint32_t *indir; +}; + +#endif /* _GVE_H_ */
drivers/net/gve/base/gve_adminq.c +59 -0
68 lines not on this page · raw message
drivers/net/gve/base/gve_adminq.h +29 -0
@@ -19,6 +19,7 @@ enum gve_adminq_opcodes {GVE_ADMINQ_DESTROY_TX_QUEUE =0x7 ,GVE_ADMINQ_DESTROY_RX_QUEUE =0x8 ,GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES =0x9 , +GVE_ADMINQ_CONFIGURE_RSS =0xA ,GVE_ADMINQ_SET_DRIVER_PARAMETER =0xB ,GVE_ADMINQ_REPORT_STATS =0xC , GVE_ADMINQ_REPORT_LINK_SPEED = 0xD,
@@ -377,6 +378,27 @@ struct gve_adminq_get_ptype_map {__be64 ptype_map_addr; }; +#define GVE_RSS_HASH_IPV4 BIT(0) +#define GVE_RSS_HASH_TCPV4 BIT(1) +#define GVE_RSS_HASH_IPV6 BIT(2) +#define GVE_RSS_HASH_IPV6_EX BIT(3) +#define GVE_RSS_HASH_TCPV6 BIT(4) +#define GVE_RSS_HASH_TCPV6_EX BIT(5) +#define GVE_RSS_HASH_UDPV4 BIT(6) +#define GVE_RSS_HASH_UDPV6 BIT(7) +#define GVE_RSS_HASH_UDPV6_EX BIT(8) + +/* RSS configuration command */ +struct gve_adminq_configure_rss { +__be16 hash_types; +u8 halg;/* hash algorithm */ +u8 reserved; +__be16 hkey_len; +__be16 indir_len; +__be64 hkey_addr; +__be64 indir_addr; +}; + union gve_adminq_command { struct { __be32 opcode;
@@ -391,6 +413,7 @@ union gve_adminq_command {struct gve_adminq_describe_device describe_device;struct gve_adminq_register_page_list reg_page_list;struct gve_adminq_unregister_page_list unreg_page_list; +struct gve_adminq_configure_rss configure_rss;struct gve_adminq_set_driver_parameter set_driver_param;struct gve_adminq_report_stats report_stats;struct gve_adminq_report_link_speed report_link_speed;
@@ -404,6 +427,8 @@ union gve_adminq_command {GVE_CHECK_UNION_LEN (64 , gve_adminq_command); +struct gve_priv ; +struct gve_queue_page_list ;int gve_adminq_alloc (struct gve_priv *priv);void gve_adminq_free (struct gve_priv *priv);void gve_adminq_release (struct gve_priv *priv);
@@ -433,4 +458,8 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,int gve_adminq_verify_driver_compatibility (struct gve_priv *priv,u64 driver_info_len,dma_addr_t driver_info_addr); + +int gve_adminq_configure_rss (struct gve_priv *priv, +struct gve_rss_config *rss_config); +#endif /* _GVE_ADMINQ_H */
drivers/net/gve/gve_ethdev.h +5 -1
@@ -39,7 +39,10 @@RTE_ETH_RSS_IPV6 | \RTE_ETH_RSS_IPV6_EX | \RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ - RTE_ETH_RSS_IPV6_TCP_EX) +RTE_ETH_RSS_IPV6_TCP_EX | \ +RTE_ETH_RSS_NONFRAG_IPV4_UDP | \ +RTE_ETH_RSS_NONFRAG_IPV6_UDP | \ + RTE_ETH_RSS_IPV6_UDP_EX)/* A list of pages registered with the device during setup and used by a queue * as buffers
@@ -264,6 +267,7 @@ struct gve_priv {uint32_t adminq_destroy_tx_queue_cnt;uint32_t adminq_destroy_rx_queue_cnt;uint32_t adminq_dcfg_device_resources_cnt; +uint32_t adminq_cfg_rss_cnt;uint32_t adminq_set_driver_parameter_cnt;uint32_t adminq_report_stats_cnt;uint32_t adminq_report_link_speed_cnt;
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v2 3/7 net/gve: add gve_rss library for handling RSS-related behaviors
In reply to Joshua WashingtonThis change includes a number of helper functions to facilitate RSS configuration on the GVE DPDK driver. These methods are declared in gve_rss.h.
--- drivers/net/gve/base/gve_adminq.h | 10 +- drivers/net/gve/gve_ethdev.c | 2 +- drivers/net/gve/gve_ethdev.h | 4 +- drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++++ drivers/net/gve/meson.build | 1 + 6 files changed, 319 insertions(+), 11 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
drivers/net/gve/base/gve_adminq.h +1 -9
@@ -378,15 +378,6 @@ struct gve_adminq_get_ptype_map {__be64 ptype_map_addr; }; -#define GVE_RSS_HASH_IPV4 BIT(0) -#define GVE_RSS_HASH_TCPV4 BIT(1) -#define GVE_RSS_HASH_IPV6 BIT(2) -#define GVE_RSS_HASH_IPV6_EX BIT(3) -#define GVE_RSS_HASH_TCPV6 BIT(4) -#define GVE_RSS_HASH_TCPV6_EX BIT(5) -#define GVE_RSS_HASH_UDPV4 BIT(6) -#define GVE_RSS_HASH_UDPV6 BIT(7) -#define GVE_RSS_HASH_UDPV6_EX BIT(8)/* RSS configuration command */ struct gve_adminq_configure_rss {
@@ -428,6 +419,7 @@ union gve_adminq_command {GVE_CHECK_UNION_LEN (64 , gve_adminq_command);struct gve_priv ; +struct gve_rss_config ;struct gve_queue_page_list ;int gve_adminq_alloc (struct gve_priv *priv);void gve_adminq_free (struct gve_priv *priv);
drivers/net/gve/gve_ethdev.c +1 -1
@@ -442,7 +442,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .nb_align = 1, }; - dev_info->flow_type_rss_offloads =GVE_RSS_OFFLOAD_ALL ; + dev_info->flow_type_rss_offloads =GVE_RTE_RSS_OFFLOAD_ALL ;return 0 ; }
drivers/net/gve/gve_ethdev.h +3 -1
@@ -33,7 +33,7 @@ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG) -#define GVE_RSS_OFFLOAD_ALL ( \ +#define GVE_RTE_RSS_OFFLOAD_ALL ( \ RTE_ETH_RSS_IPV4 | \ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ RTE_ETH_RSS_IPV6 | \
@@ -290,6 +290,8 @@ struct gve_priv {const struct rte_memzone *stats_report_mem;uint16_t stats_start_idx;/* start index of array of stats written by NIC */ uint16_t stats_end_idx;/* end index of array of stats written by NIC */ + +struct gve_rss_config rss_config; }; static inline bool
drivers/net/gve/gve_rss.c new file +206 -0
206 lines not on this page · raw message
drivers/net/gve/gve_rss.h new file +107 -0
107 lines not on this page · raw message
drivers/net/gve/meson.build +1 -0
@@ -15,5 +15,6 @@ sources = files( 'gve_tx_dqo.c', 'gve_ethdev.c', 'gve_version.c', + 'gve_rss.c', ) includes += include_directories('base')
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v2 4/7 net/gve: RSS configuration update support
In reply to Joshua WashingtonThis patch adds support for updating the RSS hash key and hash fields in the GVE PMD through the implementation of rss_hash_update and rss_hash_conf_get.
The RSS hash key for gVNIC is required to be 40 bytes. On initial configuration of the RSS hash key, the RSS redirection table will be set to a static default, using a round-robin approach for all queues. Note, however, that this patch does not include support for setting the redirection table explicitly. In dev_configure, if the static redirection table has been set, it will be updated to reflect the new queue count, if it has changed.
The RSS key must be set before any other RSS configuration can happen. As such, an attempt to set the hash types before the key is configured will fail.
--- drivers/net/gve/gve_ethdev.c | 132 ++++++++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 3 + 2 files changed, 133 insertions(+), 2 deletions(-)
drivers/net/gve/gve_ethdev.c +130 -2
@@ -1,5 +1,6 @@/* SPDX-License-Identifier: BSD-3-Clause -* Copyright(C) 2022 Intel Corporation +* Copyright(C) 2022-2023 Intel Corporation +* Copyright(C) 2023 Google LLC */ #include "gve_ethdev.h"
@@ -8,6 +9,7 @@#include "base/gve_osdep.h" #include "gve_version.h" #include "rte_ether.h" +#include "gve_rss.h" static void gve_write_version (uint8_t *driver_version_register)
@@ -88,12 +90,31 @@ gve_dev_configure(struct rte_eth_dev *dev) {struct gve_priv *priv = dev->data->dev_private; -if (dev->data->dev_conf.rxmode.mq_mode &RTE_ETH_MQ_RX_RSS_FLAG ) +if (dev->data->dev_conf.rxmode.mq_mode &RTE_ETH_MQ_RX_RSS_FLAG ) { dev->data->dev_conf.rxmode.offloads |=RTE_ETH_RX_OFFLOAD_RSS_HASH ; + priv->rss_config.alg =GVE_RSS_HASH_TOEPLITZ ; + }if (dev->data->dev_conf.rxmode.offloads &RTE_ETH_RX_OFFLOAD_TCP_LRO ) priv->enable_rsc =1 ; +/* Reset RSS RETA in case number of queues changed. */ +if (priv->rss_config.indir) { +struct gve_rss_config update_reta_config; +gve_init_rss_config_from_priv (priv, &update_reta_config); +gve_generate_rss_reta (dev, &update_reta_config); + +int err =gve_adminq_configure_rss (priv, &update_reta_config); +if (err) +PMD_DRV_LOG (ERR , +"Could not reconfigure RSS redirection table." ); +else +gve_update_priv_rss_config (priv, &update_reta_config); + +gve_free_rss_config (&update_reta_config); +return err; + } +return 0 ; }
@@ -443,6 +464,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) }; dev_info->flow_type_rss_offloads =GVE_RTE_RSS_OFFLOAD_ALL ; + dev_info->hash_key_size =GVE_RSS_HASH_KEY_SIZE ; + dev_info->reta_size =GVE_RSS_INDIR_SIZE ;return 0 ; }
@@ -646,6 +669,107 @@ gve_xstats_get_names(struct rte_eth_dev *dev,return count; } + +static int +gve_rss_hash_update (struct rte_eth_dev *dev, +struct rte_eth_rss_conf *rss_conf) +{ +struct gve_priv *priv = dev->data->dev_private; +struct gve_rss_config gve_rss_conf; +int rss_reta_size; +int err; + +if (gve_validate_rss_hf (rss_conf->rss_hf)) { +PMD_DRV_LOG (ERR ,"Unsupported hash function." ); +return -EINVAL ; + } + +if (rss_conf->algorithm !=RTE_ETH_HASH_FUNCTION_TOEPLITZ && + rss_conf->algorithm !=RTE_ETH_HASH_FUNCTION_DEFAULT ) { +PMD_DRV_LOG (ERR ,"Device only supports Toeplitz algorithm." ); +return -EINVAL ; + } + +if (rss_conf->rss_key_len) { +if (rss_conf->rss_key_len !=GVE_RSS_HASH_KEY_SIZE ) { +PMD_DRV_LOG (ERR , +"Invalid hash key size. Only RSS hash key size " +"of %u supported" ,GVE_RSS_HASH_KEY_SIZE ); +return -EINVAL ; + } + +if (!rss_conf->rss_key) { +PMD_DRV_LOG (ERR ,"RSS key must be non-null." ); +return -EINVAL ; + } + }else { +if (!priv->rss_config.key_size) { +PMD_DRV_LOG (ERR ,"RSS key must be initialized before " +"any other configuration." ); +return -EINVAL ; + } + rss_conf->rss_key_len = priv->rss_config.key_size; + } + + rss_reta_size = priv->rss_config.indir ? + priv->rss_config.indir_size : +GVE_RSS_INDIR_SIZE ; + err =gve_init_rss_config (&gve_rss_conf, rss_conf->rss_key_len, + rss_reta_size); +if (err) +return err; + + gve_rss_conf.alg =GVE_RSS_HASH_TOEPLITZ ; + err =gve_update_rss_hash_types (priv, &gve_rss_conf, rss_conf); +if (err) + goto err; + err =gve_update_rss_key (priv, &gve_rss_conf, rss_conf); +if (err) + goto err; + +/* Set redirection table to default or preexisting. */ +if (!priv->rss_config.indir) +gve_generate_rss_reta (dev, &gve_rss_conf); +else +memcpy (gve_rss_conf.indir, priv->rss_config.indir, + gve_rss_conf.indir_size *sizeof (*priv->rss_config.indir)); + + err =gve_adminq_configure_rss (priv, &gve_rss_conf); +if (!err) +gve_update_priv_rss_config (priv, &gve_rss_conf); + +err: +gve_free_rss_config (&gve_rss_conf); +return err; +} + +static int +gve_rss_hash_conf_get (struct rte_eth_dev *dev, +struct rte_eth_rss_conf *rss_conf) +{ +struct gve_priv *priv = dev->data->dev_private; + +if (!(dev->data->dev_conf.rxmode.offloads & +RTE_ETH_RX_OFFLOAD_RSS_HASH )) { +PMD_DRV_LOG (ERR ,"RSS not configured." ); +return -ENOTSUP ; + } + + +gve_to_rte_rss_hf (priv->rss_config.hash_types, rss_conf); + rss_conf->rss_key_len = priv->rss_config.key_size; +if (rss_conf->rss_key) { +if (!priv->rss_config.key) { +PMD_DRV_LOG (ERR ,"Unable to retrieve default RSS hash key." ); +return -ENOTSUP ; + } +memcpy (rss_conf->rss_key, priv->rss_config.key, + rss_conf->rss_key_len *sizeof (*rss_conf->rss_key)); + } + +return 0 ; +} +static const struct eth_dev_ops gve_eth_dev_ops = { .dev_configure = gve_dev_configure, .dev_start = gve_dev_start,
@@ -666,6 +790,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .mtu_set = gve_dev_mtu_set, .xstats_get = gve_xstats_get, .xstats_get_names = gve_xstats_get_names, + .rss_hash_update = gve_rss_hash_update, + .rss_hash_conf_get = gve_rss_hash_conf_get, };static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -688,6 +814,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = { .mtu_set = gve_dev_mtu_set, .xstats_get = gve_xstats_get, .xstats_get_names = gve_xstats_get_names, + .rss_hash_update = gve_rss_hash_update, + .rss_hash_conf_get = gve_rss_hash_conf_get, }; static void
drivers/net/gve/gve_ethdev.h +3 -0
@@ -29,6 +29,9 @@#define GVE_RX_MIN_BUF_SIZE_GQI 2048#define GVE_RX_MAX_BUF_SIZE_GQI 4096 +#define GVE_RSS_HASH_KEY_SIZE 40 +#define GVE_RSS_INDIR_SIZE 128 +#define GVE_TX_CKSUM_OFFLOAD_MASK ( \ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG)
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v2 5/7 net/gve: RSS redirection table update support
In reply to Joshua WashingtonThis patch introduces support for updating the RSS redirection table in the GVE PMD through the implementation of rss_reta_update and rss_reta_query.
Due to an infrastructure limitation, the RSS hash key must be manually configured before the redirection table can be updated or queried. The redirection table is expected to be exactly 128 bytes.
--- drivers/net/gve/gve_ethdev.c | 95 ++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+)
drivers/net/gve/gve_ethdev.c +95 -0
@@ -770,6 +770,97 @@ gve_rss_hash_conf_get(struct rte_eth_dev *dev,return 0 ; } +static int +gve_rss_reta_update (struct rte_eth_dev *dev, +struct rte_eth_rss_reta_entry64 *reta_conf,uint16_t reta_size) +{ +struct gve_priv *priv = dev->data->dev_private; +struct gve_rss_config gve_rss_conf; +int table_id; +int err; +int i; + +/* RSS key must be set before the redirection table can be set. */ +if (!priv->rss_config.key || priv->rss_config.key_size ==0 ) { +PMD_DRV_LOG (ERR ,"RSS hash key msut be set before the " +"redirection table can be updated." ); +return -ENOTSUP ; + } + +if (reta_size !=GVE_RSS_INDIR_SIZE ) { +PMD_DRV_LOG (ERR ,"Redirection table must have %hu elements" , +GVE_RSS_INDIR_SIZE ); +return -EINVAL ; + } + + err =gve_init_rss_config_from_priv (priv, &gve_rss_conf); +if (err) { +PMD_DRV_LOG (ERR ,"Error allocating new RSS config." ); +return err; + } + + table_id =0 ; +for (i =0 ; i < priv->rss_config.indir_size; i++) { +int table_entry = i %RTE_ETH_RETA_GROUP_SIZE ; +if (reta_conf[table_id].mask & (1ULL << table_entry)) + gve_rss_conf.indir[i] = + reta_conf[table_id].reta[table_entry]; + +if (table_entry ==RTE_ETH_RETA_GROUP_SIZE -1 ) + table_id++; + } + + err =gve_adminq_configure_rss (priv, &gve_rss_conf); +if (err) +PMD_DRV_LOG (ERR ,"Problem configuring RSS with device." ); +else +gve_update_priv_rss_config (priv, &gve_rss_conf); + +gve_free_rss_config (&gve_rss_conf); +return err; +} + +static int +gve_rss_reta_query (struct rte_eth_dev *dev, +struct rte_eth_rss_reta_entry64 *reta_conf,uint16_t reta_size) +{ +struct gve_priv *priv = dev->data->dev_private; +int table_id; +int i; + +if (!(dev->data->dev_conf.rxmode.offloads & +RTE_ETH_RX_OFFLOAD_RSS_HASH )) { +PMD_DRV_LOG (ERR ,"RSS not configured." ); +return -ENOTSUP ; + } + +/* RSS key must be set before the redirection table can be queried. */ +if (!priv->rss_config.key) { +PMD_DRV_LOG (ERR ,"RSS hash key must be set before the " +"redirection table can be initialized." ); +return -ENOTSUP ; + } + +if (reta_size != priv->rss_config.indir_size) { +PMD_DRV_LOG (ERR ,"RSS redirection table must have %d entries." , + priv->rss_config.indir_size); +return -EINVAL ; + } + + table_id =0 ; +for (i =0 ; i < priv->rss_config.indir_size; i++) { +int table_entry = i %RTE_ETH_RETA_GROUP_SIZE ; +if (reta_conf[table_id].mask & (1ULL << table_entry)) + reta_conf[table_id].reta[table_entry] = + priv->rss_config.indir[i]; + +if (table_entry ==RTE_ETH_RETA_GROUP_SIZE -1 ) + table_id++; + } + +return 0 ; +} +static const struct eth_dev_ops gve_eth_dev_ops = { .dev_configure = gve_dev_configure, .dev_start = gve_dev_start,
@@ -792,6 +883,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .xstats_get_names = gve_xstats_get_names, .rss_hash_update = gve_rss_hash_update, .rss_hash_conf_get = gve_rss_hash_conf_get, + .reta_update = gve_rss_reta_update, + .reta_query = gve_rss_reta_query, };static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -816,6 +909,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = { .xstats_get_names = gve_xstats_get_names, .rss_hash_update = gve_rss_hash_update, .rss_hash_conf_get = gve_rss_hash_conf_get, + .reta_update = gve_rss_reta_update, + .reta_query = gve_rss_reta_query, }; static void
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v2 6/7 net/gve: update gve.ini with RSS capabilities
In reply to Joshua WashingtonThis patch updates the DPDK feature matrix to expose that the GVE driver supports RSS hash, RSS key update, and RSS reta update.
--- doc/guides/nics/features/gve.ini | 3 +++ 1 file changed, 3 insertions(+)
doc/guides/nics/features/gve.ini +3 -0
@@ -15,3 +15,6 @@ Linux = Y x86-32 = Y x86-64 = Y Usage doc = Y +RSS hash = Y +RSS key update = Y +RSS reta update = Y
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v2 7/7 net/gve: update GVE documentation with RSS support
In reply to Joshua WashingtonThis patch updates the GVE doc page to communicate that GVE now supports RSS configuration and explains the limitations.
--- doc/guides/nics/gve.rst | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
doc/guides/nics/gve.rst +10 -6
@@ -70,6 +70,8 @@ Supported features of the GVE PMD are: - Link state information - Tx multi-segments (Scatter Tx) - Tx UDP/TCP/SCTP Checksum +- RSS hash configuration +- RSS redirection table update and query Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD. Jumbo Frame is not supported in PMD for now.
@@ -77,10 +79,12 @@ It'll be added in a future DPDK release. Also, only GQI_QPL queue format is in use on GCP since GQI_RDA hasn't been released in production. -Currently, setting MTU with value larger than 1460 is not supported. +RSS +^^^ -Currently, only "RSS hash" is force enabled -so that the backend hardware device calculated hash values -could be shared with applications. -But for RSS, there is no such API to config RSS hash function or RETA table. -So, limited RSS is supported only with default config/setting. +GVE RSS can be enabled and configured using the standard interfaces. The driver +does not support querying the initial RSS configuration. + +The RSS hash key must be exactly 40 bytes, and the redirection table must have +128 entries. The RSS hash key must be configured before the redirection table +can be set up.
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
cover net/gve: RSS Support for GVE Driver
In reply to Joshua WashingtonThis patch series introduces RSS support for the GVE poll-mode driver. This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set with a key before the redriection table is set up. When the RSS hash is configured, a default redirection table is generated based on the number of queues. When the device is re-configured, the redirection table is reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for RSS redirection table entries, while the RTE API uses 16 bit integers. However, this is unlikely to be an issue, as these values represent receive queues, and the gVNIC device does not support anywhere near 64K queues.
This series also updates the corresponding feature matrix ertries and documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches fixes.
Note: There is a warning about complex macros being parenthesized that does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.
Joshua Washington (7):
net/gve: fully expose RSS offload support in dev_info net/gve: RSS adminq command changes
net/gve: add gve_rss library for handling RSS-related behaviors net/gve: RSS configuration update support
net/gve: RSS redirection table update support net/gve: update gve.ini with RSS capabilities net/gve: update GVE documentation with RSS support
doc/guides/nics/features/gve.ini | 3 + doc/guides/nics/gve.rst | 16 ++- drivers/net/gve/base/gve.h | 15 ++ drivers/net/gve/base/gve_adminq.c | 59 ++++++++ drivers/net/gve/base/gve_adminq.h | 21 +++ drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 17 +++ drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++ drivers/net/gve/meson.build | 1 + 10 files changed, 667 insertions(+), 9 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 1/7 net/gve: fully expose RSS offload support in dev_info
This patch communicates that the GVE driver supports RSS, along with the RSS offloads supported by the driver.
--- drivers/net/gve/gve_ethdev.c | 4 +++- drivers/net/gve/gve_ethdev.h | 8 ++++++++ 2 files changed, 11 insertions(+), 1 deletion(-)
drivers/net/gve/gve_ethdev.c +3 -1
@@ -405,7 +405,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_mtu = priv->max_mtu; dev_info->min_mtu =RTE_ETHER_MIN_MTU ; - dev_info->rx_offload_capa =0 ; + dev_info->rx_offload_capa =RTE_ETH_RX_OFFLOAD_RSS_HASH ; dev_info->tx_offload_capa =RTE_ETH_TX_OFFLOAD_MULTI_SEGS | RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
@@ -442,6 +442,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .nb_align = 1, }; + dev_info->flow_type_rss_offloads =GVE_RSS_OFFLOAD_ALL ; +return 0 ; }
drivers/net/gve/gve_ethdev.h +8 -0
@@ -33,6 +33,14 @@ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG) +#define GVE_RSS_OFFLOAD_ALL ( \ + RTE_ETH_RSS_IPV4 | \ + RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ + RTE_ETH_RSS_IPV6 | \ + RTE_ETH_RSS_IPV6_EX | \ + RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ + RTE_ETH_RSS_IPV6_TCP_EX) +/* A list of pages registered with the device during setup and used by a queue * as buffers */
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 2/7 net/gve: RSS adminq command changes
In reply to Joshua WashingtonThis change introduces admin queue changes that enable the configuration of RSS parameters for the GVE driver.
--- drivers/net/gve/base/gve.h | 15 ++++++++ drivers/net/gve/base/gve_adminq.c | 58 +++++++++++++++++++++++++++++++ drivers/net/gve/base/gve_adminq.h | 29 ++++++++++++++++ drivers/net/gve/gve_ethdev.h | 6 +++- 4 files changed, 107 insertions(+), 1 deletion(-)
drivers/net/gve/base/gve.h +15 -0
@@ -51,4 +51,19 @@ enum gve_state_flags_bit { GVE_PRIV_FLAGS_NAPI_ENABLED = 4, }; +enum gve_rss_hash_algorithm { +GVE_RSS_HASH_UNDEFINED =0 , +GVE_RSS_HASH_TOEPLITZ =1 , +}; + +struct gve_rss_config { +uint16_t hash_types; +enum gve_rss_hash_algorithm alg; +uint16_t key_size; +uint16_t indir_size; +uint8_t *key; +uint32_t *indir; +}; + +#endif /* _GVE_H_ */
drivers/net/gve/base/gve_adminq.c +58 -0
67 lines not on this page · raw message
drivers/net/gve/base/gve_adminq.h +29 -0
@@ -19,6 +19,7 @@ enum gve_adminq_opcodes {GVE_ADMINQ_DESTROY_TX_QUEUE =0x7 ,GVE_ADMINQ_DESTROY_RX_QUEUE =0x8 ,GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES =0x9 , +GVE_ADMINQ_CONFIGURE_RSS =0xA ,GVE_ADMINQ_SET_DRIVER_PARAMETER =0xB ,GVE_ADMINQ_REPORT_STATS =0xC , GVE_ADMINQ_REPORT_LINK_SPEED = 0xD,
@@ -377,6 +378,27 @@ struct gve_adminq_get_ptype_map {__be64 ptype_map_addr; }; +#define GVE_RSS_HASH_IPV4 BIT(0) +#define GVE_RSS_HASH_TCPV4 BIT(1) +#define GVE_RSS_HASH_IPV6 BIT(2) +#define GVE_RSS_HASH_IPV6_EX BIT(3) +#define GVE_RSS_HASH_TCPV6 BIT(4) +#define GVE_RSS_HASH_TCPV6_EX BIT(5) +#define GVE_RSS_HASH_UDPV4 BIT(6) +#define GVE_RSS_HASH_UDPV6 BIT(7) +#define GVE_RSS_HASH_UDPV6_EX BIT(8) + +/* RSS configuration command */ +struct gve_adminq_configure_rss { +__be16 hash_types; +u8 halg;/* hash algorithm */ +u8 reserved; +__be16 hkey_len; +__be16 indir_len; +__be64 hkey_addr; +__be64 indir_addr; +}; + union gve_adminq_command { struct { __be32 opcode;
@@ -391,6 +413,7 @@ union gve_adminq_command {struct gve_adminq_describe_device describe_device;struct gve_adminq_register_page_list reg_page_list;struct gve_adminq_unregister_page_list unreg_page_list; +struct gve_adminq_configure_rss configure_rss;struct gve_adminq_set_driver_parameter set_driver_param;struct gve_adminq_report_stats report_stats;struct gve_adminq_report_link_speed report_link_speed;
@@ -404,6 +427,8 @@ union gve_adminq_command {GVE_CHECK_UNION_LEN (64 , gve_adminq_command); +struct gve_priv ; +struct gve_queue_page_list ;int gve_adminq_alloc (struct gve_priv *priv);void gve_adminq_free (struct gve_priv *priv);void gve_adminq_release (struct gve_priv *priv);
@@ -433,4 +458,8 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,int gve_adminq_verify_driver_compatibility (struct gve_priv *priv,u64 driver_info_len,dma_addr_t driver_info_addr); + +int gve_adminq_configure_rss (struct gve_priv *priv, +struct gve_rss_config *rss_config); +#endif /* _GVE_ADMINQ_H */
drivers/net/gve/gve_ethdev.h +5 -1
@@ -39,7 +39,10 @@RTE_ETH_RSS_IPV6 | \RTE_ETH_RSS_IPV6_EX | \RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ - RTE_ETH_RSS_IPV6_TCP_EX) +RTE_ETH_RSS_IPV6_TCP_EX | \ +RTE_ETH_RSS_NONFRAG_IPV4_UDP | \ +RTE_ETH_RSS_NONFRAG_IPV6_UDP | \ + RTE_ETH_RSS_IPV6_UDP_EX)/* A list of pages registered with the device during setup and used by a queue * as buffers
@@ -264,6 +267,7 @@ struct gve_priv {uint32_t adminq_destroy_tx_queue_cnt;uint32_t adminq_destroy_rx_queue_cnt;uint32_t adminq_dcfg_device_resources_cnt; +uint32_t adminq_cfg_rss_cnt;uint32_t adminq_set_driver_parameter_cnt;uint32_t adminq_report_stats_cnt;uint32_t adminq_report_link_speed_cnt;
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 3/7 net/gve: add gve_rss library for handling RSS-related behaviors
In reply to Joshua WashingtonThis change includes a number of helper functions to facilitate RSS configuration on the GVE DPDK driver. These methods are declared in gve_rss.h.
--- drivers/net/gve/base/gve_adminq.h | 10 +- drivers/net/gve/gve_ethdev.c | 2 +- drivers/net/gve/gve_ethdev.h | 4 +- drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++++ drivers/net/gve/meson.build | 1 + 6 files changed, 319 insertions(+), 11 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
drivers/net/gve/base/gve_adminq.h +1 -9
@@ -378,15 +378,6 @@ struct gve_adminq_get_ptype_map {__be64 ptype_map_addr; }; -#define GVE_RSS_HASH_IPV4 BIT(0) -#define GVE_RSS_HASH_TCPV4 BIT(1) -#define GVE_RSS_HASH_IPV6 BIT(2) -#define GVE_RSS_HASH_IPV6_EX BIT(3) -#define GVE_RSS_HASH_TCPV6 BIT(4) -#define GVE_RSS_HASH_TCPV6_EX BIT(5) -#define GVE_RSS_HASH_UDPV4 BIT(6) -#define GVE_RSS_HASH_UDPV6 BIT(7) -#define GVE_RSS_HASH_UDPV6_EX BIT(8)/* RSS configuration command */ struct gve_adminq_configure_rss {
@@ -428,6 +419,7 @@ union gve_adminq_command {GVE_CHECK_UNION_LEN (64 , gve_adminq_command);struct gve_priv ; +struct gve_rss_config ;struct gve_queue_page_list ;int gve_adminq_alloc (struct gve_priv *priv);void gve_adminq_free (struct gve_priv *priv);
drivers/net/gve/gve_ethdev.c +1 -1
@@ -442,7 +442,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .nb_align = 1, }; - dev_info->flow_type_rss_offloads =GVE_RSS_OFFLOAD_ALL ; + dev_info->flow_type_rss_offloads =GVE_RTE_RSS_OFFLOAD_ALL ;return 0 ; }
drivers/net/gve/gve_ethdev.h +3 -1
@@ -33,7 +33,7 @@ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG) -#define GVE_RSS_OFFLOAD_ALL ( \ +#define GVE_RTE_RSS_OFFLOAD_ALL ( \ RTE_ETH_RSS_IPV4 | \ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ RTE_ETH_RSS_IPV6 | \
@@ -290,6 +290,8 @@ struct gve_priv {const struct rte_memzone *stats_report_mem;uint16_t stats_start_idx;/* start index of array of stats written by NIC */ uint16_t stats_end_idx;/* end index of array of stats written by NIC */ + +struct gve_rss_config rss_config; }; static inline bool
drivers/net/gve/gve_rss.c new file +206 -0
206 lines not on this page · raw message
drivers/net/gve/gve_rss.h new file +107 -0
107 lines not on this page · raw message
drivers/net/gve/meson.build +1 -0
@@ -15,5 +15,6 @@ sources = files( 'gve_tx_dqo.c', 'gve_ethdev.c', 'gve_version.c', + 'gve_rss.c', ) includes += include_directories('base')
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 4/7 net/gve: RSS configuration update support
In reply to Joshua WashingtonThis patch adds support for updating the RSS hash key and hash fields in the GVE PMD through the implementation of rss_hash_update and rss_hash_conf_get.
The RSS hash key for gVNIC is required to be 40 bytes. On initial configuration of the RSS hash key, the RSS redirection table will be set to a static default, using a round-robin approach for all queues. Note, however, that this patch does not include support for setting the redirection table explicitly. In dev_configure, if the static redirection table has been set, it will be updated to reflect the new queue count, if it has changed.
The RSS key must be set before any other RSS configuration can happen. As such, an attempt to set the hash types before the key is configured will fail.
--- drivers/net/gve/gve_ethdev.c | 132 ++++++++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 3 + 2 files changed, 133 insertions(+), 2 deletions(-)
drivers/net/gve/gve_ethdev.c +130 -2
@@ -1,5 +1,6 @@/* SPDX-License-Identifier: BSD-3-Clause -* Copyright(C) 2022 Intel Corporation +* Copyright(C) 2022-2023 Intel Corporation +* Copyright(C) 2023 Google LLC */ #include "gve_ethdev.h"
@@ -8,6 +9,7 @@#include "base/gve_osdep.h" #include "gve_version.h" #include "rte_ether.h" +#include "gve_rss.h" static void gve_write_version (uint8_t *driver_version_register)
@@ -88,12 +90,31 @@ gve_dev_configure(struct rte_eth_dev *dev) {struct gve_priv *priv = dev->data->dev_private; -if (dev->data->dev_conf.rxmode.mq_mode &RTE_ETH_MQ_RX_RSS_FLAG ) +if (dev->data->dev_conf.rxmode.mq_mode &RTE_ETH_MQ_RX_RSS_FLAG ) { dev->data->dev_conf.rxmode.offloads |=RTE_ETH_RX_OFFLOAD_RSS_HASH ; + priv->rss_config.alg =GVE_RSS_HASH_TOEPLITZ ; + }if (dev->data->dev_conf.rxmode.offloads &RTE_ETH_RX_OFFLOAD_TCP_LRO ) priv->enable_rsc =1 ; +/* Reset RSS RETA in case number of queues changed. */ +if (priv->rss_config.indir) { +struct gve_rss_config update_reta_config; +gve_init_rss_config_from_priv (priv, &update_reta_config); +gve_generate_rss_reta (dev, &update_reta_config); + +int err =gve_adminq_configure_rss (priv, &update_reta_config); +if (err) +PMD_DRV_LOG (ERR , +"Could not reconfigure RSS redirection table." ); +else +gve_update_priv_rss_config (priv, &update_reta_config); + +gve_free_rss_config (&update_reta_config); +return err; + } +return 0 ; }
@@ -443,6 +464,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) }; dev_info->flow_type_rss_offloads =GVE_RTE_RSS_OFFLOAD_ALL ; + dev_info->hash_key_size =GVE_RSS_HASH_KEY_SIZE ; + dev_info->reta_size =GVE_RSS_INDIR_SIZE ;return 0 ; }
@@ -646,6 +669,107 @@ gve_xstats_get_names(struct rte_eth_dev *dev,return count; } + +static int +gve_rss_hash_update (struct rte_eth_dev *dev, +struct rte_eth_rss_conf *rss_conf) +{ +struct gve_priv *priv = dev->data->dev_private; +struct gve_rss_config gve_rss_conf; +int rss_reta_size; +int err; + +if (gve_validate_rss_hf (rss_conf->rss_hf)) { +PMD_DRV_LOG (ERR ,"Unsupported hash function." ); +return -EINVAL ; + } + +if (rss_conf->algorithm !=RTE_ETH_HASH_FUNCTION_TOEPLITZ && + rss_conf->algorithm !=RTE_ETH_HASH_FUNCTION_DEFAULT ) { +PMD_DRV_LOG (ERR ,"Device only supports Toeplitz algorithm." ); +return -EINVAL ; + } + +if (rss_conf->rss_key_len) { +if (rss_conf->rss_key_len !=GVE_RSS_HASH_KEY_SIZE ) { +PMD_DRV_LOG (ERR , +"Invalid hash key size. Only RSS hash key size " +"of %u supported" ,GVE_RSS_HASH_KEY_SIZE ); +return -EINVAL ; + } + +if (!rss_conf->rss_key) { +PMD_DRV_LOG (ERR ,"RSS key must be non-null." ); +return -EINVAL ; + } + }else { +if (!priv->rss_config.key_size) { +PMD_DRV_LOG (ERR ,"RSS key must be initialized before " +"any other configuration." ); +return -EINVAL ; + } + rss_conf->rss_key_len = priv->rss_config.key_size; + } + + rss_reta_size = priv->rss_config.indir ? + priv->rss_config.indir_size : +GVE_RSS_INDIR_SIZE ; + err =gve_init_rss_config (&gve_rss_conf, rss_conf->rss_key_len, + rss_reta_size); +if (err) +return err; + + gve_rss_conf.alg =GVE_RSS_HASH_TOEPLITZ ; + err =gve_update_rss_hash_types (priv, &gve_rss_conf, rss_conf); +if (err) + goto err; + err =gve_update_rss_key (priv, &gve_rss_conf, rss_conf); +if (err) + goto err; + +/* Set redirection table to default or preexisting. */ +if (!priv->rss_config.indir) +gve_generate_rss_reta (dev, &gve_rss_conf); +else +memcpy (gve_rss_conf.indir, priv->rss_config.indir, + gve_rss_conf.indir_size *sizeof (*priv->rss_config.indir)); + + err =gve_adminq_configure_rss (priv, &gve_rss_conf); +if (!err) +gve_update_priv_rss_config (priv, &gve_rss_conf); + +err: +gve_free_rss_config (&gve_rss_conf); +return err; +} + +static int +gve_rss_hash_conf_get (struct rte_eth_dev *dev, +struct rte_eth_rss_conf *rss_conf) +{ +struct gve_priv *priv = dev->data->dev_private; + +if (!(dev->data->dev_conf.rxmode.offloads & +RTE_ETH_RX_OFFLOAD_RSS_HASH )) { +PMD_DRV_LOG (ERR ,"RSS not configured." ); +return -ENOTSUP ; + } + + +gve_to_rte_rss_hf (priv->rss_config.hash_types, rss_conf); + rss_conf->rss_key_len = priv->rss_config.key_size; +if (rss_conf->rss_key) { +if (!priv->rss_config.key) { +PMD_DRV_LOG (ERR ,"Unable to retrieve default RSS hash key." ); +return -ENOTSUP ; + } +memcpy (rss_conf->rss_key, priv->rss_config.key, + rss_conf->rss_key_len *sizeof (*rss_conf->rss_key)); + } + +return 0 ; +} +static const struct eth_dev_ops gve_eth_dev_ops = { .dev_configure = gve_dev_configure, .dev_start = gve_dev_start,
@@ -666,6 +790,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .mtu_set = gve_dev_mtu_set, .xstats_get = gve_xstats_get, .xstats_get_names = gve_xstats_get_names, + .rss_hash_update = gve_rss_hash_update, + .rss_hash_conf_get = gve_rss_hash_conf_get, };static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -688,6 +814,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = { .mtu_set = gve_dev_mtu_set, .xstats_get = gve_xstats_get, .xstats_get_names = gve_xstats_get_names, + .rss_hash_update = gve_rss_hash_update, + .rss_hash_conf_get = gve_rss_hash_conf_get, }; static void
drivers/net/gve/gve_ethdev.h +3 -0
@@ -29,6 +29,9 @@#define GVE_RX_MIN_BUF_SIZE_GQI 2048#define GVE_RX_MAX_BUF_SIZE_GQI 4096 +#define GVE_RSS_HASH_KEY_SIZE 40 +#define GVE_RSS_INDIR_SIZE 128 +#define GVE_TX_CKSUM_OFFLOAD_MASK ( \ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG)
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 5/7 net/gve: RSS redirection table update support
In reply to Joshua WashingtonThis patch introduces support for updating the RSS redirection table in the GVE PMD through the implementation of rss_reta_update and rss_reta_query.
Due to an infrastructure limitation, the RSS hash key must be manually configured before the redirection table can be updated or queried. The redirection table is expected to be exactly 128 bytes.
--- drivers/net/gve/gve_ethdev.c | 95 ++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+)
drivers/net/gve/gve_ethdev.c +95 -0
@@ -770,6 +770,97 @@ gve_rss_hash_conf_get(struct rte_eth_dev *dev,return 0 ; } +static int +gve_rss_reta_update (struct rte_eth_dev *dev, +struct rte_eth_rss_reta_entry64 *reta_conf,uint16_t reta_size) +{ +struct gve_priv *priv = dev->data->dev_private; +struct gve_rss_config gve_rss_conf; +int table_id; +int err; +int i; + +/* RSS key must be set before the redirection table can be set. */ +if (!priv->rss_config.key || priv->rss_config.key_size ==0 ) { +PMD_DRV_LOG (ERR ,"RSS hash key msut be set before the " +"redirection table can be updated." ); +return -ENOTSUP ; + } + +if (reta_size !=GVE_RSS_INDIR_SIZE ) { +PMD_DRV_LOG (ERR ,"Redirection table must have %hu elements" , + (uint16_t )GVE_RSS_INDIR_SIZE ); +return -EINVAL ; + } + + err =gve_init_rss_config_from_priv (priv, &gve_rss_conf); +if (err) { +PMD_DRV_LOG (ERR ,"Error allocating new RSS config." ); +return err; + } + + table_id =0 ; +for (i =0 ; i < priv->rss_config.indir_size; i++) { +int table_entry = i %RTE_ETH_RETA_GROUP_SIZE ; +if (reta_conf[table_id].mask & (1ULL << table_entry)) + gve_rss_conf.indir[i] = + reta_conf[table_id].reta[table_entry]; + +if (table_entry ==RTE_ETH_RETA_GROUP_SIZE -1 ) + table_id++; + } + + err =gve_adminq_configure_rss (priv, &gve_rss_conf); +if (err) +PMD_DRV_LOG (ERR ,"Problem configuring RSS with device." ); +else +gve_update_priv_rss_config (priv, &gve_rss_conf); + +gve_free_rss_config (&gve_rss_conf); +return err; +} + +static int +gve_rss_reta_query (struct rte_eth_dev *dev, +struct rte_eth_rss_reta_entry64 *reta_conf,uint16_t reta_size) +{ +struct gve_priv *priv = dev->data->dev_private; +int table_id; +int i; + +if (!(dev->data->dev_conf.rxmode.offloads & +RTE_ETH_RX_OFFLOAD_RSS_HASH )) { +PMD_DRV_LOG (ERR ,"RSS not configured." ); +return -ENOTSUP ; + } + +/* RSS key must be set before the redirection table can be queried. */ +if (!priv->rss_config.key) { +PMD_DRV_LOG (ERR ,"RSS hash key must be set before the " +"redirection table can be initialized." ); +return -ENOTSUP ; + } + +if (reta_size != priv->rss_config.indir_size) { +PMD_DRV_LOG (ERR ,"RSS redirection table must have %d entries." , + priv->rss_config.indir_size); +return -EINVAL ; + } + + table_id =0 ; +for (i =0 ; i < priv->rss_config.indir_size; i++) { +int table_entry = i %RTE_ETH_RETA_GROUP_SIZE ; +if (reta_conf[table_id].mask & (1ULL << table_entry)) + reta_conf[table_id].reta[table_entry] = + priv->rss_config.indir[i]; + +if (table_entry ==RTE_ETH_RETA_GROUP_SIZE -1 ) + table_id++; + } + +return 0 ; +} +static const struct eth_dev_ops gve_eth_dev_ops = { .dev_configure = gve_dev_configure, .dev_start = gve_dev_start,
@@ -792,6 +883,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .xstats_get_names = gve_xstats_get_names, .rss_hash_update = gve_rss_hash_update, .rss_hash_conf_get = gve_rss_hash_conf_get, + .reta_update = gve_rss_reta_update, + .reta_query = gve_rss_reta_query, };static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -816,6 +909,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = { .xstats_get_names = gve_xstats_get_names, .rss_hash_update = gve_rss_hash_update, .rss_hash_conf_get = gve_rss_hash_conf_get, + .reta_update = gve_rss_reta_update, + .reta_query = gve_rss_reta_query, }; static void
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 6/7 net/gve: update gve.ini with RSS capabilities
In reply to Joshua WashingtonThis patch updates the DPDK feature matrix to expose that the GVE driver supports RSS hash, RSS key update, and RSS reta update.
--- doc/guides/nics/features/gve.ini | 3 +++ 1 file changed, 3 insertions(+)
doc/guides/nics/features/gve.ini +3 -0
@@ -15,3 +15,6 @@ Linux = Y x86-32 = Y x86-64 = Y Usage doc = Y +RSS hash = Y +RSS key update = Y +RSS reta update = Y
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 7/7 net/gve: update GVE documentation with RSS support
In reply to Joshua WashingtonThis patch updates the GVE doc page to communicate that GVE now supports RSS configuration and explains the limitations.
--- doc/guides/nics/gve.rst | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
doc/guides/nics/gve.rst +10 -6
@@ -70,6 +70,8 @@ Supported features of the GVE PMD are: - Link state information - Tx multi-segments (Scatter Tx) - Tx UDP/TCP/SCTP Checksum +- RSS hash configuration +- RSS redirection table update and query Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD. Jumbo Frame is not supported in PMD for now.
@@ -77,10 +79,12 @@ It'll be added in a future DPDK release. Also, only GQI_QPL queue format is in use on GCP since GQI_RDA hasn't been released in production. -Currently, setting MTU with value larger than 1460 is not supported. +RSS +^^^ -Currently, only "RSS hash" is force enabled -so that the backend hardware device calculated hash values -could be shared with applications. -But for RSS, there is no such API to config RSS hash function or RETA table. -So, limited RSS is supported only with default config/setting. +GVE RSS can be enabled and configured using the standard interfaces. The driver +does not support querying the initial RSS configuration. + +The RSS hash key must be exactly 40 bytes, and the redirection table must have +128 entries. The RSS hash key must be configured before the redirection table +can be set up.
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
cover net/gve: RSS Support for GVE Driver
In reply to Joshua WashingtonThis patch series introduces RSS support for the GVE poll-mode driver. This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set with a key before the redriection table is set up. When the RSS hash is configured, a default redirection table is generated based on the number of queues. When the device is re-configured, the redirection table is reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for RSS redirection table entries, while the RTE API uses 16 bit integers. However, this is unlikely to be an issue, as these values represent receive queues, and the gVNIC device does not support anywhere near 64K queues.
This series also updates the corresponding feature matrix ertries and documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches fixes.
Note: There is a warning about complex macros being parenthesized that does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.
Joshua Washington (7):
net/gve: fully expose RSS offload support in dev_info net/gve: RSS adminq command changes
net/gve: add gve_rss library for handling RSS-related behaviors net/gve: RSS configuration update support
net/gve: RSS redirection table update support net/gve: update gve.ini with RSS capabilities net/gve: update GVE documentation with RSS support
doc/guides/nics/features/gve.ini | 3 + doc/guides/nics/gve.rst | 16 ++- drivers/net/gve/base/gve.h | 15 ++ drivers/net/gve/base/gve_adminq.c | 59 ++++++++ drivers/net/gve/base/gve_adminq.h | 21 +++ drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 17 +++ drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++ drivers/net/gve/meson.build | 1 + 10 files changed, 667 insertions(+), 9 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
v3 cover net/gve: RSS Support for GVE Driver
In reply to Joshua WashingtonThis patch series introduces RSS support for the GVE poll-mode driver. This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set with a key before the redriection table is set up. When the RSS hash is configured, a default redirection table is generated based on the number of queues. When the device is re-configured, the redirection table is reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for RSS redirection table entries, while the RTE API uses 16 bit integers. However, this is unlikely to be an issue, as these values represent receive queues, and the gVNIC device does not support anywhere near 64K queues.
This series also updates the corresponding feature matrix ertries and documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches fixes.
Note: There is a warning about complex macros being parenthesized that does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.
Joshua Washington (7):
net/gve: fully expose RSS offload support in dev_info net/gve: RSS adminq command changes
net/gve: add gve_rss library for handling RSS-related behaviors net/gve: RSS configuration update support
net/gve: RSS redirection table update support net/gve: update gve.ini with RSS capabilities net/gve: update GVE documentation with RSS support
doc/guides/nics/features/gve.ini | 3 + doc/guides/nics/gve.rst | 16 ++- drivers/net/gve/base/gve.h | 15 ++ drivers/net/gve/base/gve_adminq.c | 59 ++++++++ drivers/net/gve/base/gve_adminq.h | 21 +++ drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 17 +++ drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++ drivers/net/gve/meson.build | 1 + 10 files changed, 667 insertions(+), 9 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
v3 cover net/gve: RSS Support for GVE Driver
In reply to Joshua WashingtonThis patch series introduces RSS support for the GVE poll-mode driver. This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set with a key before the redriection table is set up. When the RSS hash is configured, a default redirection table is generated based on the number of queues. When the device is re-configured, the redirection table is reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for RSS redirection table entries, while the RTE API uses 16 bit integers. However, this is unlikely to be an issue, as these values represent receive queues, and the gVNIC device does not support anywhere near 64K queues.
This series also updates the corresponding feature matrix ertries and documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches fixes.
Note: There is a warning about complex macros being parenthesized that does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.
Joshua Washington (7):
net/gve: fully expose RSS offload support in dev_info net/gve: RSS adminq command changes
net/gve: add gve_rss library for handling RSS-related behaviors net/gve: RSS configuration update support
net/gve: RSS redirection table update support net/gve: update gve.ini with RSS capabilities net/gve: update GVE documentation with RSS support
doc/guides/nics/features/gve.ini | 3 + doc/guides/nics/gve.rst | 16 ++- drivers/net/gve/base/gve.h | 15 ++ drivers/net/gve/base/gve_adminq.c | 59 ++++++++ drivers/net/gve/base/gve_adminq.h | 21 +++ drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 17 +++ drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++ drivers/net/gve/meson.build | 1 + 10 files changed, 667 insertions(+), 9 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 1/7 net/gve: fully expose RSS offload support in dev_info
This patch communicates that the GVE driver supports RSS, along with the RSS offloads supported by the driver.
--- drivers/net/gve/gve_ethdev.c | 4 +++- drivers/net/gve/gve_ethdev.h | 8 ++++++++ 2 files changed, 11 insertions(+), 1 deletion(-)
drivers/net/gve/gve_ethdev.c +3 -1
@@ -405,7 +405,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_mtu = priv->max_mtu; dev_info->min_mtu =RTE_ETHER_MIN_MTU ; - dev_info->rx_offload_capa =0 ; + dev_info->rx_offload_capa =RTE_ETH_RX_OFFLOAD_RSS_HASH ; dev_info->tx_offload_capa =RTE_ETH_TX_OFFLOAD_MULTI_SEGS | RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
@@ -442,6 +442,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .nb_align = 1, }; + dev_info->flow_type_rss_offloads =GVE_RSS_OFFLOAD_ALL ; +return 0 ; }
drivers/net/gve/gve_ethdev.h +8 -0
@@ -33,6 +33,14 @@ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG) +#define GVE_RSS_OFFLOAD_ALL ( \ + RTE_ETH_RSS_IPV4 | \ + RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ + RTE_ETH_RSS_IPV6 | \ + RTE_ETH_RSS_IPV6_EX | \ + RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ + RTE_ETH_RSS_IPV6_TCP_EX) +/* A list of pages registered with the device during setup and used by a queue * as buffers */
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 2/7 net/gve: RSS adminq command changes
In reply to Joshua WashingtonThis change introduces admin queue changes that enable the configuration of RSS parameters for the GVE driver.
--- drivers/net/gve/base/gve.h | 15 ++++++++ drivers/net/gve/base/gve_adminq.c | 58 +++++++++++++++++++++++++++++++ drivers/net/gve/base/gve_adminq.h | 29 ++++++++++++++++ drivers/net/gve/gve_ethdev.h | 6 +++- 4 files changed, 107 insertions(+), 1 deletion(-)
drivers/net/gve/base/gve.h +15 -0
@@ -51,4 +51,19 @@ enum gve_state_flags_bit { GVE_PRIV_FLAGS_NAPI_ENABLED = 4, }; +enum gve_rss_hash_algorithm { +GVE_RSS_HASH_UNDEFINED =0 , +GVE_RSS_HASH_TOEPLITZ =1 , +}; + +struct gve_rss_config { +uint16_t hash_types; +enum gve_rss_hash_algorithm alg; +uint16_t key_size; +uint16_t indir_size; +uint8_t *key; +uint32_t *indir; +}; + +#endif /* _GVE_H_ */
drivers/net/gve/base/gve_adminq.c +58 -0
67 lines not on this page · raw message
drivers/net/gve/base/gve_adminq.h +29 -0
@@ -19,6 +19,7 @@ enum gve_adminq_opcodes {GVE_ADMINQ_DESTROY_TX_QUEUE =0x7 ,GVE_ADMINQ_DESTROY_RX_QUEUE =0x8 ,GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES =0x9 , +GVE_ADMINQ_CONFIGURE_RSS =0xA ,GVE_ADMINQ_SET_DRIVER_PARAMETER =0xB ,GVE_ADMINQ_REPORT_STATS =0xC , GVE_ADMINQ_REPORT_LINK_SPEED = 0xD,
@@ -377,6 +378,27 @@ struct gve_adminq_get_ptype_map {__be64 ptype_map_addr; }; +#define GVE_RSS_HASH_IPV4 BIT(0) +#define GVE_RSS_HASH_TCPV4 BIT(1) +#define GVE_RSS_HASH_IPV6 BIT(2) +#define GVE_RSS_HASH_IPV6_EX BIT(3) +#define GVE_RSS_HASH_TCPV6 BIT(4) +#define GVE_RSS_HASH_TCPV6_EX BIT(5) +#define GVE_RSS_HASH_UDPV4 BIT(6) +#define GVE_RSS_HASH_UDPV6 BIT(7) +#define GVE_RSS_HASH_UDPV6_EX BIT(8) + +/* RSS configuration command */ +struct gve_adminq_configure_rss { +__be16 hash_types; +u8 halg;/* hash algorithm */ +u8 reserved; +__be16 hkey_len; +__be16 indir_len; +__be64 hkey_addr; +__be64 indir_addr; +}; + union gve_adminq_command { struct { __be32 opcode;
@@ -391,6 +413,7 @@ union gve_adminq_command {struct gve_adminq_describe_device describe_device;struct gve_adminq_register_page_list reg_page_list;struct gve_adminq_unregister_page_list unreg_page_list; +struct gve_adminq_configure_rss configure_rss;struct gve_adminq_set_driver_parameter set_driver_param;struct gve_adminq_report_stats report_stats;struct gve_adminq_report_link_speed report_link_speed;
@@ -404,6 +427,8 @@ union gve_adminq_command {GVE_CHECK_UNION_LEN (64 , gve_adminq_command); +struct gve_priv ; +struct gve_queue_page_list ;int gve_adminq_alloc (struct gve_priv *priv);void gve_adminq_free (struct gve_priv *priv);void gve_adminq_release (struct gve_priv *priv);
@@ -433,4 +458,8 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,int gve_adminq_verify_driver_compatibility (struct gve_priv *priv,u64 driver_info_len,dma_addr_t driver_info_addr); + +int gve_adminq_configure_rss (struct gve_priv *priv, +struct gve_rss_config *rss_config); +#endif /* _GVE_ADMINQ_H */
drivers/net/gve/gve_ethdev.h +5 -1
@@ -39,7 +39,10 @@RTE_ETH_RSS_IPV6 | \RTE_ETH_RSS_IPV6_EX | \RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ - RTE_ETH_RSS_IPV6_TCP_EX) +RTE_ETH_RSS_IPV6_TCP_EX | \ +RTE_ETH_RSS_NONFRAG_IPV4_UDP | \ +RTE_ETH_RSS_NONFRAG_IPV6_UDP | \ + RTE_ETH_RSS_IPV6_UDP_EX)/* A list of pages registered with the device during setup and used by a queue * as buffers
@@ -264,6 +267,7 @@ struct gve_priv {uint32_t adminq_destroy_tx_queue_cnt;uint32_t adminq_destroy_rx_queue_cnt;uint32_t adminq_dcfg_device_resources_cnt; +uint32_t adminq_cfg_rss_cnt;uint32_t adminq_set_driver_parameter_cnt;uint32_t adminq_report_stats_cnt;uint32_t adminq_report_link_speed_cnt;
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 3/7 net/gve: add gve_rss library for handling RSS-related behaviors
In reply to Joshua WashingtonThis change includes a number of helper functions to facilitate RSS configuration on the GVE DPDK driver. These methods are declared in gve_rss.h.
--- drivers/net/gve/base/gve_adminq.h | 10 +- drivers/net/gve/gve_ethdev.c | 2 +- drivers/net/gve/gve_ethdev.h | 4 +- drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++++ drivers/net/gve/meson.build | 1 + 6 files changed, 319 insertions(+), 11 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
drivers/net/gve/base/gve_adminq.h +1 -9
@@ -378,15 +378,6 @@ struct gve_adminq_get_ptype_map {__be64 ptype_map_addr; }; -#define GVE_RSS_HASH_IPV4 BIT(0) -#define GVE_RSS_HASH_TCPV4 BIT(1) -#define GVE_RSS_HASH_IPV6 BIT(2) -#define GVE_RSS_HASH_IPV6_EX BIT(3) -#define GVE_RSS_HASH_TCPV6 BIT(4) -#define GVE_RSS_HASH_TCPV6_EX BIT(5) -#define GVE_RSS_HASH_UDPV4 BIT(6) -#define GVE_RSS_HASH_UDPV6 BIT(7) -#define GVE_RSS_HASH_UDPV6_EX BIT(8)/* RSS configuration command */ struct gve_adminq_configure_rss {
@@ -428,6 +419,7 @@ union gve_adminq_command {GVE_CHECK_UNION_LEN (64 , gve_adminq_command);struct gve_priv ; +struct gve_rss_config ;struct gve_queue_page_list ;int gve_adminq_alloc (struct gve_priv *priv);void gve_adminq_free (struct gve_priv *priv);
drivers/net/gve/gve_ethdev.c +1 -1
@@ -442,7 +442,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .nb_align = 1, }; - dev_info->flow_type_rss_offloads =GVE_RSS_OFFLOAD_ALL ; + dev_info->flow_type_rss_offloads =GVE_RTE_RSS_OFFLOAD_ALL ;return 0 ; }
drivers/net/gve/gve_ethdev.h +3 -1
@@ -33,7 +33,7 @@ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG) -#define GVE_RSS_OFFLOAD_ALL ( \ +#define GVE_RTE_RSS_OFFLOAD_ALL ( \ RTE_ETH_RSS_IPV4 | \ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ RTE_ETH_RSS_IPV6 | \
@@ -290,6 +290,8 @@ struct gve_priv {const struct rte_memzone *stats_report_mem;uint16_t stats_start_idx;/* start index of array of stats written by NIC */ uint16_t stats_end_idx;/* end index of array of stats written by NIC */ + +struct gve_rss_config rss_config; }; static inline bool
drivers/net/gve/gve_rss.c new file +206 -0
206 lines not on this page · raw message
drivers/net/gve/gve_rss.h new file +107 -0
107 lines not on this page · raw message
drivers/net/gve/meson.build +1 -0
@@ -15,5 +15,6 @@ sources = files( 'gve_tx_dqo.c', 'gve_ethdev.c', 'gve_version.c', + 'gve_rss.c', ) includes += include_directories('base')
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 4/7 net/gve: RSS configuration update support
In reply to Joshua WashingtonThis patch adds support for updating the RSS hash key and hash fields in the GVE PMD through the implementation of rss_hash_update and rss_hash_conf_get.
The RSS hash key for gVNIC is required to be 40 bytes. On initial configuration of the RSS hash key, the RSS redirection table will be set to a static default, using a round-robin approach for all queues. Note, however, that this patch does not include support for setting the redirection table explicitly. In dev_configure, if the static redirection table has been set, it will be updated to reflect the new queue count, if it has changed.
The RSS key must be set before any other RSS configuration can happen. As such, an attempt to set the hash types before the key is configured will fail.
--- drivers/net/gve/gve_ethdev.c | 132 ++++++++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 3 + 2 files changed, 133 insertions(+), 2 deletions(-)
drivers/net/gve/gve_ethdev.c +130 -2
@@ -1,5 +1,6 @@/* SPDX-License-Identifier: BSD-3-Clause -* Copyright(C) 2022 Intel Corporation +* Copyright(C) 2022-2023 Intel Corporation +* Copyright(C) 2023 Google LLC */ #include "gve_ethdev.h"
@@ -8,6 +9,7 @@#include "base/gve_osdep.h" #include "gve_version.h" #include "rte_ether.h" +#include "gve_rss.h" static void gve_write_version (uint8_t *driver_version_register)
@@ -88,12 +90,31 @@ gve_dev_configure(struct rte_eth_dev *dev) {struct gve_priv *priv = dev->data->dev_private; -if (dev->data->dev_conf.rxmode.mq_mode &RTE_ETH_MQ_RX_RSS_FLAG ) +if (dev->data->dev_conf.rxmode.mq_mode &RTE_ETH_MQ_RX_RSS_FLAG ) { dev->data->dev_conf.rxmode.offloads |=RTE_ETH_RX_OFFLOAD_RSS_HASH ; + priv->rss_config.alg =GVE_RSS_HASH_TOEPLITZ ; + }if (dev->data->dev_conf.rxmode.offloads &RTE_ETH_RX_OFFLOAD_TCP_LRO ) priv->enable_rsc =1 ; +/* Reset RSS RETA in case number of queues changed. */ +if (priv->rss_config.indir) { +struct gve_rss_config update_reta_config; +gve_init_rss_config_from_priv (priv, &update_reta_config); +gve_generate_rss_reta (dev, &update_reta_config); + +int err =gve_adminq_configure_rss (priv, &update_reta_config); +if (err) +PMD_DRV_LOG (ERR , +"Could not reconfigure RSS redirection table." ); +else +gve_update_priv_rss_config (priv, &update_reta_config); + +gve_free_rss_config (&update_reta_config); +return err; + } +return 0 ; }
@@ -443,6 +464,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) }; dev_info->flow_type_rss_offloads =GVE_RTE_RSS_OFFLOAD_ALL ; + dev_info->hash_key_size =GVE_RSS_HASH_KEY_SIZE ; + dev_info->reta_size =GVE_RSS_INDIR_SIZE ;return 0 ; }
@@ -646,6 +669,107 @@ gve_xstats_get_names(struct rte_eth_dev *dev,return count; } + +static int +gve_rss_hash_update (struct rte_eth_dev *dev, +struct rte_eth_rss_conf *rss_conf) +{ +struct gve_priv *priv = dev->data->dev_private; +struct gve_rss_config gve_rss_conf; +int rss_reta_size; +int err; + +if (gve_validate_rss_hf (rss_conf->rss_hf)) { +PMD_DRV_LOG (ERR ,"Unsupported hash function." ); +return -EINVAL ; + } + +if (rss_conf->algorithm !=RTE_ETH_HASH_FUNCTION_TOEPLITZ && + rss_conf->algorithm !=RTE_ETH_HASH_FUNCTION_DEFAULT ) { +PMD_DRV_LOG (ERR ,"Device only supports Toeplitz algorithm." ); +return -EINVAL ; + } + +if (rss_conf->rss_key_len) { +if (rss_conf->rss_key_len !=GVE_RSS_HASH_KEY_SIZE ) { +PMD_DRV_LOG (ERR , +"Invalid hash key size. Only RSS hash key size " +"of %u supported" ,GVE_RSS_HASH_KEY_SIZE ); +return -EINVAL ; + } + +if (!rss_conf->rss_key) { +PMD_DRV_LOG (ERR ,"RSS key must be non-null." ); +return -EINVAL ; + } + }else { +if (!priv->rss_config.key_size) { +PMD_DRV_LOG (ERR ,"RSS key must be initialized before " +"any other configuration." ); +return -EINVAL ; + } + rss_conf->rss_key_len = priv->rss_config.key_size; + } + + rss_reta_size = priv->rss_config.indir ? + priv->rss_config.indir_size : +GVE_RSS_INDIR_SIZE ; + err =gve_init_rss_config (&gve_rss_conf, rss_conf->rss_key_len, + rss_reta_size); +if (err) +return err; + + gve_rss_conf.alg =GVE_RSS_HASH_TOEPLITZ ; + err =gve_update_rss_hash_types (priv, &gve_rss_conf, rss_conf); +if (err) + goto err; + err =gve_update_rss_key (priv, &gve_rss_conf, rss_conf); +if (err) + goto err; + +/* Set redirection table to default or preexisting. */ +if (!priv->rss_config.indir) +gve_generate_rss_reta (dev, &gve_rss_conf); +else +memcpy (gve_rss_conf.indir, priv->rss_config.indir, + gve_rss_conf.indir_size *sizeof (*priv->rss_config.indir)); + + err =gve_adminq_configure_rss (priv, &gve_rss_conf); +if (!err) +gve_update_priv_rss_config (priv, &gve_rss_conf); + +err: +gve_free_rss_config (&gve_rss_conf); +return err; +} + +static int +gve_rss_hash_conf_get (struct rte_eth_dev *dev, +struct rte_eth_rss_conf *rss_conf) +{ +struct gve_priv *priv = dev->data->dev_private; + +if (!(dev->data->dev_conf.rxmode.offloads & +RTE_ETH_RX_OFFLOAD_RSS_HASH )) { +PMD_DRV_LOG (ERR ,"RSS not configured." ); +return -ENOTSUP ; + } + + +gve_to_rte_rss_hf (priv->rss_config.hash_types, rss_conf); + rss_conf->rss_key_len = priv->rss_config.key_size; +if (rss_conf->rss_key) { +if (!priv->rss_config.key) { +PMD_DRV_LOG (ERR ,"Unable to retrieve default RSS hash key." ); +return -ENOTSUP ; + } +memcpy (rss_conf->rss_key, priv->rss_config.key, + rss_conf->rss_key_len *sizeof (*rss_conf->rss_key)); + } + +return 0 ; +} +static const struct eth_dev_ops gve_eth_dev_ops = { .dev_configure = gve_dev_configure, .dev_start = gve_dev_start,
@@ -666,6 +790,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .mtu_set = gve_dev_mtu_set, .xstats_get = gve_xstats_get, .xstats_get_names = gve_xstats_get_names, + .rss_hash_update = gve_rss_hash_update, + .rss_hash_conf_get = gve_rss_hash_conf_get, };static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -688,6 +814,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = { .mtu_set = gve_dev_mtu_set, .xstats_get = gve_xstats_get, .xstats_get_names = gve_xstats_get_names, + .rss_hash_update = gve_rss_hash_update, + .rss_hash_conf_get = gve_rss_hash_conf_get, }; static void
drivers/net/gve/gve_ethdev.h +3 -0
@@ -29,6 +29,9 @@#define GVE_RX_MIN_BUF_SIZE_GQI 2048#define GVE_RX_MAX_BUF_SIZE_GQI 4096 +#define GVE_RSS_HASH_KEY_SIZE 40 +#define GVE_RSS_INDIR_SIZE 128 +#define GVE_TX_CKSUM_OFFLOAD_MASK ( \ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG)
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 5/7 net/gve: RSS redirection table update support
In reply to Joshua WashingtonThis patch introduces support for updating the RSS redirection table in the GVE PMD through the implementation of rss_reta_update and rss_reta_query.
Due to an infrastructure limitation, the RSS hash key must be manually configured before the redirection table can be updated or queried. The redirection table is expected to be exactly 128 bytes.
--- drivers/net/gve/gve_ethdev.c | 95 ++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+)
drivers/net/gve/gve_ethdev.c +95 -0
@@ -770,6 +770,97 @@ gve_rss_hash_conf_get(struct rte_eth_dev *dev,return 0 ; } +static int +gve_rss_reta_update (struct rte_eth_dev *dev, +struct rte_eth_rss_reta_entry64 *reta_conf,uint16_t reta_size) +{ +struct gve_priv *priv = dev->data->dev_private; +struct gve_rss_config gve_rss_conf; +int table_id; +int err; +int i; + +/* RSS key must be set before the redirection table can be set. */ +if (!priv->rss_config.key || priv->rss_config.key_size ==0 ) { +PMD_DRV_LOG (ERR ,"RSS hash key msut be set before the " +"redirection table can be updated." ); +return -ENOTSUP ; + } + +if (reta_size !=GVE_RSS_INDIR_SIZE ) { +PMD_DRV_LOG (ERR ,"Redirection table must have %hu elements" , + (uint16_t )GVE_RSS_INDIR_SIZE ); +return -EINVAL ; + } + + err =gve_init_rss_config_from_priv (priv, &gve_rss_conf); +if (err) { +PMD_DRV_LOG (ERR ,"Error allocating new RSS config." ); +return err; + } + + table_id =0 ; +for (i =0 ; i < priv->rss_config.indir_size; i++) { +int table_entry = i %RTE_ETH_RETA_GROUP_SIZE ; +if (reta_conf[table_id].mask & (1ULL << table_entry)) + gve_rss_conf.indir[i] = + reta_conf[table_id].reta[table_entry]; + +if (table_entry ==RTE_ETH_RETA_GROUP_SIZE -1 ) + table_id++; + } + + err =gve_adminq_configure_rss (priv, &gve_rss_conf); +if (err) +PMD_DRV_LOG (ERR ,"Problem configuring RSS with device." ); +else +gve_update_priv_rss_config (priv, &gve_rss_conf); + +gve_free_rss_config (&gve_rss_conf); +return err; +} + +static int +gve_rss_reta_query (struct rte_eth_dev *dev, +struct rte_eth_rss_reta_entry64 *reta_conf,uint16_t reta_size) +{ +struct gve_priv *priv = dev->data->dev_private; +int table_id; +int i; + +if (!(dev->data->dev_conf.rxmode.offloads & +RTE_ETH_RX_OFFLOAD_RSS_HASH )) { +PMD_DRV_LOG (ERR ,"RSS not configured." ); +return -ENOTSUP ; + } + +/* RSS key must be set before the redirection table can be queried. */ +if (!priv->rss_config.key) { +PMD_DRV_LOG (ERR ,"RSS hash key must be set before the " +"redirection table can be initialized." ); +return -ENOTSUP ; + } + +if (reta_size != priv->rss_config.indir_size) { +PMD_DRV_LOG (ERR ,"RSS redirection table must have %d entries." , + priv->rss_config.indir_size); +return -EINVAL ; + } + + table_id =0 ; +for (i =0 ; i < priv->rss_config.indir_size; i++) { +int table_entry = i %RTE_ETH_RETA_GROUP_SIZE ; +if (reta_conf[table_id].mask & (1ULL << table_entry)) + reta_conf[table_id].reta[table_entry] = + priv->rss_config.indir[i]; + +if (table_entry ==RTE_ETH_RETA_GROUP_SIZE -1 ) + table_id++; + } + +return 0 ; +} +static const struct eth_dev_ops gve_eth_dev_ops = { .dev_configure = gve_dev_configure, .dev_start = gve_dev_start,
@@ -792,6 +883,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .xstats_get_names = gve_xstats_get_names, .rss_hash_update = gve_rss_hash_update, .rss_hash_conf_get = gve_rss_hash_conf_get, + .reta_update = gve_rss_reta_update, + .reta_query = gve_rss_reta_query, };static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -816,6 +909,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = { .xstats_get_names = gve_xstats_get_names, .rss_hash_update = gve_rss_hash_update, .rss_hash_conf_get = gve_rss_hash_conf_get, + .reta_update = gve_rss_reta_update, + .reta_query = gve_rss_reta_query, }; static void
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 6/7 net/gve: update gve.ini with RSS capabilities
In reply to Joshua WashingtonThis patch updates the DPDK feature matrix to expose that the GVE driver supports RSS hash, RSS key update, and RSS reta update.
--- doc/guides/nics/features/gve.ini | 3 +++ 1 file changed, 3 insertions(+)
doc/guides/nics/features/gve.ini +3 -0
@@ -15,3 +15,6 @@ Linux = Y x86-32 = Y x86-64 = Y Usage doc = Y +RSS hash = Y +RSS key update = Y +RSS reta update = Y
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v3 7/7 net/gve: update GVE documentation with RSS support
In reply to Joshua WashingtonThis patch updates the GVE doc page to communicate that GVE now supports RSS configuration and explains the limitations.
--- doc/guides/nics/gve.rst | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
doc/guides/nics/gve.rst +10 -6
@@ -70,6 +70,8 @@ Supported features of the GVE PMD are: - Link state information - Tx multi-segments (Scatter Tx) - Tx UDP/TCP/SCTP Checksum +- RSS hash configuration +- RSS redirection table update and query Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD. Jumbo Frame is not supported in PMD for now.
@@ -77,10 +79,12 @@ It'll be added in a future DPDK release. Also, only GQI_QPL queue format is in use on GCP since GQI_RDA hasn't been released in production. -Currently, setting MTU with value larger than 1460 is not supported. +RSS +^^^ -Currently, only "RSS hash" is force enabled -so that the backend hardware device calculated hash values -could be shared with applications. -But for RSS, there is no such API to config RSS hash function or RETA table. -So, limited RSS is supported only with default config/setting. +GVE RSS can be enabled and configured using the standard interfaces. The driver +does not support querying the initial RSS configuration. + +The RSS hash key must be exactly 40 bytes, and the redirection table must have +128 entries. The RSS hash key must be configured before the redirection table +can be set up.
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH v4 1/7 net/gve: fully expose RSS offload support in dev_info
This patch communicates that the GVE driver supports RSS, along with the RSS offloads supported by the driver.
--- drivers/net/gve/gve_ethdev.c | 4 +++- drivers/net/gve/gve_ethdev.h | 8 ++++++++ 2 files changed, 11 insertions(+), 1 deletion(-)
drivers/net/gve/gve_ethdev.c +3 -1
@@ -405,7 +405,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_mtu = priv->max_mtu; dev_info->min_mtu =RTE_ETHER_MIN_MTU ; - dev_info->rx_offload_capa =0 ; + dev_info->rx_offload_capa =RTE_ETH_RX_OFFLOAD_RSS_HASH ; dev_info->tx_offload_capa =RTE_ETH_TX_OFFLOAD_MULTI_SEGS | RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
@@ -442,6 +442,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .nb_align = 1, }; + dev_info->flow_type_rss_offloads =GVE_RSS_OFFLOAD_ALL ; +return 0 ; }
drivers/net/gve/gve_ethdev.h +8 -0
@@ -33,6 +33,14 @@ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG) +#define GVE_RSS_OFFLOAD_ALL ( \ + RTE_ETH_RSS_IPV4 | \ + RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ + RTE_ETH_RSS_IPV6 | \ + RTE_ETH_RSS_IPV6_EX | \ + RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ + RTE_ETH_RSS_IPV6_TCP_EX) +/* A list of pages registered with the device during setup and used by a queue * as buffers */
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
v5 cover net/gve: RSS Support for GVE Driver
This patch series introduces RSS support for the GVE poll-mode driver. This series includes implementations of the following eth_dev_ops:
1) rss_hash_update
2) rss_hash_conf_get
3) reta_query
4) reta_update
In rss_hash_update, the GVE driver supports the following RSS hash types:
* RTE_ETH_RSS_IPV4
* RTE_ETH_RSS_NONFRAG_IPV4_TCP
* RTE_ETH_RSS_NONFRAG_IPV4_UDP
* RTE_ETH_RSS_IPV6
* RTE_ETH_RSS_IPV6_EX
* RTE_ETH_RSS_NONFRAG_IPV6_TCP
* RTE_ETH_RSS_NONFRAG_IPV6_UDP
* RTE_ETH_RSS_IPV6_TCP_EX
* RTE_ETH_RSS_IPV6_UDP_EX
The hash key is 40B, and the lookup table has 128 entries. These values are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set with a key before the redriection table is set up. When the RSS hash is configured, a default redirection table is generated based on the number of queues. When the device is re-configured, the redirection table is reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for RSS redirection table entries, while the RTE API uses 16 bit integers. However, this is unlikely to be an issue, as these values represent receive queues, and the gVNIC device does not support anywhere near 64K queues.
This series also updates the corresponding feature matrix ertries and documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches fixes.
Note: There is a warning about complex macros being parenthesized that does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.
v4:
Fix formatting in gve_adminq.c
v5:
Update commit messages to comply with check-git-log.
Fold feature matrix and documentation updates into relevant commits.
Joshua Washington (5):
net/gve: expose RSS offload support in dev info net/gve: add RSS adminq command
net/gve: add gve RSS library
net/gve: add RSS configuration update support net/gve: add RSS redirection table update support
doc/guides/nics/features/gve.ini | 2 + doc/guides/nics/gve.rst | 28 +++- drivers/net/gve/base/gve.h | 15 ++ drivers/net/gve/base/gve_adminq.c | 58 ++++++++ drivers/net/gve/base/gve_adminq.h | 21 +++ drivers/net/gve/gve_ethdev.c | 231 +++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 17 +++ drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++ drivers/net/gve/meson.build | 1 + 10 files changed, 677 insertions(+), 9 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v5 1/5 net/gve: expose RSS offload support in dev info
This patch communicates that the GVE driver supports RSS, along with the RSS offloads supported by the driver.
--- drivers/net/gve/gve_ethdev.c | 4 +++- drivers/net/gve/gve_ethdev.h | 8 ++++++++ 2 files changed, 11 insertions(+), 1 deletion(-)
drivers/net/gve/gve_ethdev.c +3 -1
@@ -405,7 +405,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_mtu = priv->max_mtu; dev_info->min_mtu =RTE_ETHER_MIN_MTU ; - dev_info->rx_offload_capa =0 ; + dev_info->rx_offload_capa =RTE_ETH_RX_OFFLOAD_RSS_HASH ; dev_info->tx_offload_capa =RTE_ETH_TX_OFFLOAD_MULTI_SEGS | RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
@@ -442,6 +442,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .nb_align = 1, }; + dev_info->flow_type_rss_offloads =GVE_RSS_OFFLOAD_ALL ; +return 0 ; }
drivers/net/gve/gve_ethdev.h +8 -0
@@ -33,6 +33,14 @@ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG) +#define GVE_RSS_OFFLOAD_ALL ( \ + RTE_ETH_RSS_IPV4 | \ + RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ + RTE_ETH_RSS_IPV6 | \ + RTE_ETH_RSS_IPV6_EX | \ + RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ + RTE_ETH_RSS_IPV6_TCP_EX) +/* A list of pages registered with the device during setup and used by a queue * as buffers */
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v5 2/5 net/gve: add RSS adminq command
In reply to Joshua WashingtonThis change introduces admin queue changes that enable the configuration of RSS parameters for the GVE driver.
--- drivers/net/gve/base/gve.h | 15 ++++++++ drivers/net/gve/base/gve_adminq.c | 58 +++++++++++++++++++++++++++++++ drivers/net/gve/base/gve_adminq.h | 29 ++++++++++++++++ drivers/net/gve/gve_ethdev.h | 6 +++- 4 files changed, 107 insertions(+), 1 deletion(-)
drivers/net/gve/base/gve.h +15 -0
@@ -51,4 +51,19 @@ enum gve_state_flags_bit { GVE_PRIV_FLAGS_NAPI_ENABLED = 4, }; +enum gve_rss_hash_algorithm { +GVE_RSS_HASH_UNDEFINED =0 , +GVE_RSS_HASH_TOEPLITZ =1 , +}; + +struct gve_rss_config { +uint16_t hash_types; +enum gve_rss_hash_algorithm alg; +uint16_t key_size; +uint16_t indir_size; +uint8_t *key; +uint32_t *indir; +}; + +#endif /* _GVE_H_ */
drivers/net/gve/base/gve_adminq.c +58 -0
67 lines not on this page · raw message
drivers/net/gve/base/gve_adminq.h +29 -0
@@ -19,6 +19,7 @@ enum gve_adminq_opcodes {GVE_ADMINQ_DESTROY_TX_QUEUE =0x7 ,GVE_ADMINQ_DESTROY_RX_QUEUE =0x8 ,GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES =0x9 , +GVE_ADMINQ_CONFIGURE_RSS =0xA ,GVE_ADMINQ_SET_DRIVER_PARAMETER =0xB ,GVE_ADMINQ_REPORT_STATS =0xC , GVE_ADMINQ_REPORT_LINK_SPEED = 0xD,
@@ -377,6 +378,27 @@ struct gve_adminq_get_ptype_map {__be64 ptype_map_addr; }; +#define GVE_RSS_HASH_IPV4 BIT(0) +#define GVE_RSS_HASH_TCPV4 BIT(1) +#define GVE_RSS_HASH_IPV6 BIT(2) +#define GVE_RSS_HASH_IPV6_EX BIT(3) +#define GVE_RSS_HASH_TCPV6 BIT(4) +#define GVE_RSS_HASH_TCPV6_EX BIT(5) +#define GVE_RSS_HASH_UDPV4 BIT(6) +#define GVE_RSS_HASH_UDPV6 BIT(7) +#define GVE_RSS_HASH_UDPV6_EX BIT(8) + +/* RSS configuration command */ +struct gve_adminq_configure_rss { +__be16 hash_types; +u8 halg;/* hash algorithm */ +u8 reserved; +__be16 hkey_len; +__be16 indir_len; +__be64 hkey_addr; +__be64 indir_addr; +}; + union gve_adminq_command { struct { __be32 opcode;
@@ -391,6 +413,7 @@ union gve_adminq_command {struct gve_adminq_describe_device describe_device;struct gve_adminq_register_page_list reg_page_list;struct gve_adminq_unregister_page_list unreg_page_list; +struct gve_adminq_configure_rss configure_rss;struct gve_adminq_set_driver_parameter set_driver_param;struct gve_adminq_report_stats report_stats;struct gve_adminq_report_link_speed report_link_speed;
@@ -404,6 +427,8 @@ union gve_adminq_command {GVE_CHECK_UNION_LEN (64 , gve_adminq_command); +struct gve_priv ; +struct gve_queue_page_list ;int gve_adminq_alloc (struct gve_priv *priv);void gve_adminq_free (struct gve_priv *priv);void gve_adminq_release (struct gve_priv *priv);
@@ -433,4 +458,8 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,int gve_adminq_verify_driver_compatibility (struct gve_priv *priv,u64 driver_info_len,dma_addr_t driver_info_addr); + +int gve_adminq_configure_rss (struct gve_priv *priv, +struct gve_rss_config *rss_config); +#endif /* _GVE_ADMINQ_H */
drivers/net/gve/gve_ethdev.h +5 -1
@@ -39,7 +39,10 @@RTE_ETH_RSS_IPV6 | \RTE_ETH_RSS_IPV6_EX | \RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ - RTE_ETH_RSS_IPV6_TCP_EX) +RTE_ETH_RSS_IPV6_TCP_EX | \ +RTE_ETH_RSS_NONFRAG_IPV4_UDP | \ +RTE_ETH_RSS_NONFRAG_IPV6_UDP | \ + RTE_ETH_RSS_IPV6_UDP_EX)/* A list of pages registered with the device during setup and used by a queue * as buffers
@@ -264,6 +267,7 @@ struct gve_priv {uint32_t adminq_destroy_tx_queue_cnt;uint32_t adminq_destroy_rx_queue_cnt;uint32_t adminq_dcfg_device_resources_cnt; +uint32_t adminq_cfg_rss_cnt;uint32_t adminq_set_driver_parameter_cnt;uint32_t adminq_report_stats_cnt;uint32_t adminq_report_link_speed_cnt;
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v5 3/5 net/gve: add gve RSS library
In reply to Joshua WashingtonThis change includes a number of helper functions to facilitate RSS configuration on the GVE DPDK driver. These methods are declared in gve_rss.h.
--- drivers/net/gve/base/gve_adminq.h | 10 +- drivers/net/gve/gve_ethdev.c | 2 +- drivers/net/gve/gve_ethdev.h | 4 +- drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++++ drivers/net/gve/meson.build | 1 + 6 files changed, 319 insertions(+), 11 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
drivers/net/gve/base/gve_adminq.h +1 -9
@@ -378,15 +378,6 @@ struct gve_adminq_get_ptype_map {__be64 ptype_map_addr; }; -#define GVE_RSS_HASH_IPV4 BIT(0) -#define GVE_RSS_HASH_TCPV4 BIT(1) -#define GVE_RSS_HASH_IPV6 BIT(2) -#define GVE_RSS_HASH_IPV6_EX BIT(3) -#define GVE_RSS_HASH_TCPV6 BIT(4) -#define GVE_RSS_HASH_TCPV6_EX BIT(5) -#define GVE_RSS_HASH_UDPV4 BIT(6) -#define GVE_RSS_HASH_UDPV6 BIT(7) -#define GVE_RSS_HASH_UDPV6_EX BIT(8)/* RSS configuration command */ struct gve_adminq_configure_rss {
@@ -428,6 +419,7 @@ union gve_adminq_command {GVE_CHECK_UNION_LEN (64 , gve_adminq_command);struct gve_priv ; +struct gve_rss_config ;struct gve_queue_page_list ;int gve_adminq_alloc (struct gve_priv *priv);void gve_adminq_free (struct gve_priv *priv);
drivers/net/gve/gve_ethdev.c +1 -1
@@ -442,7 +442,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .nb_align = 1, }; - dev_info->flow_type_rss_offloads =GVE_RSS_OFFLOAD_ALL ; + dev_info->flow_type_rss_offloads =GVE_RTE_RSS_OFFLOAD_ALL ;return 0 ; }
drivers/net/gve/gve_ethdev.h +3 -1
@@ -33,7 +33,7 @@ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG) -#define GVE_RSS_OFFLOAD_ALL ( \ +#define GVE_RTE_RSS_OFFLOAD_ALL ( \ RTE_ETH_RSS_IPV4 | \ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ RTE_ETH_RSS_IPV6 | \
@@ -290,6 +290,8 @@ struct gve_priv {const struct rte_memzone *stats_report_mem;uint16_t stats_start_idx;/* start index of array of stats written by NIC */ uint16_t stats_end_idx;/* end index of array of stats written by NIC */ + +struct gve_rss_config rss_config; }; static inline bool
drivers/net/gve/gve_rss.c new file +206 -0
206 lines not on this page · raw message
drivers/net/gve/gve_rss.h new file +107 -0
107 lines not on this page · raw message
drivers/net/gve/meson.build +1 -0
@@ -15,5 +15,6 @@ sources = files( 'gve_tx_dqo.c', 'gve_ethdev.c', 'gve_version.c', + 'gve_rss.c', ) includes += include_directories('base')
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v5 4/5 net/gve: add RSS configuration update support
In reply to Joshua WashingtonThis patch adds support for updating the RSS hash key and hash fields in the GVE PMD through the implementation of rss_hash_update and rss_hash_conf_get.
The RSS hash key for gVNIC is required to be 40 bytes. On initial configuration of the RSS hash key, the RSS redirection table will be set to a static default, using a round-robin approach for all queues. Note, however, that this patch does not include support for setting the redirection table explicitly. In dev_configure, if the static redirection table has been set, it will be updated to reflect the new queue count, if it has changed.
The RSS key must be set before any other RSS configuration can happen. As such, an attempt to set the hash types before the key is configured will fail.
--- doc/guides/nics/features/gve.ini | 1 + doc/guides/nics/gve.rst | 20 +++-- drivers/net/gve/gve_ethdev.c | 132 ++++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 3 + 4 files changed, 148 insertions(+), 8 deletions(-)
doc/guides/nics/features/gve.ini +1 -0
@@ -9,6 +9,7 @@ Link status = Y MTU update = Y TSO = Y RSS hash = Y +RSS key update = Y L4 checksum offload = Y Basic stats = Y Linux = Y
doc/guides/nics/gve.rst +14 -6
@@ -70,6 +70,7 @@ Supported features of the GVE PMD are: - Link state information - Tx multi-segments (Scatter Tx) - Tx UDP/TCP/SCTP Checksum +- RSS hash configuration Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD. Jumbo Frame is not supported in PMD for now.
@@ -77,10 +78,17 @@ It'll be added in a future DPDK release. Also, only GQI_QPL queue format is in use on GCP since GQI_RDA hasn't been released in production. -Currently, setting MTU with value larger than 1460 is not supported. +RSS +^^^ + +GVE RSS can be enabled and configured using the standard interfaces. The driver +does not support querying the initial RSS configuration. + +The RSS hash key must be exactly 40 bytes. Upon RSS hash configuration, a +default redirection table will be set using a round-robin assignment of hash +values to queues. The default GVE redirection table has 128 entries. + +Note that the initial configuration requires a hash key to be provided if one +had not been provided before. Attempting to set hash types alone without the +existence of a set key will result in a failed request. -Currently, only "RSS hash" is force enabled -so that the backend hardware device calculated hash values -could be shared with applications. -But for RSS, there is no such API to config RSS hash function or RETA table. -So, limited RSS is supported only with default config/setting.
drivers/net/gve/gve_ethdev.c +130 -2
177 lines not on this page · raw message
drivers/net/gve/gve_ethdev.h +3 -0
@@ -29,6 +29,9 @@#define GVE_RX_MIN_BUF_SIZE_GQI 2048#define GVE_RX_MAX_BUF_SIZE_GQI 4096 +#define GVE_RSS_HASH_KEY_SIZE 40 +#define GVE_RSS_INDIR_SIZE 128 +#define GVE_TX_CKSUM_OFFLOAD_MASK ( \ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG)
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v5 5/5 net/gve: add RSS redirection table update support
In reply to Joshua WashingtonThis patch introduces support for updating the RSS redirection table in the GVE PMD through the implementation of rss_reta_update and rss_reta_query.
Due to an infrastructure limitation, the RSS hash key must be manually configured before the redirection table can be updated or queried. The redirection table is expected to be exactly 128 bytes.
--- doc/guides/nics/features/gve.ini | 1 + doc/guides/nics/gve.rst | 8 +++ drivers/net/gve/gve_ethdev.c | 95 ++++++++++++++++++++++++++++++++ 3 files changed, 104 insertions(+)
doc/guides/nics/features/gve.ini +1 -0
@@ -10,6 +10,7 @@ MTU update = Y TSO = Y RSS hash = Y RSS key update = Y +RSS reta update = Y L4 checksum offload = Y Basic stats = Y Linux = Y
doc/guides/nics/gve.rst +8 -0
@@ -71,6 +71,7 @@ Supported features of the GVE PMD are: - Tx multi-segments (Scatter Tx) - Tx UDP/TCP/SCTP Checksum - RSS hash configuration +- RSS redirection table query and update Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD. Jumbo Frame is not supported in PMD for now.
@@ -92,3 +93,10 @@ Note that the initial configuration requires a hash key to be provided if one had not been provided before. Attempting to set hash types alone without the existence of a set key will result in a failed request. +As stated above, the RSS redirection table has exactly 128 entries. The RSS hash +must be configured before the redirection table can be updated using the standard +interface. Because the initial RSS hash creates a default redirection table, the +redirection table will be available for querying upon initial hash configuration. +When performing redirection table updates, it is possible to update individual +table entries. +
drivers/net/gve/gve_ethdev.c +95 -0
113 lines not on this page · raw message
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
Quoted 56 lines · Joshua Washington
This patch series introduces RSS support for the GVE poll-mode driver. This series includes implementations of the following
eth_dev_ops:1)
rss_hash_update
2)rss_hash_conf_get
3)reta_query
4)reta_updateIn
rss_hash_update, the GVE driver supports the following RSS hash types:*
RTE_ETH_RSS_IPV4
*RTE_ETH_RSS_NONFRAG_IPV4_TCP
*RTE_ETH_RSS_NONFRAG_IPV4_UDP
*RTE_ETH_RSS_IPV6
*RTE_ETH_RSS_IPV6_EX
*RTE_ETH_RSS_NONFRAG_IPV6_TCP
*RTE_ETH_RSS_NONFRAG_IPV6_UDP
*RTE_ETH_RSS_IPV6_TCP_EX
*RTE_ETH_RSS_IPV6_UDP_EXThe hash key is 40B, and the lookup table has 128 entries. These values are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set with a key before the redriection table is set up. When the RSS hash is configured, a default redirection table is generated based on the number of queues. When the device is re-configured, the redirection table is reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for RSS redirection table entries, while the RTE API uses 16 bit integers. However, this is unlikely to be an issue, as these values represent receive queues, and the gVNIC device does not support anywhere near 64K queues.
This series also updates the corresponding feature matrix ertries and documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches fixes.Note: There is a warning about complex macros being parenthesized that does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.v4:
Fix formatting ingve_adminq.cv5:
Update commit messages to comply with check-git-log.
Fold feature matrix and documentation updates into relevant commits.
Quoted 6 lines
Joshua Washington (5):
net/gve: expose RSS offload support in dev info net/gve: add RSS adminq command
net/gve: add gve RSS library
net/gve: add RSS configuration update support net/gve: add RSS redirection table update support
Series applied to dpdk-next-net/main, thanks.
PATCH v4 2/7 net/gve: RSS adminq command changes
In reply to a message not in the archiveThis change introduces admin queue changes that enable the configuration of RSS parameters for the GVE driver.
--- drivers/net/gve/base/gve.h | 15 ++++++++ drivers/net/gve/base/gve_adminq.c | 58 +++++++++++++++++++++++++++++++ drivers/net/gve/base/gve_adminq.h | 29 ++++++++++++++++ drivers/net/gve/gve_ethdev.h | 6 +++- 4 files changed, 107 insertions(+), 1 deletion(-)
drivers/net/gve/base/gve.h +15 -0
@@ -51,4 +51,19 @@ enum gve_state_flags_bit { GVE_PRIV_FLAGS_NAPI_ENABLED = 4, }; +enum gve_rss_hash_algorithm { +GVE_RSS_HASH_UNDEFINED =0 , +GVE_RSS_HASH_TOEPLITZ =1 , +}; + +struct gve_rss_config { +uint16_t hash_types; +enum gve_rss_hash_algorithm alg; +uint16_t key_size; +uint16_t indir_size; +uint8_t *key; +uint32_t *indir; +}; + +#endif /* _GVE_H_ */
drivers/net/gve/base/gve_adminq.c +58 -0
67 lines not on this page · raw message
drivers/net/gve/base/gve_adminq.h +29 -0
@@ -19,6 +19,7 @@ enum gve_adminq_opcodes {GVE_ADMINQ_DESTROY_TX_QUEUE =0x7 ,GVE_ADMINQ_DESTROY_RX_QUEUE =0x8 ,GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES =0x9 , +GVE_ADMINQ_CONFIGURE_RSS =0xA ,GVE_ADMINQ_SET_DRIVER_PARAMETER =0xB ,GVE_ADMINQ_REPORT_STATS =0xC , GVE_ADMINQ_REPORT_LINK_SPEED = 0xD,
@@ -377,6 +378,27 @@ struct gve_adminq_get_ptype_map {__be64 ptype_map_addr; }; +#define GVE_RSS_HASH_IPV4 BIT(0) +#define GVE_RSS_HASH_TCPV4 BIT(1) +#define GVE_RSS_HASH_IPV6 BIT(2) +#define GVE_RSS_HASH_IPV6_EX BIT(3) +#define GVE_RSS_HASH_TCPV6 BIT(4) +#define GVE_RSS_HASH_TCPV6_EX BIT(5) +#define GVE_RSS_HASH_UDPV4 BIT(6) +#define GVE_RSS_HASH_UDPV6 BIT(7) +#define GVE_RSS_HASH_UDPV6_EX BIT(8) + +/* RSS configuration command */ +struct gve_adminq_configure_rss { +__be16 hash_types; +u8 halg;/* hash algorithm */ +u8 reserved; +__be16 hkey_len; +__be16 indir_len; +__be64 hkey_addr; +__be64 indir_addr; +}; + union gve_adminq_command { struct { __be32 opcode;
@@ -391,6 +413,7 @@ union gve_adminq_command {struct gve_adminq_describe_device describe_device;struct gve_adminq_register_page_list reg_page_list;struct gve_adminq_unregister_page_list unreg_page_list; +struct gve_adminq_configure_rss configure_rss;struct gve_adminq_set_driver_parameter set_driver_param;struct gve_adminq_report_stats report_stats;struct gve_adminq_report_link_speed report_link_speed;
@@ -404,6 +427,8 @@ union gve_adminq_command {GVE_CHECK_UNION_LEN (64 , gve_adminq_command); +struct gve_priv ; +struct gve_queue_page_list ;int gve_adminq_alloc (struct gve_priv *priv);void gve_adminq_free (struct gve_priv *priv);void gve_adminq_release (struct gve_priv *priv);
@@ -433,4 +458,8 @@ int gve_adminq_get_ptype_map_dqo(struct gve_priv *priv,int gve_adminq_verify_driver_compatibility (struct gve_priv *priv,u64 driver_info_len,dma_addr_t driver_info_addr); + +int gve_adminq_configure_rss (struct gve_priv *priv, +struct gve_rss_config *rss_config); +#endif /* _GVE_ADMINQ_H */
drivers/net/gve/gve_ethdev.h +5 -1
@@ -39,7 +39,10 @@RTE_ETH_RSS_IPV6 | \RTE_ETH_RSS_IPV6_EX | \RTE_ETH_RSS_NONFRAG_IPV6_TCP | \ - RTE_ETH_RSS_IPV6_TCP_EX) +RTE_ETH_RSS_IPV6_TCP_EX | \ +RTE_ETH_RSS_NONFRAG_IPV4_UDP | \ +RTE_ETH_RSS_NONFRAG_IPV6_UDP | \ + RTE_ETH_RSS_IPV6_UDP_EX)/* A list of pages registered with the device during setup and used by a queue * as buffers
@@ -264,6 +267,7 @@ struct gve_priv {uint32_t adminq_destroy_tx_queue_cnt;uint32_t adminq_destroy_rx_queue_cnt;uint32_t adminq_dcfg_device_resources_cnt; +uint32_t adminq_cfg_rss_cnt;uint32_t adminq_set_driver_parameter_cnt;uint32_t adminq_report_stats_cnt;uint32_t adminq_report_link_speed_cnt;
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v4 3/7 net/gve: add gve_rss library for handling RSS-related behaviors
In reply to a message not in the archiveThis change includes a number of helper functions to facilitate RSS configuration on the GVE DPDK driver. These methods are declared in gve_rss.h.
--- drivers/net/gve/base/gve_adminq.h | 10 +- drivers/net/gve/gve_ethdev.c | 2 +- drivers/net/gve/gve_ethdev.h | 4 +- drivers/net/gve/gve_rss.c | 206 ++++++++++++++++++++++++++++++ drivers/net/gve/gve_rss.h | 107 ++++++++++++++++ drivers/net/gve/meson.build | 1 + 6 files changed, 319 insertions(+), 11 deletions(-) create mode 100644 drivers/net/gve/gve_rss.c create mode 100644 drivers/net/gve/gve_rss.h
drivers/net/gve/base/gve_adminq.h +1 -9
@@ -378,15 +378,6 @@ struct gve_adminq_get_ptype_map {__be64 ptype_map_addr; }; -#define GVE_RSS_HASH_IPV4 BIT(0) -#define GVE_RSS_HASH_TCPV4 BIT(1) -#define GVE_RSS_HASH_IPV6 BIT(2) -#define GVE_RSS_HASH_IPV6_EX BIT(3) -#define GVE_RSS_HASH_TCPV6 BIT(4) -#define GVE_RSS_HASH_TCPV6_EX BIT(5) -#define GVE_RSS_HASH_UDPV4 BIT(6) -#define GVE_RSS_HASH_UDPV6 BIT(7) -#define GVE_RSS_HASH_UDPV6_EX BIT(8)/* RSS configuration command */ struct gve_adminq_configure_rss {
@@ -428,6 +419,7 @@ union gve_adminq_command {GVE_CHECK_UNION_LEN (64 , gve_adminq_command);struct gve_priv ; +struct gve_rss_config ;struct gve_queue_page_list ;int gve_adminq_alloc (struct gve_priv *priv);void gve_adminq_free (struct gve_priv *priv);
drivers/net/gve/gve_ethdev.c +1 -1
@@ -442,7 +442,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) .nb_align = 1, }; - dev_info->flow_type_rss_offloads =GVE_RSS_OFFLOAD_ALL ; + dev_info->flow_type_rss_offloads =GVE_RTE_RSS_OFFLOAD_ALL ;return 0 ; }
drivers/net/gve/gve_ethdev.h +3 -1
@@ -33,7 +33,7 @@ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG) -#define GVE_RSS_OFFLOAD_ALL ( \ +#define GVE_RTE_RSS_OFFLOAD_ALL ( \ RTE_ETH_RSS_IPV4 | \ RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ RTE_ETH_RSS_IPV6 | \
@@ -290,6 +290,8 @@ struct gve_priv {const struct rte_memzone *stats_report_mem;uint16_t stats_start_idx;/* start index of array of stats written by NIC */ uint16_t stats_end_idx;/* end index of array of stats written by NIC */ + +struct gve_rss_config rss_config; }; static inline bool
drivers/net/gve/gve_rss.c new file +206 -0
206 lines not on this page · raw message
drivers/net/gve/gve_rss.h new file +107 -0
107 lines not on this page · raw message
drivers/net/gve/meson.build +1 -0
@@ -15,5 +15,6 @@ sources = files( 'gve_tx_dqo.c', 'gve_ethdev.c', 'gve_version.c', + 'gve_rss.c', ) includes += include_directories('base')
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v4 4/7 net/gve: RSS configuration update support
In reply to a message not in the archiveThis patch adds support for updating the RSS hash key and hash fields in the GVE PMD through the implementation of rss_hash_update and rss_hash_conf_get.
The RSS hash key for gVNIC is required to be 40 bytes. On initial configuration of the RSS hash key, the RSS redirection table will be set to a static default, using a round-robin approach for all queues. Note, however, that this patch does not include support for setting the redirection table explicitly. In dev_configure, if the static redirection table has been set, it will be updated to reflect the new queue count, if it has changed.
The RSS key must be set before any other RSS configuration can happen. As such, an attempt to set the hash types before the key is configured will fail.
--- drivers/net/gve/gve_ethdev.c | 132 ++++++++++++++++++++++++++++++++++- drivers/net/gve/gve_ethdev.h | 3 + 2 files changed, 133 insertions(+), 2 deletions(-)
drivers/net/gve/gve_ethdev.c +130 -2
@@ -1,5 +1,6 @@/* SPDX-License-Identifier: BSD-3-Clause -* Copyright(C) 2022 Intel Corporation +* Copyright(C) 2022-2023 Intel Corporation +* Copyright(C) 2023 Google LLC */ #include "gve_ethdev.h"
@@ -8,6 +9,7 @@#include "base/gve_osdep.h" #include "gve_version.h" #include "rte_ether.h" +#include "gve_rss.h" static void gve_write_version (uint8_t *driver_version_register)
@@ -88,12 +90,31 @@ gve_dev_configure(struct rte_eth_dev *dev) {struct gve_priv *priv = dev->data->dev_private; -if (dev->data->dev_conf.rxmode.mq_mode &RTE_ETH_MQ_RX_RSS_FLAG ) +if (dev->data->dev_conf.rxmode.mq_mode &RTE_ETH_MQ_RX_RSS_FLAG ) { dev->data->dev_conf.rxmode.offloads |=RTE_ETH_RX_OFFLOAD_RSS_HASH ; + priv->rss_config.alg =GVE_RSS_HASH_TOEPLITZ ; + }if (dev->data->dev_conf.rxmode.offloads &RTE_ETH_RX_OFFLOAD_TCP_LRO ) priv->enable_rsc =1 ; +/* Reset RSS RETA in case number of queues changed. */ +if (priv->rss_config.indir) { +struct gve_rss_config update_reta_config; +gve_init_rss_config_from_priv (priv, &update_reta_config); +gve_generate_rss_reta (dev, &update_reta_config); + +int err =gve_adminq_configure_rss (priv, &update_reta_config); +if (err) +PMD_DRV_LOG (ERR , +"Could not reconfigure RSS redirection table." ); +else +gve_update_priv_rss_config (priv, &update_reta_config); + +gve_free_rss_config (&update_reta_config); +return err; + } +return 0 ; }
@@ -443,6 +464,8 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) }; dev_info->flow_type_rss_offloads =GVE_RTE_RSS_OFFLOAD_ALL ; + dev_info->hash_key_size =GVE_RSS_HASH_KEY_SIZE ; + dev_info->reta_size =GVE_RSS_INDIR_SIZE ;return 0 ; }
@@ -646,6 +669,107 @@ gve_xstats_get_names(struct rte_eth_dev *dev,return count; } + +static int +gve_rss_hash_update (struct rte_eth_dev *dev, +struct rte_eth_rss_conf *rss_conf) +{ +struct gve_priv *priv = dev->data->dev_private; +struct gve_rss_config gve_rss_conf; +int rss_reta_size; +int err; + +if (gve_validate_rss_hf (rss_conf->rss_hf)) { +PMD_DRV_LOG (ERR ,"Unsupported hash function." ); +return -EINVAL ; + } + +if (rss_conf->algorithm !=RTE_ETH_HASH_FUNCTION_TOEPLITZ && + rss_conf->algorithm !=RTE_ETH_HASH_FUNCTION_DEFAULT ) { +PMD_DRV_LOG (ERR ,"Device only supports Toeplitz algorithm." ); +return -EINVAL ; + } + +if (rss_conf->rss_key_len) { +if (rss_conf->rss_key_len !=GVE_RSS_HASH_KEY_SIZE ) { +PMD_DRV_LOG (ERR , +"Invalid hash key size. Only RSS hash key size " +"of %u supported" ,GVE_RSS_HASH_KEY_SIZE ); +return -EINVAL ; + } + +if (!rss_conf->rss_key) { +PMD_DRV_LOG (ERR ,"RSS key must be non-null." ); +return -EINVAL ; + } + }else { +if (!priv->rss_config.key_size) { +PMD_DRV_LOG (ERR ,"RSS key must be initialized before " +"any other configuration." ); +return -EINVAL ; + } + rss_conf->rss_key_len = priv->rss_config.key_size; + } + + rss_reta_size = priv->rss_config.indir ? + priv->rss_config.indir_size : +GVE_RSS_INDIR_SIZE ; + err =gve_init_rss_config (&gve_rss_conf, rss_conf->rss_key_len, + rss_reta_size); +if (err) +return err; + + gve_rss_conf.alg =GVE_RSS_HASH_TOEPLITZ ; + err =gve_update_rss_hash_types (priv, &gve_rss_conf, rss_conf); +if (err) + goto err; + err =gve_update_rss_key (priv, &gve_rss_conf, rss_conf); +if (err) + goto err; + +/* Set redirection table to default or preexisting. */ +if (!priv->rss_config.indir) +gve_generate_rss_reta (dev, &gve_rss_conf); +else +memcpy (gve_rss_conf.indir, priv->rss_config.indir, + gve_rss_conf.indir_size *sizeof (*priv->rss_config.indir)); + + err =gve_adminq_configure_rss (priv, &gve_rss_conf); +if (!err) +gve_update_priv_rss_config (priv, &gve_rss_conf); + +err: +gve_free_rss_config (&gve_rss_conf); +return err; +} + +static int +gve_rss_hash_conf_get (struct rte_eth_dev *dev, +struct rte_eth_rss_conf *rss_conf) +{ +struct gve_priv *priv = dev->data->dev_private; + +if (!(dev->data->dev_conf.rxmode.offloads & +RTE_ETH_RX_OFFLOAD_RSS_HASH )) { +PMD_DRV_LOG (ERR ,"RSS not configured." ); +return -ENOTSUP ; + } + + +gve_to_rte_rss_hf (priv->rss_config.hash_types, rss_conf); + rss_conf->rss_key_len = priv->rss_config.key_size; +if (rss_conf->rss_key) { +if (!priv->rss_config.key) { +PMD_DRV_LOG (ERR ,"Unable to retrieve default RSS hash key." ); +return -ENOTSUP ; + } +memcpy (rss_conf->rss_key, priv->rss_config.key, + rss_conf->rss_key_len *sizeof (*rss_conf->rss_key)); + } + +return 0 ; +} +static const struct eth_dev_ops gve_eth_dev_ops = { .dev_configure = gve_dev_configure, .dev_start = gve_dev_start,
@@ -666,6 +790,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .mtu_set = gve_dev_mtu_set, .xstats_get = gve_xstats_get, .xstats_get_names = gve_xstats_get_names, + .rss_hash_update = gve_rss_hash_update, + .rss_hash_conf_get = gve_rss_hash_conf_get, };static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -688,6 +814,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = { .mtu_set = gve_dev_mtu_set, .xstats_get = gve_xstats_get, .xstats_get_names = gve_xstats_get_names, + .rss_hash_update = gve_rss_hash_update, + .rss_hash_conf_get = gve_rss_hash_conf_get, }; static void
drivers/net/gve/gve_ethdev.h +3 -0
@@ -29,6 +29,9 @@#define GVE_RX_MIN_BUF_SIZE_GQI 2048#define GVE_RX_MAX_BUF_SIZE_GQI 4096 +#define GVE_RSS_HASH_KEY_SIZE 40 +#define GVE_RSS_INDIR_SIZE 128 +#define GVE_TX_CKSUM_OFFLOAD_MASK ( \ RTE_MBUF_F_TX_L4_MASK | \ RTE_MBUF_F_TX_TCP_SEG)
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v4 5/7 net/gve: RSS redirection table update support
In reply to a message not in the archiveThis patch introduces support for updating the RSS redirection table in the GVE PMD through the implementation of rss_reta_update and rss_reta_query.
Due to an infrastructure limitation, the RSS hash key must be manually configured before the redirection table can be updated or queried. The redirection table is expected to be exactly 128 bytes.
--- drivers/net/gve/gve_ethdev.c | 95 ++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+)
drivers/net/gve/gve_ethdev.c +95 -0
@@ -770,6 +770,97 @@ gve_rss_hash_conf_get(struct rte_eth_dev *dev,return 0 ; } +static int +gve_rss_reta_update (struct rte_eth_dev *dev, +struct rte_eth_rss_reta_entry64 *reta_conf,uint16_t reta_size) +{ +struct gve_priv *priv = dev->data->dev_private; +struct gve_rss_config gve_rss_conf; +int table_id; +int err; +int i; + +/* RSS key must be set before the redirection table can be set. */ +if (!priv->rss_config.key || priv->rss_config.key_size ==0 ) { +PMD_DRV_LOG (ERR ,"RSS hash key msut be set before the " +"redirection table can be updated." ); +return -ENOTSUP ; + } + +if (reta_size !=GVE_RSS_INDIR_SIZE ) { +PMD_DRV_LOG (ERR ,"Redirection table must have %hu elements" , + (uint16_t )GVE_RSS_INDIR_SIZE ); +return -EINVAL ; + } + + err =gve_init_rss_config_from_priv (priv, &gve_rss_conf); +if (err) { +PMD_DRV_LOG (ERR ,"Error allocating new RSS config." ); +return err; + } + + table_id =0 ; +for (i =0 ; i < priv->rss_config.indir_size; i++) { +int table_entry = i %RTE_ETH_RETA_GROUP_SIZE ; +if (reta_conf[table_id].mask & (1ULL << table_entry)) + gve_rss_conf.indir[i] = + reta_conf[table_id].reta[table_entry]; + +if (table_entry ==RTE_ETH_RETA_GROUP_SIZE -1 ) + table_id++; + } + + err =gve_adminq_configure_rss (priv, &gve_rss_conf); +if (err) +PMD_DRV_LOG (ERR ,"Problem configuring RSS with device." ); +else +gve_update_priv_rss_config (priv, &gve_rss_conf); + +gve_free_rss_config (&gve_rss_conf); +return err; +} + +static int +gve_rss_reta_query (struct rte_eth_dev *dev, +struct rte_eth_rss_reta_entry64 *reta_conf,uint16_t reta_size) +{ +struct gve_priv *priv = dev->data->dev_private; +int table_id; +int i; + +if (!(dev->data->dev_conf.rxmode.offloads & +RTE_ETH_RX_OFFLOAD_RSS_HASH )) { +PMD_DRV_LOG (ERR ,"RSS not configured." ); +return -ENOTSUP ; + } + +/* RSS key must be set before the redirection table can be queried. */ +if (!priv->rss_config.key) { +PMD_DRV_LOG (ERR ,"RSS hash key must be set before the " +"redirection table can be initialized." ); +return -ENOTSUP ; + } + +if (reta_size != priv->rss_config.indir_size) { +PMD_DRV_LOG (ERR ,"RSS redirection table must have %d entries." , + priv->rss_config.indir_size); +return -EINVAL ; + } + + table_id =0 ; +for (i =0 ; i < priv->rss_config.indir_size; i++) { +int table_entry = i %RTE_ETH_RETA_GROUP_SIZE ; +if (reta_conf[table_id].mask & (1ULL << table_entry)) + reta_conf[table_id].reta[table_entry] = + priv->rss_config.indir[i]; + +if (table_entry ==RTE_ETH_RETA_GROUP_SIZE -1 ) + table_id++; + } + +return 0 ; +} +static const struct eth_dev_ops gve_eth_dev_ops = { .dev_configure = gve_dev_configure, .dev_start = gve_dev_start,
@@ -792,6 +883,8 @@ static const struct eth_dev_ops gve_eth_dev_ops = { .xstats_get_names = gve_xstats_get_names, .rss_hash_update = gve_rss_hash_update, .rss_hash_conf_get = gve_rss_hash_conf_get, + .reta_update = gve_rss_reta_update, + .reta_query = gve_rss_reta_query, };static const struct eth_dev_ops gve_eth_dev_ops_dqo = {
@@ -816,6 +909,8 @@ static const struct eth_dev_ops gve_eth_dev_ops_dqo = { .xstats_get_names = gve_xstats_get_names, .rss_hash_update = gve_rss_hash_update, .rss_hash_conf_get = gve_rss_hash_conf_get, + .reta_update = gve_rss_reta_update, + .reta_query = gve_rss_reta_query, }; static void
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
PATCH v4 6/7 net/gve: update gve.ini with RSS capabilities
In reply to a message not in the archiveThis patch updates the DPDK feature matrix to expose that the GVE driver supports RSS hash, RSS key update, and RSS reta update.
--- doc/guides/nics/features/gve.ini | 3 +++ 1 file changed, 3 insertions(+)
doc/guides/nics/features/gve.ini +3 -0
@@ -15,3 +15,6 @@ Linux = Y x86-32 = Y x86-64 = Y Usage doc = Y +RSS hash = Y +RSS key update = Y +RSS reta update = Y
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
Re: [PATCH v4 6/7] net/gve: update gve.ini with RSS capabilities
Quoted 13 lines · Joshua Washington
This patch updates the DPDK feature matrix to expose that the GVE driver supports RSS hash, RSS key update, and RSS reta update.
Signed-off-by Joshua Washington <[email protected]>--- doc/guides/nics/features/gve.ini | 3 +++ 1 file changed, 3 insertions(+)doc/guides/nics/features/gve.ini +0
-0
doc/guides/nics/features/gve.ini Linux = Y · line 20
Quoted 6 lines
doc/guides/nics/features/gve.ini +3
-0@@ -15,3 +15,6 @@ Linux = Y x86-32 = Y x86-64 = Y Usage doc = Y +RSS hash = Y +RSS key update = Y +RSS reta update = Y
Please distribute this patch to other patches, each update can go to the patch that adds the support.
Please keep the order of capabilities above same as order in default.ini
Re: [PATCH v4 6/7] net/gve: update gve.ini with RSS capabilities
Quoted 24 lines · Ferruh Yigit
Quoted 13 lines · Joshua Washington
This patch updates the DPDK feature matrix to expose that the GVE driver supports RSS hash, RSS key update, and RSS reta update.
Signed-off-by Joshua Washington <[email protected]>--- doc/guides/nics/features/gve.ini | 3 +++ 1 file changed, 3 insertions(+)doc/guides/nics/features/gve.ini +0
-0doc/guides/nics/features/gve.ini Linux = Y · line 20
Quoted 6 lines
doc/guides/nics/features/gve.ini +3
-0@@ -15,3 +15,6 @@ Linux = Y x86-32 = Y x86-64 = Y Usage doc = Y +RSS hash = Y +RSS key update = Y +RSS reta update = YPlease distribute this patch to other patches, each update can go to the patch that adds the support.
Quoted 2 lines
Please keep the order of capabilities above same as order in default.ini
btw, "RSS hash" seems duplicated.
PATCH v4 7/7 net/gve: update GVE documentation with RSS support
In reply to a message not in the archiveThis patch updates the GVE doc page to communicate that GVE now supports RSS configuration and explains the limitations.
--- doc/guides/nics/gve.rst | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
doc/guides/nics/gve.rst +10 -6
@@ -70,6 +70,8 @@ Supported features of the GVE PMD are: - Link state information - Tx multi-segments (Scatter Tx) - Tx UDP/TCP/SCTP Checksum +- RSS hash configuration +- RSS redirection table update and query Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD. Jumbo Frame is not supported in PMD for now.
@@ -77,10 +79,12 @@ It'll be added in a future DPDK release. Also, only GQI_QPL queue format is in use on GCP since GQI_RDA hasn't been released in production. -Currently, setting MTU with value larger than 1460 is not supported. +RSS +^^^ -Currently, only "RSS hash" is force enabled -so that the backend hardware device calculated hash values -could be shared with applications. -But for RSS, there is no such API to config RSS hash function or RETA table. -So, limited RSS is supported only with default config/setting. +GVE RSS can be enabled and configured using the standard interfaces. The driver +does not support querying the initial RSS configuration. + +The RSS hash key must be exactly 40 bytes, and the redirection table must have +128 entries. The RSS hash key must be configured before the redirection table +can be set up.
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
Re: [PATCH v4 7/7] net/gve: update GVE documentation with RSS support
Quoted 31 lines · Joshua Washington
This patch updates the GVE doc page to communicate that GVE now supports RSS configuration and explains the limitations.
Signed-off-by Joshua Washington <[email protected]>--- doc/guides/nics/gve.rst | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)doc/guides/nics/gve.rst +4
-2@@ -70,6 +70,8 @@ Supported features of the GVE PMD are: - Link state information - Tx multi-segments (Scatter Tx) - Tx UDP/TCP/SCTP Checksum +- RSS hash configuration +- RSS redirection table update and query Currently, only GQI_QPL and GQI_RDA queue format are supported in PMD. Jumbo Frame is not supported in PMD for now.@@ -77,10 +79,12 @@ It'll be added in a future DPDK release. Also, only GQI_QPL queue format is in use on GCP since GQI_RDA hasn't been released in production. -Currently, setting MTU with value larger than 1460 is not supported. +RSS +^^^ -Currently, only "RSS hash" is force enabled
doc/guides/nics/gve.rst It'll be added in a future DPDK release. · line 90
Quoted 10 lines
doc/guides/nics/gve.rst +6
-4-so that the backend hardware device calculated hash values -could be shared with applications. -But for RSS, there is no such API to config RSS hash function or RETA table. -So, limited RSS is supported only with default config/setting. +GVE RSS can be enabled and configured using the standard interfaces. The driver +does not support querying the initial RSS configuration. + +The RSS hash key must be exactly 40 bytes, and the redirection table must have +128 entries. The RSS hash key must be configured before the redirection table +can be set up.
Similarly can you please update documentation on each relevant patch, instead of having a separate documentation patch.
Re:
In reply to a message not in the archiveQuoted 54 lines · Joshua Washington
Subject: [PATCH v4 0/7] net/gve: RSS Support for GVE Driver
This patch series introduces RSS support for the GVE poll-mode driver. This series includes implementations of the following
eth_dev_ops:1)
rss_hash_update
2)rss_hash_conf_get
3)reta_query
4)reta_updateIn
rss_hash_update, the GVE driver supports the following RSS hash types:*
RTE_ETH_RSS_IPV4
*RTE_ETH_RSS_NONFRAG_IPV4_TCP
*RTE_ETH_RSS_NONFRAG_IPV4_UDP
*RTE_ETH_RSS_IPV6
*RTE_ETH_RSS_IPV6_EX
*RTE_ETH_RSS_NONFRAG_IPV6_TCP
*RTE_ETH_RSS_NONFRAG_IPV6_UDP
*RTE_ETH_RSS_IPV6_TCP_EX
*RTE_ETH_RSS_IPV6_UDP_EXThe hash key is 40B, and the lookup table has 128 entries. These values are not configurable in this implementation.
In general, the DPDK driver expects the RSS hash configuration to be set with a key before the redriection table is set up. When the RSS hash is configured, a default redirection table is generated based on the number of queues. When the device is re-configured, the redirection table is reset to the default value based on the queue count.
An important note is that the gVNIC device expects 32 bit integers for RSS redirection table entries, while the RTE API uses 16 bit integers. However, this is unlikely to be an issue, as these values represent receive queues, and the gVNIC device does not support anywhere near 64K queues.
This series also updates the corresponding feature matrix ertries and documentation as it pertains to RSS support in the GVE driver.
v2:
Add commmit messages for patches with it missing, and other checkpatches fixes.Note: There is a warning about complex macros being parenthesized that does not seem to be well-founded.
v3:
Fix build warnings that come up on certain distros.v4:
Fix formatting ingve_adminq.c
Quoted 10 lines
Joshua Washington (7):
net/gve: fully expose RSS offload support indev_infonet/gve: RSS adminq command changes
net/gve: addgve_rsslibrary for handling RSS-related behaviors net/gve: RSS configuration update support
net/gve: RSS redirection table update support net/gve: update gve.ini with RSS capabilities net/gve: update GVE documentation with RSS support
'./devtools/check-git-log.sh' script is giving warnings [1]:
Expected patch title format is:
net/gve: <verb> <object>
<verb> should start with lowercase.
[1]
- check-git-log:
Wrong headline format:
net/gve: fully expose RSS offload support in dev_info
net/gve: add gve_rss library for handling RSS-related behaviors
Wrong headline uppercase:
net/gve: RSS adminq command changes
net/gve: RSS configuration update support
net/gve: RSS redirection table update support
Headline too long:
net/gve: add gve_rss library for handling RSS-related behaviors