Introduce ethtool's set_channels
Subsystems (3)
v2 cover Introduce ethtool's set_channels
From: Sameeh Jubran <[email protected]>
Difference from v1:
* Dropped the print from patch 0002 - "net: ena: multiple queue creation related cleanups" as requested by David Miller
Sameeh Jubran (5):
net: ena: change num_queues to num_io_queues for clarity and
consistency
net: ena: multiple queue creation related cleanups
net: ena: make ethtool -l show correct max number of queues
net: ena: remove redundant print of number of queues
net: ena: ethtool: support set_channels callback
drivers/net/ethernet/amazon/ena/ena_ethtool.c | 37 ++-- drivers/net/ethernet/amazon/ena/ena_netdev.c | 160 ++++++++++-------- drivers/net/ethernet/amazon/ena/ena_netdev.h | 14 +- 3 files changed, 121 insertions(+), 90 deletions(-)
Signature · 1 line
2.17.1
PATCH v2 1/5 net: ena: change num_queues to num_io_queues for clarity and consistency
From: Sameeh Jubran <[email protected]>
Most places in the code refer to the IO queues as io_queues and not simply queues. Examples - max_io_queues_per_vf, ENA_MAX_NUM_IO_QUEUES, ena_destroy_all_io_queues() etc..
We are also adding the new max_num_io_queues field to struct ena_adapter in the following commit.
The changes included in this commit are:
struct ena_adapter->num_queues => struct ena_adapter->num_io_queues
--- drivers/net/ethernet/amazon/ena/ena_ethtool.c | 20 +++--- drivers/net/ethernet/amazon/ena/ena_netdev.c | 66 +++++++++---------- drivers/net/ethernet/amazon/ena/ena_netdev.h | 2 +- 3 files changed, 44 insertions(+), 44 deletions(-)
drivers/net/ethernet/amazon/ena/ena_ethtool.c +10 -10
@@ -133,7 +133,7 @@ static void ena_queue_stats(struct ena_adapter *adapter, u64 **data)u64 *ptr;int i, j; -for (i =0 ; i < adapter->num_queues; i++) { +for (i =0 ; i < adapter->num_io_queues; i++) {/* Tx stats */ ring = &adapter->tx_ring[i];
@@ -205,7 +205,7 @@ int ena_get_sset_count(struct net_device *netdev, int sset)if (sset !=ETH_SS_STATS )return -EOPNOTSUPP ; -return adapter->num_queues * (ENA_STATS_ARRAY_TX +ENA_STATS_ARRAY_RX ) +return adapter->num_io_queues * (ENA_STATS_ARRAY_TX +ENA_STATS_ARRAY_RX ) +ENA_STATS_ARRAY_GLOBAL +ENA_STATS_ARRAY_ENA_COM ; }
@@ -214,7 +214,7 @@ static void ena_queue_strings(struct ena_adapter *adapter, u8 **data)const struct ena_stats *ena_stats;int i, j; -for (i =0 ; i < adapter->num_queues; i++) { +for (i =0 ; i < adapter->num_io_queues; i++) {/* Tx stats */ for (j =0 ; j <ENA_STATS_ARRAY_TX ; j++) { ena_stats = &ena_stats_tx_strings[j];
@@ -333,7 +333,7 @@ static void ena_update_tx_rings_intr_moderation(struct ena_adapter *adapter) val =ena_com_get_nonadaptive_moderation_interval_tx (adapter->ena_dev); -for (i =0 ; i < adapter->num_queues; i++) +for (i =0 ; i < adapter->num_io_queues; i++) adapter->tx_ring[i].smoothed_interval = val; }
@@ -344,7 +344,7 @@ static void ena_update_rx_rings_intr_moderation(struct ena_adapter *adapter) val =ena_com_get_nonadaptive_moderation_interval_rx (adapter->ena_dev); -for (i =0 ; i < adapter->num_queues; i++) +for (i =0 ; i < adapter->num_io_queues; i++) adapter->rx_ring[i].smoothed_interval = val; }
@@ -612,7 +612,7 @@ static int ena_get_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *info,switch (info->cmd) {case ETHTOOL_GRXRINGS : - info->data = adapter->num_queues; + info->data = adapter->num_io_queues; rc =0 ;break ;case ETHTOOL_GRXFH :
@@ -734,12 +734,12 @@ static void ena_get_channels(struct net_device *netdev, {struct ena_adapter *adapter =netdev_priv (netdev); - channels->max_rx = adapter->num_queues; - channels->max_tx = adapter->num_queues; + channels->max_rx = adapter->num_io_queues; + channels->max_tx = adapter->num_io_queues; channels->max_other =0 ; channels->max_combined =0 ; - channels->rx_count = adapter->num_queues; - channels->tx_count = adapter->num_queues; + channels->rx_count = adapter->num_io_queues; + channels->tx_count = adapter->num_io_queues; channels->other_count =0 ; channels->combined_count =0 ; }
drivers/net/ethernet/amazon/ena/ena_netdev.c +33 -33
259 lines not on this page · raw message
drivers/net/ethernet/amazon/ena/ena_netdev.h +1 -1
@@ -324,7 +324,7 @@ struct ena_adapter {u32 rx_copybreak;u32 max_mtu; -int num_queues; +int num_io_queues;int msix_vecs;
Signature · 1 line
2.17.1
PATCH v2 2/5 net: ena: multiple queue creation related cleanups
In reply to [email protected]From: Sameeh Jubran <[email protected]>
- Rename ena_calc_queue_size() to ena_calc_io_queue_size() for clarity and consistency
- Remove redundant number of io queues parameter in functions ena_enable_msix() and ena_enable_msix_and_set_admin_interrupts(), which already get adapter parameter, so use adapter->num_io_queues in the function instead.
- Use the local variable ena_dev instead of ctx->ena_dev in ena_calc_io_queue_size
- Fix multi row comment alignments
--- drivers/net/ethernet/amazon/ena/ena_netdev.c | 33 +++++++------------- 1 file changed, 12 insertions(+), 21 deletions(-)
drivers/net/ethernet/amazon/ena/ena_netdev.c +12 -21
@@ -1332,7 +1332,7 @@ static irqreturn_t ena_intr_msix_io(int irq, void *data)* the number of potential io queues is the minimum of what the device * supports and the number of vCPUs. */ -static int ena_enable_msix (struct ena_adapter *adapter,int num_queues) +static int ena_enable_msix (struct ena_adapter *adapter) {int msix_vecs, irq_cnt;
@@ -1343,7 +1343,7 @@ static int ena_enable_msix(struct ena_adapter *adapter, int num_queues) }/* Reserved the max msix vectors we might need */ - msix_vecs =ENA_MAX_MSIX_VEC (num_queues); + msix_vecs =ENA_MAX_MSIX_VEC (adapter->num_io_queues);netif_dbg (adapter, probe, adapter->netdev,"trying to enable MSI-X, vectors %d\n" , msix_vecs);
@@ -2683,14 +2683,13 @@ err_mmio_read_less:return rc; } -static int ena_enable_msix_and_set_admin_interrupts (struct ena_adapter *adapter, -int io_vectors) +static int ena_enable_msix_and_set_admin_interrupts (struct ena_adapter *adapter) {struct ena_com_dev *ena_dev = adapter->ena_dev;struct device *dev = &adapter->pdev->dev;int rc; - rc =ena_enable_msix (adapter, io_vectors); + rc =ena_enable_msix (adapter);if (rc) {dev_err (dev,"Can not reserve msix vectors\n" );return rc;
@@ -2783,8 +2782,7 @@ static int ena_restore_device(struct ena_adapter *adapter) goto err_device_destroy; } - rc =ena_enable_msix_and_set_admin_interrupts (adapter, - adapter->num_io_queues); + rc =ena_enable_msix_and_set_admin_interrupts (adapter);if (rc) {dev_err (&pdev->dev,"Enable MSI-X failed\n" ); goto err_device_destroy;
@@ -3350,7 +3348,7 @@ static void set_default_llq_configurations(struct ena_llq_configurations *llq_co llq_config->llq_ring_entry_size_value =128 ; } -static int ena_calc_queue_size (struct ena_calc_queue_size_ctx *ctx) +static int ena_calc_io_queue_size (struct ena_calc_queue_size_ctx *ctx) {struct ena_admin_feature_llq_desc *llq = &ctx->get_feat_ctx->llq;struct ena_com_dev *ena_dev = ctx->ena_dev;
@@ -3359,7 +3357,7 @@ static int ena_calc_queue_size(struct ena_calc_queue_size_ctx *ctx)u32 max_tx_queue_size;u32 max_rx_queue_size; -if (ctx->ena_dev->supported_features &BIT (ENA_ADMIN_MAX_QUEUES_EXT )) { +if (ena_dev->supported_features &BIT (ENA_ADMIN_MAX_QUEUES_EXT )) {struct ena_admin_queue_ext_feature_fields *max_queue_ext = &ctx->get_feat_ctx->max_queue_ext.max_queue_ext; max_rx_queue_size = min_t(u32, max_queue_ext->max_rx_cq_depth,
@@ -3497,26 +3495,19 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent) calc_queue_ctx.get_feat_ctx = &get_feat_ctx; calc_queue_ctx.pdev = pdev; -/* Initial Tx and RX interrupt delay. Assumes 1 usec granularity. -* Updated during device initialization with the real granularity -*/ +/* initial Tx interrupt delay, Assumes 1 usec granularity. +* Updated during device initialization with the real granularity +*/ ena_dev->intr_moder_tx_interval =ENA_INTR_INITIAL_TX_INTERVAL_USECS ; ena_dev->intr_moder_rx_interval =ENA_INTR_INITIAL_RX_INTERVAL_USECS ; ena_dev->intr_delay_resolution =ENA_DEFAULT_INTR_DELAY_RESOLUTION ; io_queue_num =ena_calc_io_queue_num (pdev, ena_dev, &get_feat_ctx); - rc =ena_calc_queue_size (&calc_queue_ctx); + rc =ena_calc_io_queue_size (&calc_queue_ctx);if (rc || io_queue_num <=0 ) { rc = -EFAULT ; goto err_device_destroy; } -dev_info (&pdev->dev,"creating %d io queues. rx queue size: %d tx queue size. %d LLQ is %s\n" , - io_queue_num, - calc_queue_ctx.rx_queue_size, - calc_queue_ctx.tx_queue_size, - (ena_dev->tx_mem_queue_type ==ENA_ADMIN_PLACEMENT_POLICY_DEV ) ? -"ENABLED" :"DISABLED" ); -/* dev zeroed in init_etherdev */ netdev =alloc_etherdev_mq (sizeof (struct ena_adapter ), io_queue_num);if (!netdev) {
@@ -3570,7 +3561,7 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent)u64_stats_init (&adapter->syncp); - rc =ena_enable_msix_and_set_admin_interrupts (adapter, io_queue_num); + rc =ena_enable_msix_and_set_admin_interrupts (adapter);if (rc) {dev_err (&pdev->dev,"Failed to enable and set the admin interrupts\n" );
Signature · 1 line
2.17.1
PATCH v2 3/5 net: ena: make ethtool -l show correct max number of queues
In reply to [email protected]From: Sameeh Jubran <[email protected]>
- Update ena_ethtool:ena_get_channels() to return adapter->max_io_queues so that ethtool -l returns the correct maximum queue number.
- Change the name of ena_calc_io_queue_num() to ena_calc_max_io_queue_num() as it returns the maximum number of io queues and actual number of queues can be smaller if changed by ethtool -L which is implemented in a later commit.
- Change variable name from io_queue_num to max_num_io_queues in ena_calc_max_io_queue_num() and ena_probe().
- Make all types of variables that convey the number and sizeof queues to be u32, for consistency with the API between the driver and the device.
--- drivers/net/ethernet/amazon/ena/ena_ethtool.c | 4 +- drivers/net/ethernet/amazon/ena/ena_netdev.c | 41 ++++++++++--------- drivers/net/ethernet/amazon/ena/ena_netdev.h | 11 ++--- 3 files changed, 30 insertions(+), 26 deletions(-)
drivers/net/ethernet/amazon/ena/ena_ethtool.c +2 -2
@@ -734,8 +734,8 @@ static void ena_get_channels(struct net_device *netdev, {struct ena_adapter *adapter =netdev_priv (netdev); - channels->max_rx = adapter->num_io_queues; - channels->max_tx = adapter->num_io_queues; + channels->max_rx = adapter->max_num_io_queues; + channels->max_tx = adapter->max_num_io_queues; channels->max_other =0 ; channels->max_combined =0 ; channels->rx_count = adapter->num_io_queues;
drivers/net/ethernet/amazon/ena/ena_netdev.c +22 -19
103 lines not on this page · raw message
drivers/net/ethernet/amazon/ena/ena_netdev.h +6 -5
@@ -161,10 +161,10 @@ struct ena_calc_queue_size_ctx {struct ena_com_dev_get_features_ctx *get_feat_ctx;struct ena_com_dev *ena_dev;struct pci_dev *pdev; -u16 tx_queue_size; -u16 rx_queue_size; -u16 max_tx_queue_size; -u16 max_rx_queue_size; +u32 tx_queue_size; +u32 rx_queue_size; +u32 max_tx_queue_size; +u32 max_rx_queue_size;u16 max_tx_sgl_size;u16 max_rx_sgl_size; };
@@ -324,7 +324,8 @@ struct ena_adapter {u32 rx_copybreak;u32 max_mtu; -int num_io_queues; +u32 num_io_queues; +u32 max_num_io_queues;int msix_vecs;
Signature · 1 line
2.17.1
PATCH v2 4/5 net: ena: remove redundant print of number of queues
In reply to [email protected]From: Sameeh Jubran <[email protected]>
The number of queues can be derived using ethtool, no need to print it in ena_probe()
--- drivers/net/ethernet/amazon/ena/ena_netdev.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/amazon/ena/ena_netdev.c +2 -2
@@ -3606,9 +3606,9 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent) queue_type_str ="Low Latency" ;dev_info (&pdev->dev, -"%s found at mem %lx, mac addr %pM Queues %d, Placement policy: %s\n" , +"%s found at mem %lx, mac addr %pM, Placement policy: %s\n" ,DEVICE_NAME , (long )pci_resource_start (pdev,0 ), - netdev->dev_addr, max_num_io_queues, queue_type_str); + netdev->dev_addr, queue_type_str);set_bit (ENA_FLAG_DEVICE_RUNNING , &adapter->flags);
Signature · 1 line
2.17.1
PATCH v2 5/5 net: ena: ethtool: support set_channels callback
In reply to [email protected]From: Sameeh Jubran <[email protected]>
Set channels callback enables the user to change the count of queues used by the driver using ethtool. We decided to currently support only equal number of rx and tx queues, this might change in the future.
Also rename dev_up to dev_was_up in ena_update_queue_count() to make it clearer.
--- drivers/net/ethernet/amazon/ena/ena_ethtool.c | 17 ++++++++++++++ drivers/net/ethernet/amazon/ena/ena_netdev.c | 22 ++++++++++++++++--- drivers/net/ethernet/amazon/ena/ena_netdev.h | 3 +++ 3 files changed, 39 insertions(+), 3 deletions(-)
drivers/net/ethernet/amazon/ena/ena_ethtool.c +17 -0
@@ -744,6 +744,22 @@ static void ena_get_channels(struct net_device *netdev, channels->combined_count =0 ; } +static int ena_set_channels (struct net_device *netdev, +struct ethtool_channels *channels) +{ +struct ena_adapter *adapter =netdev_priv (netdev); +u32 new_channel_count; + +if (channels->rx_count != channels->tx_count || + channels->max_tx != channels->max_rx) +return -EINVAL ; + + new_channel_count =clamp_val (channels->tx_count, +ENA_MIN_NUM_IO_QUEUES , channels->max_tx); + +return ena_update_queue_count (adapter, new_channel_count); +} +static int ena_get_tunable (struct net_device *netdev,const struct ethtool_tunable *tuna,void *data) {
@@ -807,6 +823,7 @@ static const struct ethtool_ops ena_ethtool_ops = { .get_rxfh = ena_get_rxfh, .set_rxfh = ena_set_rxfh, .get_channels = ena_get_channels, + .set_channels = ena_set_channels, .get_tunable = ena_get_tunable, .set_tunable = ena_set_tunable, };
drivers/net/ethernet/amazon/ena/ena_netdev.c +19 -3
@@ -2044,14 +2044,30 @@ int ena_update_queue_sizes(struct ena_adapter *adapter,u32 new_tx_size, u32 new_rx_size) { - bool dev_up; + bool dev_was_up; - dev_up =test_bit (ENA_FLAG_DEV_UP , &adapter->flags); + dev_was_up =test_bit (ENA_FLAG_DEV_UP , &adapter->flags);ena_close (adapter->netdev); adapter->requested_tx_ring_size = new_tx_size; adapter->requested_rx_ring_size = new_rx_size;ena_init_io_rings (adapter); -return dev_up ?ena_up (adapter) :0 ; +return dev_was_up ?ena_up (adapter) :0 ; +} + +int ena_update_queue_count (struct ena_adapter *adapter,u32 new_channel_count) +{ +struct ena_com_dev *ena_dev = adapter->ena_dev; +bool dev_was_up; + + dev_was_up =test_bit (ENA_FLAG_DEV_UP , &adapter->flags); +ena_close (adapter->netdev); + adapter->num_io_queues = new_channel_count; +/* We need to destroy the rss table so that the indirection +* table will be reinitialized by ena_up() +*/ +ena_com_rss_destroy (ena_dev); +ena_init_io_rings (adapter); +return dev_was_up ?ena_open (adapter->netdev) :0 ; }static void ena_tx_csum (struct ena_com_tx_ctx *ena_tx_ctx,struct sk_buff *skb)
drivers/net/ethernet/amazon/ena/ena_netdev.h +3 -0
@@ -82,6 +82,8 @@#define ENA_DEFAULT_RING_SIZE (1024)#define ENA_MIN_RING_SIZE (256) +#define ENA_MIN_NUM_IO_QUEUES (1) +#define ENA_TX_WAKEUP_THRESH (MAX_SKB_FRAGS + 2)#define ENA_DEFAULT_RX_COPYBREAK (256 - NET_IP_ALIGN)
@@ -388,6 +390,7 @@ void ena_dump_stats_to_buf(struct ena_adapter *adapter, u8 *buf);int ena_update_queue_sizes (struct ena_adapter *adapter,u32 new_tx_size,u32 new_rx_size); +int ena_update_queue_count (struct ena_adapter *adapter,u32 new_channel_count);int ena_get_sset_count (struct net_device *netdev,int sset);
Signature · 1 line
2.17.1
Re: [PATCH V2 net-next 5/5] net: ena: ethtool: support set_channels callback
Quoted 21 lines · On Wed, 2 Oct 2019 11:20:52 +0300, [email protected] wrote:
From: Sameeh Jubran <[email protected]>
Set channels callback enables the user to change the count of queues used by the driver using ethtool. We decided to currently support only equal number of rx and tx queues, this might change in the future.
Also rename
dev_uptodev_was_upinena_update_queue_count()to make it clearer.Signed-off-by Sameeh Jubran <[email protected]>--- drivers/net/ethernet/amazon/ena/ena_ethtool.c | 17 ++++++++++++++ drivers/net/ethernet/amazon/ena/ena_netdev.c | 22 ++++++++++++++++--- drivers/net/ethernet/amazon/ena/ena_netdev.h | 3 +++ 3 files changed, 39 insertions(+), 3 deletions(-)drivers/net/ethernet/amazon/ena/ena_ethtool.c +0
-0
drivers/net/ethernet/amazon/ena/ena_ethtool.c static void ena_get_channels(struct net_device *netdev, · line 753
Quoted 10 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c +7
-0@@ -744,6 +744,22 @@ static void ena_get_channels(struct net_device *netdev, channels->combined_count =0 ; } +static int ena_set_channels (struct net_device *netdev, +struct ethtool_channels *channels) +{ +struct ena_adapter *adapter =netdev_priv (netdev); +u32 new_channel_count; + + if (channels->rx_count != channels->tx_count ||
If you use the same IRQ and NAPI to service RX and TX it's a combined channel, not rx and tx channels.
drivers/net/ethernet/amazon/ena/ena_ethtool.c static void ena_get_channels(struct net_device *netdev, · line 754
Quoted 1 line
drivers/net/ethernet/amazon/ena/ena_ethtool.c +1
-0+ channels->max_tx != channels->max_rx)
Hm.. maxes are generally ignored on set 🤔
drivers/net/ethernet/amazon/ena/ena_ethtool.c static void ena_get_channels(struct net_device *netdev, · line 758
Quoted 4 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c +4
-0+return -EINVAL ; + + new_channel_count =clamp_val (channels->tx_count, +ENA_MIN_NUM_IO_QUEUES , channels->max_tx);
You should return an error if the value is not within bounds, rather than guessing.
Quoted 39 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c +4
-0+return ena_update_queue_count (adapter, new_channel_count); +} +static int ena_get_tunable (struct net_device *netdev,const struct ethtool_tunable *tuna,void *data) {@@ -807,6 +823,7 @@ static const struct ethtool_ops ena_ethtool_ops = { .get_rxfh = ena_get_rxfh, .set_rxfh = ena_set_rxfh, .get_channels = ena_get_channels, + .set_channels = ena_set_channels, .get_tunable = ena_get_tunable, .set_tunable = ena_set_tunable, };drivers/net/ethernet/amazon/ena/ena_netdev.c +9
-3@@ -2044,14 +2044,30 @@ int ena_update_queue_sizes(struct ena_adapter *adapter,u32 new_tx_size, u32 new_rx_size) { - bool dev_up; + bool dev_was_up; - dev_up =test_bit (ENA_FLAG_DEV_UP , &adapter->flags); + dev_was_up =test_bit (ENA_FLAG_DEV_UP , &adapter->flags);ena_close (adapter->netdev); adapter->requested_tx_ring_size = new_tx_size; adapter->requested_rx_ring_size = new_rx_size;ena_init_io_rings (adapter); -return dev_up ?ena_up (adapter) :0 ; +return dev_was_up ?ena_up (adapter) :0 ; +} + +int ena_update_queue_count (struct ena_adapter *adapter,u32 new_channel_count) +{ +struct ena_com_dev *ena_dev = adapter->ena_dev; +bool dev_was_up;
drivers/net/ethernet/amazon/ena/ena_netdev.c int ena_update_queue_sizes(struct ena_adapter *adapter, · line 2070
Quoted 10 lines
drivers/net/ethernet/amazon/ena/ena_netdev.c +10
-0+ + dev_was_up =test_bit (ENA_FLAG_DEV_UP , &adapter->flags); +ena_close (adapter->netdev); + adapter->num_io_queues = new_channel_count; +/* We need to destroy the rss table so that the indirection +* table will be reinitialized by ena_up() +*/ +ena_com_rss_destroy (ena_dev); +ena_init_io_rings (adapter); +return dev_was_up ?ena_open (adapter->netdev) :0 ;
You should try to prepare the resources for the new configuration before you attempt the change. Otherwise if allocation of new rings fails the open will leave the device in a broken state.
This is not always enforced upstream, but you can see mlx5 or nfp for examples of drivers which do this right..
Quoted 3 lines
drivers/net/ethernet/amazon/ena/ena_netdev.c +0
-0}static void ena_tx_csum (struct ena_com_tx_ctx *ena_tx_ctx,struct sk_buff *skb)
RE: [PATCH V2 net-next 5/5] net: ena: ethtool: support set_channels callback
Quoted 47 lines
-----Original Message-----
From: Jakub Kicinski <[email protected]>
Sent: Wednesday, October 2, 2019 11:12 PM
To: Jubran, Samih <[email protected]>
Cc: [email protected]; [email protected]; Woodhouse, David <[email protected]>; Machulsky, Zorik <[email protected]>; Matushevsky, Alexander <[email protected]>; Bshara, Saeed <[email protected]>; Wilson, Matt <[email protected]>; Liguori, Anthony <[email protected]>; Bshara, Nafea <[email protected]>; Tzalik, Guy <[email protected]>; Belgazal, Netanel <[email protected]>; Saidi, Ali <[email protected]>; Herrenschmidt, Benjamin <[email protected]>; Kiyanovski, Arthur <[email protected]>
Subject: Re: [PATCH V2 net-next 5/5] net: ena: ethtool: supportset_channelscallbackQuoted 7 lines · On Wed, 2 Oct 2019 11:20:52 +0300, [email protected] wrote:
From: Sameeh Jubran <[email protected]>
Set channels callback enables the user to change the count of queues used by the driver using ethtool. We decided to currently support only equal number of rx and tx queues, this might change in the future.
Also rename
dev_uptodev_was_upinena_update_queue_count()tomake
Quoted 6 lines
it clearer.
Signed-off-by Sameeh Jubran <[email protected]>--- drivers/net/ethernet/amazon/ena/ena_ethtool.c | 17 ++++++++++++++
drivers/net/ethernet/amazon/ena/ena_netdev.c| 22++++++++++++++++---
drivers/net/ethernet/amazon/ena/ena_ethtool.c static void ena_get_channels(struct net_device
Quoted 9 lines
drivers/net/ethernet/amazon/ena/ena_netdev.h| 3 +++3 files changed, 39 insertions(+), 3 deletions(-)drivers/net/ethernet/amazon/ena/ena_ethtool.c +0
-0@@ -744,6 +744,22 @@ static void ena_get_channels(struct net_device*netdev,
Quoted 3 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c +0
-0channels->combined_count =0 ; }
Quoted 9 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c static void ena_get_channels(struct net_device *netdev, · line 753
Quoted 6 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c +6
-0+static int ena_set_channels (struct net_device *netdev, +struct ethtool_channels *channels) { +struct ena_adapter *adapter =netdev_priv (netdev); +u32 new_channel_count; + + if (channels->rx_count != channels->tx_count ||If you use the same IRQ and NAPI to service RX and TX it's a combined channel, not rx and tx channels.
Will do.
Quoted 5 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c static void ena_get_channels(struct net_device *netdev, · line 754
Quoted 2 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c +1
-0+ channels->max_tx != channels->max_rx)Hm.. maxes are generally ignored on set 🤔
Quoted 8 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c static void ena_get_channels(struct net_device *netdev,
Quoted 5 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c +4
-0+return -EINVAL ; + + new_channel_count =clamp_val (channels->tx_count, +ENA_MIN_NUM_IO_QUEUES , channels- max_tx);You should return an error if the value is not within bounds, rather than guessing.
Will do.
Quoted 46 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c static void ena_get_channels(struct net_device *netdev,
Quoted 5 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c +2
-0+return ena_update_queue_count (adapter, new_channel_count); } +static int ena_get_tunable (struct net_device *netdev,const struct ethtool_tunable *tuna,void *data) {@@ -807,6
drivers/net/ethernet/amazon/ena/ena_netdev.c int ena_update_queue_sizes(struct
Quoted 14 lines
drivers/net/ethernet/amazon/ena/ena_ethtool.c +2
-0+823,7 @@ static const struct ethtool_ops ena_ethtool_ops = { .get_rxfh = ena_get_rxfh, .set_rxfh = ena_set_rxfh, .get_channels = ena_get_channels, + .set_channels = ena_set_channels, .get_tunable = ena_get_tunable, .set_tunable = ena_set_tunable, };drivers/net/ethernet/amazon/ena/ena_netdev.c +0
-0@@ -2044,14 +2044,30 @@ int ena_update_queue_sizes(struct
ena_adapter*adapter,Quoted 25 lines
drivers/net/ethernet/amazon/ena/ena_netdev.c +14
-3u32 new_tx_size, u32 new_rx_size) { - bool dev_up; + bool dev_was_up; - dev_up =test_bit (ENA_FLAG_DEV_UP , &adapter->flags); + dev_was_up =test_bit (ENA_FLAG_DEV_UP , &adapter->flags);ena_close (adapter->netdev); adapter->requested_tx_ring_size = new_tx_size; adapter->requested_rx_ring_size = new_rx_size;ena_init_io_rings (adapter); -return dev_up ?ena_up (adapter) :0 ; +return dev_was_up ?ena_up (adapter) :0 ; } + +int ena_update_queue_count (struct ena_adapter *adapter,u32 +new_channel_count) { +struct ena_com_dev *ena_dev = adapter->ena_dev; +bool dev_was_up; + + dev_was_up =test_bit (ENA_FLAG_DEV_UP , &adapter->flags); +ena_close (adapter->netdev); + adapter->num_io_queues = new_channel_count; +/* We need to destroy the rss table so that the indirection +* table will be reinitialized by ena_up()
Quoted 8 lines
drivers/net/ethernet/amazon/ena/ena_netdev.c int ena_update_queue_sizes(struct ena_adapter *adapter, · line 2070
Quoted 4 lines
drivers/net/ethernet/amazon/ena/ena_netdev.c +4
-0+*/ +ena_com_rss_destroy (ena_dev); +ena_init_io_rings (adapter); +return dev_was_up ?ena_open (adapter->netdev) :0 ;You should try to prepare the resources for the new configuration before you attempt the change. Otherwise if allocation of new rings fails the open will leave the device in a broken state.
Our ena_up() applies logarithmic backoff mechanism on the rings size if the allocations fail, and therefore the device will stay functional.
Quoted 8 lines
This is not always enforced upstream, but you can see mlx5 or nfp for examples of drivers which do this right..
Quoted 4 lines
drivers/net/ethernet/amazon/ena/ena_netdev.c +0
-0} static void ena_tx_csum(struct ena_com_tx_ctx *ena_tx_ctx, struct
sk_buff*skb)
Re: [PATCH V2 net-next 5/5] net: ena: ethtool: support set_channels callback
Quoted 10 lines · Samih
Quoted 10 lines
drivers/net/ethernet/amazon/ena/ena_netdev.c int ena_update_queue_sizes(struct ena_adapter *adapter, · line 2067
Quoted 10 lines
drivers/net/ethernet/amazon/ena/ena_netdev.c +10
-0+int ena_update_queue_count (struct ena_adapter *adapter,u32 new_channel_count) { +struct ena_com_dev *ena_dev = adapter->ena_dev; +bool dev_was_up; + + dev_was_up =test_bit (ENA_FLAG_DEV_UP , &adapter->flags); +ena_close (adapter->netdev); + adapter->num_io_queues = new_channel_count; +/* We need to destroy the rss table so that the indirection +* table will be reinitialized by ena_up() +*/
Ugh, this comment is broken in the same way the comment Colin fixed in commit 4208966f65f5 ("net: ena: clean up indentation issue") was. Please double check your submission for this and use checkpatch.
Quoted 10 lines
Quoted 7 lines
drivers/net/ethernet/amazon/ena/ena_netdev.c int ena_update_queue_sizes(struct ena_adapter *adapter, · line 2070
Quoted 3 lines
drivers/net/ethernet/amazon/ena/ena_netdev.c +3
-0+ena_com_rss_destroy (ena_dev); +ena_init_io_rings (adapter); +return dev_was_up ?ena_open (adapter->netdev) :0 ;You should try to prepare the resources for the new configuration before you attempt the change. Otherwise if allocation of new rings fails the open will leave the device in a broken state.
Our
ena_up()applies logarithmic backoff mechanism on the rings size if the allocations fail, and therefore the device will stay functional.
Sure, a heuristic like that will help, but doesn't give hard guarantees, which is what engineers like :)
Quoted 7 lines
Quoted 7 lines
This is not always enforced upstream, but you can see mlx5 or nfp for examples of drivers which do this right..
Quoted 4 lines
}
static void ena_tx_csum(struct
ena_com_tx_ctx*ena_tx_ctx, structsk_buff*skb)
Re: [PATCH V2 net-next 5/5] net: ena: ethtool: support set_channels callback
In reply to Jakub KicinskiQuoted 7 lines · Jakub Kicinski
drivers/net/ethernet/amazon/ena/ena_ethtool.c static void ena_get_channels(struct net_device *netdev, · line 758
Quoted 3 lines · On Wed, 2 Oct 2019 11:20:52 +0300, [email protected] wrote:
drivers/net/ethernet/amazon/ena/ena_ethtool.c +3
-0+ + new_channel_count =clamp_val (channels->tx_count, +ENA_MIN_NUM_IO_QUEUES , channels->max_tx);You should return an error if the value is not within bounds, rather than guessing.
And ethtool_set_channels() already does that if any of the requested counts exceeds the corresponding maximum so that the upper bound check here is superfluous.
Michal