ethtool: Dynamic RSS context indirection table resizing
Subsystems (8)
v6 cover ethtool: Dynamic RSS context indirection table resizing
Hi!
Some NICs (e.g. bnxt) change their RSS indirection table size based on the queue count, because the hardware table is a shared resource. The ethtool core locks ctx->indir_size at context creation, so drivers have to reject channel changes when RSS contexts exist.
This series adds resize helpers and wires them up in bnxt. It also adds tracking of the user provided indirection table size to the ethtool core.
Patch 1 tracks the user-provided indirection table size (user_size) in ctx->indir_user_size for non-default RSS contexts and in dev->ethtool->rss_indir_user_size for context 0. It is set when the indirection table is configured via netlink or ioctl, and cleared to zero on reset-to-default.
IFF_RXFH_CONFIGURED is removed, and replaced with rss_indir_user_size. The flag is redundant now that user_size captures the same information.
Patch 2 adds core resize helpers:
ethtool_rxfh_indir_can_resize() - read-only validation for context 0 ethtool_rxfh_indir_resize() - fold/unfold context 0 table in place ethtool_rxfh_ctxs_can_resize() - validate all non-default contexts ethtool_rxfh_ctxs_resize() - resize all non-default contexts,
with locking and RSS_NTF notifications
Patch 3 uses the resize helpers in bnxt_set_channels().
Patch 4 adds HW tests in rss_drv.py (devices without dynamic table sizing are skipped):
resize_periodic - fold/unfold with a non-default [3,2,1,0]
sub-table (user_size=4), verifying exact content preservation (main + ctx)
resize_below_user_size_reject - periodic sub-table with user_size
between big and small device table sizes; verifies that shrinking below user_size is rejected even when the table is periodic (main + ctx)
resize_nonperiodic_reject - non-periodic table blocks channel
reduction, with an extra periodic context to exercise multi-context validation (main + ctx)
resize_nonperiodic_no_corruption - failed resize leaves table
contents and channel count unchanged (main + ctx)
Running the tests:
# On real hardware
sudo NETIF=eth0 ./rss_drv.py
Changes v5 -> v6:
- rss_set_prep_indir() returns user_size (>= 0) on success instead of using a u32 *user_sizep output parameter (Jakub)
- Removed IFF_RXFH_CONFIGURED; netif_is_rxfh_configured() now checks dev->ethtool->rss_indir_user_size instead of priv_flags (Jakub)
- Added ethtool_rxfh_indir_clear() helper for drivers that forcibly reset indirection tables. Sends ETHTOOL_MSG_RSS_NTF notification. Converted bnxt and mlx5 to use it (Jakub)
- ethtool_rxfh_indir_can_resize()/ethtool_rxfh_indir_resize() wrap the internal resize logic for context 0, reading user_size from dev->ethtool->rss_indir_user_size. Removes the need for netif_is_rxfh_configured() guards in drivers (Jakub)
- Removed netif_is_rxfh_configured() guards in bnxt_set_channels() and nsim_set_channels() (Jakub)
- More consistent function names
Changes v4 -> v5:
- Track user-provided indirection table size (user_size) as a resize floor. Added indir_user_size to ethtool_rxfh_context and rss_indir_user_size to ethtool_netdev_state. ethtool_rxfh_can_resize() and ethtool_rxfh_resize() now take a user_size parameter and reject shrinking below it. (Jakub)
- Propagated user_size out of rss_set_prep_indir() and stored it on successful set in both netlink and ioctl paths.
- resize_periodic test now sends a 4-entry sub-table (user_size=4) instead of replicating to full device table size
- Added resize_below_user_size_reject test to verify user_size floor.
- Removed "Open items" section - user_size tracking is now implemented.
Changes v3 -> v4:
- Rebased onto net-next
- Added Reviewed-by: from Michael
- Added missing Cc: to make the pwbots happier
Changes v2 -> v3:
- Changed ethtool_rxfh_can_resize() to return bool instead of int; true means resize is possible, false means it is not. Inverted callers accordingly. (Jakub)
- Added Tested-by from Pavan
Changes v1 -> v2:
- Dropped netdevsim support and netdevsim selftest (Jakub)
- Split ethtool_rxfh_contexts_resize_all() into separate validate (ethtool_rxfh_ctxs_can_resize) and apply (ethtool_rxfh_ctxs_resize) so drivers can validate before closing the device (Jakub)
- Shortened helper names (Jakub)
- Replaced scoped_guard(mutex) with explicit mutex_lock/unlock (Jakub)
- Removed defensive zero-size check, bare expressions instead of != 0 comparisons, ! instead of == 0 (Jakub)
- In bnxt, moved bnxt_check_rings() before RSS validation and deferred actual resize to after bnxt_close_nic() (Jakub, Michael)
- Added comment that RSS table size only changes on P5 chips with older firmware (Michael)
- Use non-default [3,2,1,0]xN pattern set via netlink to distinguish correct fold from driver resetting to defaults (Jakub)
- Check exact indirection table pattern, not just set(indir) (Jakub)
- Use ksft_raises() instead of try/except/else (Jakub)
- Removed queue_count=8 from NetDrvEnv (Jakub)
- Added ksft_variants to resize_nonperiodic_reject for ctx coverage
- Added extra periodic context in reject test for multi-context validation coverage
- Added resize_nonperiodic_no_corruption test
Björn Töpel (4):
ethtool: Track user-provided RSS indirection table size
ethtool: Add RSS indirection table resize helpers
bnxt_en: Resize RSS contexts on channel count change
selftests: rss_drv: Add RSS indirection table resize tests
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 +- .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 35 ++- .../net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- include/linux/ethtool.h | 13 + include/linux/netdevice.h | 7 +- net/ethtool/common.c | 175 +++++++++++++ net/ethtool/ioctl.c | 9 +- net/ethtool/rss.c | 24 +- .../selftests/drivers/net/hw/rss_drv.py | 233 +++++++++++++++++- 9 files changed, 469 insertions(+), 31 deletions(-)
base-commit: 70729af783af1d66944918b33e5b345b500a5399
Signature · 1 line
2.53.0
PATCH v6 1/4 ethtool: Track user-provided RSS indirection table size
Track the number of indirection table entries the user originally provided (context 0/default as well!).
Replace IFF_RXFH_CONFIGURED with rss_indir_user_size: the flag is redundant now that user_size captures the same information.
Add ethtool_rxfh_indir_clear() for drivers that must reset the indirection table.
Convert bnxt and mlx5 to use it.
--- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 +- .../net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- include/linux/ethtool.h | 7 ++++++ include/linux/netdevice.h | 7 +----- net/ethtool/common.c | 20 ++++++++++++++++ net/ethtool/ioctl.c | 9 +++---- net/ethtool/rss.c | 24 ++++++++++++------- 7 files changed, 51 insertions(+), 20 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1 -1
@@ -8117,7 +8117,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp) bnxt_get_nr_rss_ctxs(bp, rx_rings) || bnxt_get_max_rss_ring(bp) >= rx_rings)) {netdev_warn (bp->dev,"RSS table entries reverting to default\n" ); - bp->dev->priv_flags &= ~IFF_RXFH_CONFIGURED ; +ethtool_rxfh_indir_clear (bp->dev); } } bp->rx_nr_rings = rx_rings;
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1 -1
@@ -6488,7 +6488,7 @@ int mlx5e_attach_netdev(struct mlx5e_priv *priv)/* Reducing the number of channels - RXFH has to be reset, and * mlx5e_num_channels_changed below will build the RQT. */ - priv->netdev->priv_flags &= ~IFF_RXFH_CONFIGURED ; +ethtool_rxfh_indir_clear (priv->netdev); priv->channels.params.num_channels = max_nch;if (priv->channels.params.mqprio.mode ==TC_MQPRIO_MODE_CHANNEL ) {mlx5_core_warn (priv->mdev,"MLX5E: Disabling MQPRIO channel mode\n" );
include/linux/ethtool.h +7 -0
@@ -176,6 +176,8 @@ static inline u32 ethtool_rxfh_indir_default(u32 index, u32 n_rx_rings)* struct ethtool_rxfh_context - a custom RSS context configuration * @indir_size: Number of u32 entries in indirection table * @key_size: Size of hash key, in bytes +* @indir_user_size: number of user provided entries for the +* indirection table * @priv_size: Size of driver private data, in bytes * @hfunc: RSS hash function identifier. One of the %ETH_RSS_HASH_* * @input_xfrm: Defines how the input data is transformed. Valid values are one
@@ -186,6 +188,7 @@ static inline u32 ethtool_rxfh_indir_default(u32 index, u32 n_rx_rings)struct ethtool_rxfh_context {u32 indir_size;u32 key_size; +u32 indir_user_size;u16 priv_size;u8 hfunc;u8 input_xfrm;
@@ -214,6 +217,7 @@ static inline u8 *ethtool_rxfh_context_key(struct ethtool_rxfh_context *ctx) }void ethtool_rxfh_context_lost (struct net_device *dev,u32 context_id); +void ethtool_rxfh_indir_clear (struct net_device *dev);struct link_mode_info {int speed;
@@ -1333,12 +1337,15 @@ int ethtool_virtdev_set_link_ksettings(struct net_device *dev,* @rss_ctx: XArray of custom RSS contexts * @rss_lock: Protects entries in @rss_ctx. May be taken from * within RTNL. +* @rss_indir_user_size: Number of user provided entries for the default +* (context 0) indirection table. * @wol_enabled: Wake-on-LAN is enabled * @module_fw_flash_in_progress: Module firmware flashing is in progress. */ struct ethtool_netdev_state {struct xarray rss_ctx;struct mutex rss_lock; +u32 rss_indir_user_size;unsigned wol_enabled:1 ;unsigned module_fw_flash_in_progress:1 ; };
include/linux/netdevice.h +1 -6
@@ -1714,7 +1714,6 @@ struct net_device_ops {* @IFF_OPENVSWITCH: device is a Open vSwitch master * @IFF_L3MDEV_SLAVE: device is enslaved to an L3 master device * @IFF_TEAM: device is a team device -* @IFF_RXFH_CONFIGURED: device has had Rx Flow indirection table configured * @IFF_PHONY_HEADROOM: the headroom value is controlled by an external * entity (i.e. the master device for bridged veth) * @IFF_MACSEC: device is a MACsec device
@@ -1750,7 +1749,6 @@ enum netdev_priv_flags {IFF_OPENVSWITCH =1 <<20 ,IFF_L3MDEV_SLAVE =1 <<21 ,IFF_TEAM =1 <<22 , -IFF_RXFH_CONFIGURED =1 <<23 ,IFF_PHONY_HEADROOM =1 <<24 ,IFF_MACSEC =1 <<25 , IFF_NO_RX_HANDLER = 1<<26,
@@ -5568,10 +5566,7 @@ static inline bool netif_is_lag_port(const struct net_device *dev)return netif_is_bond_slave (dev) ||netif_is_team_port (dev); } -static inline bool netif_is_rxfh_configured (const struct net_device *dev) -{ -return dev->priv_flags &IFF_RXFH_CONFIGURED ; -} +bool netif_is_rxfh_configured (const struct net_device *dev);static inline bool netif_is_failover (const struct net_device *dev) {
net/ethtool/common.c +20 -0
@@ -1204,6 +1204,26 @@ void ethtool_rxfh_context_lost(struct net_device *dev, u32 context_id) }EXPORT_SYMBOL (ethtool_rxfh_context_lost); +bool netif_is_rxfh_configured (const struct net_device *dev) +{ +return dev->ethtool->rss_indir_user_size; +} +EXPORT_SYMBOL (netif_is_rxfh_configured); + +/** +* ethtool_rxfh_indir_clear - Clear user indirection table config +* @dev: network device +* +* Mark the default RSS context indirection table as unconfigured and +* send an %ETHTOOL_MSG_RSS_NTF notification. +*/ +void ethtool_rxfh_indir_clear (struct net_device *dev) +{ + dev->ethtool->rss_indir_user_size =0 ; +ethtool_rss_notify (dev,ETHTOOL_MSG_RSS_NTF ,0 ); +} +EXPORT_SYMBOL (ethtool_rxfh_indir_clear); +enum ethtool_link_medium ethtool_str_to_medium (const char *str) {int i;
net/ethtool/ioctl.c +5 -4
@@ -1404,9 +1404,9 @@ static noinline_for_stack int ethtool_set_rxfh_indir(struct net_device *dev,/* indicate whether rxfh was set to default */ if (user_size ==0 ) - dev->priv_flags &= ~IFF_RXFH_CONFIGURED ; + dev->ethtool->rss_indir_user_size =0 ;else - dev->priv_flags |=IFF_RXFH_CONFIGURED ; + dev->ethtool->rss_indir_user_size = rxfh_dev.indir_size; out_unlock:mutex_unlock (&dev->ethtool->rss_lock);
@@ -1721,9 +1721,9 @@ static noinline_for_stack int ethtool_set_rxfh(struct net_device *dev,if (!rxfh_dev.rss_context) {/* indicate whether rxfh was set to default */ if (rxfh.indir_size ==0 ) - dev->priv_flags &= ~IFF_RXFH_CONFIGURED ; + dev->ethtool->rss_indir_user_size =0 ;else if (rxfh.indir_size !=ETH_RXFH_INDIR_NO_CHANGE ) - dev->priv_flags |=IFF_RXFH_CONFIGURED ; + dev->ethtool->rss_indir_user_size = dev_indir_size; }/* Update rss_ctx tracking */ if (rxfh_dev.rss_delete) {
@@ -1736,6 +1736,7 @@ static noinline_for_stack int ethtool_set_rxfh(struct net_device *dev, ctx->indir_configured = rxfh.indir_size && rxfh.indir_size !=ETH_RXFH_INDIR_NO_CHANGE ; + ctx->indir_user_size = dev_indir_size; }if (rxfh_dev.key) { memcpy(ethtool_rxfh_context_key(ctx), rxfh_dev.key,
net/ethtool/rss.c +16 -8
@@ -686,7 +686,7 @@ rss_set_prep_indir(struct net_device *dev, struct genl_info *info, *mod |=memcmp (rxfh->indir, data->indir_table, data->indir_size); -return 0 ; +return user_size; err_free:kfree (rxfh->indir);
@@ -833,6 +833,7 @@ ethnl_rss_set(struct ethnl_req_info *req_info, struct genl_info *info)struct nlattr **tb = info->attrs;struct rss_reply_data data = {};const struct ethtool_ops *ops; +u32 indir_user_size;int ret; ops = dev->ethtool_ops;
@@ -845,8 +846,9 @@ ethnl_rss_set(struct ethnl_req_info *req_info, struct genl_info *info) rxfh.rss_context = request->rss_context; ret =rss_set_prep_indir (dev, info, &data, &rxfh, &indir_reset, &mod); -if (ret) +if (ret <0 ) goto exit_clean_data; + indir_user_size = ret; indir_mod = !!tb[ETHTOOL_A_RSS_INDIR ]; rxfh.hfunc = data.hfunc;
@@ -889,12 +891,15 @@ ethnl_rss_set(struct ethnl_req_info *req_info, struct genl_info *info)if (ret) goto exit_unlock; -if (ctx) +if (ctx) {rss_set_ctx_update (ctx, tb, &data, &rxfh); -else if (indir_reset) - dev->priv_flags &= ~IFF_RXFH_CONFIGURED ; -else if (indir_mod) - dev->priv_flags |=IFF_RXFH_CONFIGURED ; +if (indir_user_size) + ctx->indir_user_size = indir_user_size; + }else if (indir_reset) { + dev->ethtool->rss_indir_user_size =0 ; + }else if (indir_mod) { + dev->ethtool->rss_indir_user_size = indir_user_size; + } exit_unlock:mutex_unlock (&dev->ethtool->rss_lock);
@@ -999,6 +1004,7 @@ int ethnl_rss_create_doit(struct sk_buff *skb, struct genl_info *info)const struct ethtool_ops *ops;struct rss_req_info req = {};struct net_device *dev; +u32 indir_user_size;struct sk_buff *rsp;void *hdr;u32 limit;
@@ -1035,8 +1041,9 @@ int ethnl_rss_create_doit(struct sk_buff *skb, struct genl_info *info) goto exit_ops; ret =rss_set_prep_indir (dev, info, &data, &rxfh, &indir_dflt, &mod); -if (ret) +if (ret <0 ) goto exit_clean_data; + indir_user_size = ret;ethnl_update_u8 (&rxfh.hfunc, tb[ETHTOOL_A_RSS_HFUNC ], &mod);
@@ -1080,6 +1087,7 @@ int ethnl_rss_create_doit(struct sk_buff *skb, struct genl_info *info)/* Store the config from rxfh to Xarray.. */ rss_set_ctx_update (ctx, tb, &data, &rxfh); + ctx->indir_user_size = indir_user_size;/* .. copy from Xarray to data. */ __rss_prepare_ctx (dev, &data, ctx);
Signature · 1 line
2.53.0
Jakub Kicinski
maintainerRe: [PATCH net-next v6 1/4] ethtool: Track user-provided RSS indirection table size
Quoted 10 lines · On Wed, 18 Mar 2026 13:25:58 +0100 Björn Töpel wrote:
net/ethtool/common.c +2
-0@@ -1204,6 +1204,26 @@ void ethtool_rxfh_context_lost(struct net_device *dev, u32 context_id) }EXPORT_SYMBOL (ethtool_rxfh_context_lost); +bool netif_is_rxfh_configured (const struct net_device *dev) +{
net/ethtool/common.c void ethtool_rxfh_context_lost(struct net_device *dev, u32 context_id) · line 1218
Quoted 10 lines
net/ethtool/common.c +10
-0+return dev->ethtool->rss_indir_user_size; +} +EXPORT_SYMBOL (netif_is_rxfh_configured); + +/** +* ethtool_rxfh_indir_clear - Clear user indirection table config +* @dev: network device +* +* Mark the default RSS context indirection table as unconfigured and +* send an %ETHTOOL_MSG_RSS_NTF notification.
If you're adding kdoc you should very clearly state this is function should only be called if the driver lost the ability to maintain the indirection table, usually after HW fault recovery reduced the max queue count.
net/ethtool/common.c void ethtool_rxfh_context_lost(struct net_device *dev, u32 context_id) · line 1220
Quoted 2 lines
net/ethtool/common.c +2
-0+*/ +void ethtool_rxfh_indir_clear (struct net_device *dev)
ethtool_rxfh_indir_lost() would be a better name, it would align with ethtool_rxfh_context_lost() right above the code you're adding, right?
net/ethtool/common.c void ethtool_rxfh_context_lost(struct net_device *dev, u32 context_id) · line 1221
Quoted 1 line
net/ethtool/common.c +1
-0+{
Let's add an error in here like the one we have in ethtool_rxfh_context_lost(), this should stop arbitrary use. And you can remove the local warning in bnxt if we do that.
Quoted 8 lines
net/ethtool/common.c +5
-0+ dev->ethtool->rss_indir_user_size =0 ; +ethtool_rss_notify (dev,ETHTOOL_MSG_RSS_NTF ,0 ); +} +EXPORT_SYMBOL (ethtool_rxfh_indir_clear); +enum ethtool_link_medium ethtool_str_to_medium (const char *str) {int i;
PATCH v6 2/4 ethtool: Add RSS indirection table resize helpers
In reply to Björn TöpelThe core locks ctx->indir_size when an RSS context is created. Some NICs (e.g. bnxt) change their indirection table size based on the channel count, because the hardware table is a shared resource. This forces drivers to reject channel changes when RSS contexts exist.
Add driver helpers to resize indirection tables:
ethtool_rxfh_indir_can_resize() checks whether the default context indirection table can be resized.
ethtool_rxfh_indir_resize() resizes the default context table in place. Folding (shrink) requires the table to be periodic at the new size; non-periodic tables are rejected. Unfolding (grow) replicates the existing pattern. Sizes must be multiples of each other.
ethtool_rxfh_ctxs_can_resize() validates all non-default RSS contexts can be resized.
ethtool_rxfh_ctxs_resize() applies the resize.
--- include/linux/ethtool.h | 6 ++ net/ethtool/common.c | 155 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 161 insertions(+)
include/linux/ethtool.h +6 -0
@@ -218,6 +218,12 @@ static inline u8 *ethtool_rxfh_context_key(struct ethtool_rxfh_context *ctx)void ethtool_rxfh_context_lost (struct net_device *dev,u32 context_id);void ethtool_rxfh_indir_clear (struct net_device *dev); +bool ethtool_rxfh_indir_can_resize (struct net_device *dev,const u32 *tbl, +u32 old_size,u32 new_size); +void ethtool_rxfh_indir_resize (struct net_device *dev,u32 *tbl, +u32 old_size,u32 new_size); +int ethtool_rxfh_ctxs_can_resize (struct net_device *dev,u32 new_indir_size); +void ethtool_rxfh_ctxs_resize (struct net_device *dev,u32 new_indir_size);struct link_mode_info {int speed;
net/ethtool/common.c +155 -0
161 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH v6 3/4 bnxt_en: Resize RSS contexts on channel count change
In reply to Björn Töpelbnxt_set_channels() previously rejected channel changes that alter the RSS table size when RSS contexts exist, because non-default context sizes were locked at creation.
Replace the rejection with the new resize helpers.
RSS table size only changes on P5 chips with older firmware; newer firmware always uses the largest table size.
--- .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 35 +++++++++++++++---- 1 file changed, 28 insertions(+), 7 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +28 -7
@@ -942,6 +942,7 @@ static int bnxt_set_channels(struct net_device *dev, {struct bnxt *bp =netdev_priv (dev);int req_tx_rings, req_rx_rings, tcs; +u32 new_tbl_size =0 , old_tbl_size;bool sh = false;int tx_xdp =0 ;int rc =0 ;
@@ -977,19 +978,33 @@ static int bnxt_set_channels(struct net_device *dev, tx_xdp = req_rx_rings; } -if (bnxt_get_nr_rss_ctxs (bp, req_rx_rings) != -bnxt_get_nr_rss_ctxs (bp, bp->rx_nr_rings) && - (netif_is_rxfh_configured (dev) || bp->num_rss_ctx)) { -netdev_warn (dev,"RSS table size change required, RSS table entries must be default (with no additional RSS contexts present) to proceed\n" ); -return -EINVAL ; - } - rc =bnxt_check_rings (bp, req_tx_rings, req_rx_rings, sh, tcs, tx_xdp);if (rc) {netdev_warn (dev,"Unable to allocate the requested rings\n" );return rc; } +/* RSS table size only changes on P5 chips with older firmware; +* newer firmware always uses the largest table size. +*/ +if (bnxt_get_nr_rss_ctxs (bp, req_rx_rings) != +bnxt_get_nr_rss_ctxs (bp, bp->rx_nr_rings)) { + new_tbl_size =bnxt_get_nr_rss_ctxs (bp, req_rx_rings) * +BNXT_RSS_TABLE_ENTRIES_P5 ; + old_tbl_size =bnxt_get_rxfh_indir_size (dev); + +if (!ethtool_rxfh_indir_can_resize (dev, bp->rss_indir_tbl, + old_tbl_size, + new_tbl_size)) { +netdev_warn (dev,"RSS table resize not possible\n" ); +return -EINVAL ; + } + + rc =ethtool_rxfh_ctxs_can_resize (dev, new_tbl_size); +if (rc) +return rc; + } +if (netif_running (dev)) {if (BNXT_PF (bp)) {/* TODO CHIMP_FW: Send message to all VF's
@@ -999,6 +1014,12 @@ static int bnxt_set_channels(struct net_device *dev,bnxt_close_nic (bp, true, false); } +if (new_tbl_size) { +ethtool_rxfh_indir_resize (dev, bp->rss_indir_tbl, + old_tbl_size, new_tbl_size); +ethtool_rxfh_ctxs_resize (dev, new_tbl_size); + } +if (sh) { bp->flags |=BNXT_FLAG_SHARED_RINGS ; bp->rx_nr_rings = channel->combined_count;
Signature · 1 line
2.53.0
PATCH v6 4/4 selftests: rss_drv: Add RSS indirection table resize tests
In reply to Björn TöpelAdd resize tests to rss_drv.py. Devices without dynamic table sizing are skipped via _require_dynamic_indir_size().
resize_periodic: set a periodic 4-entry table, shrink channels to fold, grow back to unfold. Check the exact pattern is preserved. Has main and non-default context variants.
resize_below_user_size_reject: send a periodic table with user_size between the big and small device table sizes. Verify that shrinking below user_size is rejected even though the table is periodic. Has main and non-default context variants.
resize_nonperiodic_reject: set a non-periodic table (equal N), verify that channel reduction is rejected.
resize_nonperiodic_no_corruption: verify a failed resize leaves both the indirection table contents and the channel count unchanged.
--- .../selftests/drivers/net/hw/rss_drv.py | 233 +++++++++++++++++- 1 file changed, 229 insertions(+), 4 deletions(-)
tools/testing/selftests/drivers/net/hw/rss_drv.py +229 -4
@@ -5,9 +5,9 @@ Driver-related behavior tests for RSS. """ -from lib.py import ksft_run, ksft_exit, ksft_ge -from lib.py import ksft_variants, KsftNamedVariant, KsftSkipEx -from lib.py import defer, ethtool +from lib.py import ksft_run, ksft_exit, ksft_eq, ksft_ge +from lib.py import ksft_variants, KsftNamedVariant, KsftSkipEx, ksft_raises +from lib.py import defer, ethtool, CmdExitFailure from lib.py import EthtoolFamily, NlError from lib.py import NetDrvEnv
@@ -45,6 +45,18 @@ def _maybe_create_context(cfg, create_context): return ctx_id +def _require_dynamic_indir_size(cfg, ch_max): + """Skip if the device does not dynamically size its indirection table.""" + ethtool(f"-X {cfg.ifname} default") + ethtool(f"-L {cfg.ifname} combined 2") + small = len(_get_rss(cfg)['rss-indirection-table']) + ethtool(f"-L {cfg.ifname} combined {ch_max}") + large = len(_get_rss(cfg)['rss-indirection-table']) + + if small == large: + raise KsftSkipEx("Device does not dynamically size indirection table") + + @ksft_variants([ KsftNamedVariant("main", False), KsftNamedVariant("ctx", True),
@@ -76,11 +88,224 @@ def indir_size_4x(cfg, create_context): _test_rss_indir_size(cfg, test_max, context=ctx_id) +@ksft_variants([ + KsftNamedVariant("main", False), + KsftNamedVariant("ctx", True), +]) +def resize_periodic(cfg, create_context): + """Test that a periodic indirection table survives channel changes. + + Set a non-default periodic table ([3, 2, 1, 0] x N) via netlink, + reduce channels to trigger a fold, then increase to trigger an + unfold. Using a reversed pattern (instead of [0, 1, 2, 3]) ensures + the test can distinguish a correct fold from a driver that silently + resets the table to defaults. Verify the exact pattern is preserved + and the size tracks the channel count. + """ + channels = cfg.ethnl.channels_get({'header': {'dev-index': cfg.ifindex}}) + ch_max = channels.get('combined-max', 0) + qcnt = channels['combined-count'] + + if ch_max < 4: + raise KsftSkipEx(f"Not enough queues for the test: max={ch_max}") + + defer(ethtool, f"-L {cfg.ifname} combined {qcnt}") + + _require_dynamic_indir_size(cfg, ch_max) + + ctx_id = _maybe_create_context(cfg, create_context) + + # Set a non-default periodic pattern via netlink. + # Send only 4 entries (user_size=4) so the kernel replicates it + # to fill the device table. This allows folding down to 4 entries. + rss = _get_rss(cfg, context=ctx_id) + orig_size = len(rss['rss-indirection-table']) + pattern = [3, 2, 1, 0] + req = {'header': {'dev-index': cfg.ifindex}, 'indir': pattern} + if ctx_id: + req['context'] = ctx_id + else: + defer(ethtool, f"-X {cfg.ifname} default") + cfg.ethnl.rss_set(req) + + # Shrink — should fold + ethtool(f"-L {cfg.ifname} combined 4") + rss = _get_rss(cfg, context=ctx_id) + indir = rss['rss-indirection-table'] + + ksft_ge(orig_size, len(indir), "Table did not shrink") + ksft_eq(indir, [3, 2, 1, 0] * (len(indir) // 4), + "Folded table has wrong pattern") + + # Grow back — should unfold + ethtool(f"-L {cfg.ifname} combined {ch_max}") + rss = _get_rss(cfg, context=ctx_id) + indir = rss['rss-indirection-table'] + + ksft_eq(len(indir), orig_size, "Table size not restored") + ksft_eq(indir, [3, 2, 1, 0] * (len(indir) // 4), + "Unfolded table has wrong pattern") + + +@ksft_variants([ + KsftNamedVariant("main", False), + KsftNamedVariant("ctx", True), +]) +def resize_below_user_size_reject(cfg, create_context): + """Test that shrinking below user_size is rejected. + + Send a table via netlink whose size (user_size) sits between + the small and large device table sizes. The table is periodic, + so folding would normally succeed, but the user_size floor must + prevent it. + """ + channels = cfg.ethnl.channels_get({'header': {'dev-index': cfg.ifindex}}) + ch_max = channels.get('combined-max', 0) + qcnt = channels['combined-count'] + + if ch_max < 4: + raise KsftSkipEx(f"Not enough queues for the test: max={ch_max}") + + defer(ethtool, f"-L {cfg.ifname} combined {qcnt}") + + _require_dynamic_indir_size(cfg, ch_max) + + ctx_id = _maybe_create_context(cfg, create_context) + + # Measure the table size at max channels + rss = _get_rss(cfg, context=ctx_id) + big_size = len(rss['rss-indirection-table']) + + # Measure the table size at reduced channels + ethtool(f"-L {cfg.ifname} combined 4") + rss = _get_rss(cfg, context=ctx_id) + small_size = len(rss['rss-indirection-table']) + ethtool(f"-L {cfg.ifname} combined {ch_max}") + + if small_size >= big_size: + raise KsftSkipEx("Table did not shrink at reduced channels") + + # Find a user_size + user_size = None + for div in [2, 4]: + candidate = big_size // div + if candidate > small_size and big_size % candidate == 0: + user_size = candidate + break + if user_size is None: + raise KsftSkipEx("No suitable user_size between small and big table") + + # Send a periodic sub-table of exactly user_size entries. + # Pattern safe for 4 channels. + pattern = [0, 1, 2, 3] * (user_size // 4) + if len(pattern) != user_size: + raise KsftSkipEx(f"user_size ({user_size}) not divisible by 4") + req = {'header': {'dev-index': cfg.ifindex}, 'indir': pattern} + if ctx_id: + req['context'] = ctx_id + else: + defer(ethtool, f"-X {cfg.ifname} default") + cfg.ethnl.rss_set(req) + + # Shrink channels — table would go to small_size < user_size. + # The table is periodic so folding would work, but user_size + # floor must reject it. + with ksft_raises(CmdExitFailure): + ethtool(f"-L {cfg.ifname} combined 4") + + +@ksft_variants([ + KsftNamedVariant("main", False), + KsftNamedVariant("ctx", True), +]) +def resize_nonperiodic_reject(cfg, create_context): + """Test that a non-periodic table blocks channel reduction. + + Set equal weight across all queues so the table is not periodic + at any smaller size, then verify channel reduction is rejected. + An additional context with a periodic table is created to verify + that validation catches the non-periodic one even when others + are fine. + """ + channels = cfg.ethnl.channels_get({'header': {'dev-index': cfg.ifindex}}) + ch_max = channels.get('combined-max', 0) + qcnt = channels['combined-count'] + + if ch_max < 4: + raise KsftSkipEx(f"Not enough queues for the test: max={ch_max}") + + defer(ethtool, f"-L {cfg.ifname} combined {qcnt}") + + _require_dynamic_indir_size(cfg, ch_max) + + ctx_id = _maybe_create_context(cfg, create_context) + ctx_ref = f"context {ctx_id}" if ctx_id else "" + + # Create an extra context with a periodic (foldable) table so that + # the validation must iterate all contexts to find the bad one. + extra_ctx = _maybe_create_context(cfg, True) + ethtool(f"-X {cfg.ifname} context {extra_ctx} equal 2") + + ethtool(f"-X {cfg.ifname} {ctx_ref} equal {ch_max}") + if not create_context: + defer(ethtool, f"-X {cfg.ifname} default") + + with ksft_raises(CmdExitFailure): + ethtool(f"-L {cfg.ifname} combined 2") + + +@ksft_variants([ + KsftNamedVariant("main", False), + KsftNamedVariant("ctx", True), +]) +def resize_nonperiodic_no_corruption(cfg, create_context): + """Test that a failed resize does not corrupt table or channel count. + + Set a non-periodic table, attempt a channel reduction (which must + fail), then verify both the indirection table contents and the + channel count are unchanged. + """ + channels = cfg.ethnl.channels_get({'header': {'dev-index': cfg.ifindex}}) + ch_max = channels.get('combined-max', 0) + qcnt = channels['combined-count'] + + if ch_max < 4: + raise KsftSkipEx(f"Not enough queues for the test: max={ch_max}") + + defer(ethtool, f"-L {cfg.ifname} combined {qcnt}") + + _require_dynamic_indir_size(cfg, ch_max) + + ctx_id = _maybe_create_context(cfg, create_context) + ctx_ref = f"context {ctx_id}" if ctx_id else "" + + ethtool(f"-X {cfg.ifname} {ctx_ref} equal {ch_max}") + if not create_context: + defer(ethtool, f"-X {cfg.ifname} default") + + rss_before = _get_rss(cfg, context=ctx_id) + + with ksft_raises(CmdExitFailure): + ethtool(f"-L {cfg.ifname} combined 2") + + rss_after = _get_rss(cfg, context=ctx_id) + ksft_eq(rss_after['rss-indirection-table'], + rss_before['rss-indirection-table'], + "Indirection table corrupted after failed resize") + + channels = cfg.ethnl.channels_get({'header': {'dev-index': cfg.ifindex}}) + ksft_eq(channels['combined-count'], ch_max, + "Channel count changed after failed resize") + + def main() -> None: """ Ksft boiler plate main """ with NetDrvEnv(__file__) as cfg: cfg.ethnl = EthtoolFamily() - ksft_run([indir_size_4x], args=(cfg, )) + ksft_run([indir_size_4x, resize_periodic, + resize_below_user_size_reject, + resize_nonperiodic_reject, + resize_nonperiodic_no_corruption], args=(cfg, )) ksft_exit()
Signature · 1 line
2.53.0