mlxsw: Various updates
Subsystems (5)
Ido Schimmel
maintainercover mlxsw: Various updates
Patches #1-#3 add missing topology diagrams in selftests and perform small cleanups.
Patches #4-#5 make small adjustments in QoS configuration. See detailed description in the commit messages.
Patches #6-#8 reduce the number of background EMAD transactions. The driver periodically queries the device (via EMAD transactions) about updates that cannot happen in certain situations. This can negatively impact the latency of time critical transactions, as the device is busy processing other transactions.
Before:
# perf stat -a -e devlink:devlink_hwmsg -- sleep 10
Performance counter stats for 'system wide':
452 devlink:devlink_hwmsg
10.009736160 seconds time elapsedAfter:
# perf stat -a -e devlink:devlink_hwmsg -- sleep 10
Performance counter stats for 'system wide':
0 devlink:devlink_hwmsg
10.001726333 seconds time elapsedIdo Schimmel (3):
mlxsw: spectrum_acl: Do not report activity for multicast routes
mlxsw: spectrum_switchdev: Only query FDB notifications when necessary
mlxsw: spectrum_router: Only query neighbour activity when necessary
Petr Machata (5):
selftests: mlxsw: bail_on_lldpad before installing the cleanup trap
selftests: router_vid_1: Add a diagram, fix coding style
selftests: router.sh: Add a diagram
mlxsw: spectrum_dcb: Do not warn about priority changes
mlxsw: Treat LLDP packets as control
.../mellanox/mlxsw/spectrum_acl_tcam.c | 5 ++- .../ethernet/mellanox/mlxsw/spectrum_dcb.c | 13 -------- .../ethernet/mellanox/mlxsw/spectrum_router.c | 6 ++++ .../ethernet/mellanox/mlxsw/spectrum_router.h | 1 + .../mellanox/mlxsw/spectrum_switchdev.c | 31 ++++++++++++------- .../ethernet/mellanox/mlxsw/spectrum_trap.c | 2 +- .../drivers/net/mlxsw/qos_headroom.sh | 4 +-- .../selftests/drivers/net/mlxsw/qos_pfc.sh | 4 +-- .../drivers/net/mlxsw/sch_red_ets.sh | 5 ++- .../drivers/net/mlxsw/sch_red_root.sh | 5 ++- .../selftests/net/forwarding/router.sh | 18 +++++++++++ .../selftests/net/forwarding/router_vid_1.sh | 27 +++++++++++++++- 12 files changed, 82 insertions(+), 39 deletions(-)
Signature · 1 line
2.35.1
Ido Schimmel
maintainerPATCH 1/8 selftests: mlxsw: bail_on_lldpad before installing the cleanup trap
From: Petr Machata <[email protected]>
A number of mlxsw-specific QoS tests use manual QoS DCB management. As such, they need to make sure lldpad is not running, because it would override the configuration the test has applied using other tools. To that end, these selftests invoke the bail_on_lldpad() helper, which terminates the selftest if th lldpad is running.
Some of these tests however first install the bash exit trap, which invokes a cleanup() at the test exit. If bail_on_lldpad() has terminated the script even before the setup part was run, the cleanup part will be very confused.
Therefore make sure bail_on_lldpad() is invoked before the cleanup is registered.
While there are still edge cases where the user terminates the script before the setup was fully done, this takes care of a common situation where the cleanup would be invoked in an inconsistent state.
--- tools/testing/selftests/drivers/net/mlxsw/qos_headroom.sh | 4 ++-- tools/testing/selftests/drivers/net/mlxsw/qos_pfc.sh | 4 ++-- tools/testing/selftests/drivers/net/mlxsw/sch_red_ets.sh | 5 ++--- tools/testing/selftests/drivers/net/mlxsw/sch_red_root.sh | 5 ++--- 4 files changed, 8 insertions(+), 10 deletions(-)
tools/testing/selftests/drivers/net/mlxsw/qos_headroom.sh +2 -2
@@ -371,9 +371,9 @@ test_tc_int_buf() tc qdisc delete dev $swp root } -trap cleanup EXIT - bail_on_lldpad + +trap cleanup EXIT setup_wait tests_run
tools/testing/selftests/drivers/net/mlxsw/qos_pfc.sh +2 -2
@@ -393,9 +393,9 @@ test_qos_pfc() log_test "PFC" } -trap cleanup EXIT - bail_on_lldpad + +trap cleanup EXIT setup_prepare setup_wait tests_run
tools/testing/selftests/drivers/net/mlxsw/sch_red_ets.sh +2 -3
@@ -166,12 +166,11 @@ ecn_mirror_test() uninstall_qdisc } -trap cleanup EXIT +bail_on_lldpad +trap cleanup EXIT setup_prepare setup_wait - -bail_on_lldpad tests_run exit $EXIT_STATUS
tools/testing/selftests/drivers/net/mlxsw/sch_red_root.sh +2 -3
@@ -73,12 +73,11 @@ red_mirror_test() uninstall_qdisc } -trap cleanup EXIT +bail_on_lldpad +trap cleanup EXIT setup_prepare setup_wait - -bail_on_lldpad tests_run exit $EXIT_STATUS
Signature · 1 line
2.35.1
Ido Schimmel
maintainerPATCH 2/8 selftests: router_vid_1: Add a diagram, fix coding style
In reply to Ido SchimmelFrom: Petr Machata <[email protected]>
It is customary for selftests to have a comment with a topology diagram, which serves to illustrate the situation in which the test is done. This selftest lacks it. Add it.
While at it, fix the list of tests so that the test names are enumerated one at a line.
--- .../selftests/net/forwarding/router_vid_1.sh | 27 ++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
tools/testing/selftests/net/forwarding/router_vid_1.sh +26 -1
@@ -1,7 +1,32 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0 -ALL_TESTS="ping_ipv4 ping_ipv6" +# +--------------------+ +----------------------+ +# | H1 | | H2 | +# | | | | +# | $h1.1 + | | + $h2.1 | +# | 192.0.2.2/24 | | | | 198.51.100.2/24 | +# | 2001:db8:1::2/64 | | | | 2001:db8:2::2/64 | +# | | | | | | +# | $h1 + | | + $h2 | +# | | | | | | +# +------------------|-+ +-|--------------------+ +# | | +# +------------------|-------------------------|--------------------+ +# | SW | | | +# | | | | +# | $rp1 + + $rp2 | +# | | | | +# | $rp1.1 + + $rp2.1 | +# | 192.0.2.1/24 198.51.100.1/24 | +# | 2001:db8:1::1/64 2001:db8:2::1/64 | +# | | +# +-----------------------------------------------------------------+ + +ALL_TESTS=" + ping_ipv4 + ping_ipv6 +" NUM_NETIFS=4 source lib.sh
Signature · 1 line
2.35.1
Ido Schimmel
maintainerPATCH 3/8 selftests: router.sh: Add a diagram
In reply to Ido SchimmelFrom: Petr Machata <[email protected]>
It is customary for selftests to have a comment with a topology diagram, which serves to illustrate the situation in which the test is done. This selftest lacks it. Add it.
--- .../testing/selftests/net/forwarding/router.sh | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
tools/testing/selftests/net/forwarding/router.sh +18 -0
@@ -1,6 +1,24 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0 +# +--------------------+ +----------------------+ +# | H1 | | H2 | +# | | | | +# | $h1 + | | + $h2 | +# | 192.0.2.2/24 | | | | 198.51.100.2/24 | +# | 2001:db8:1::2/64 | | | | 2001:db8:2::2/64 | +# | | | | | | +# +------------------|-+ +-|--------------------+ +# | | +# +------------------|-------------------------|--------------------+ +# | SW | | | +# | | | | +# | $rp1 + + $rp2 | +# | 192.0.2.1/24 198.51.100.1/24 | +# | 2001:db8:1::1/64 2001:db8:2::1/64 | +# | | +# +-----------------------------------------------------------------+ + ALL_TESTS=" ping_ipv4 ping_ipv6
Signature · 1 line
2.35.1
Ido Schimmel
maintainerPATCH 4/8 mlxsw: spectrum_dcb: Do not warn about priority changes
In reply to Ido SchimmelFrom: Petr Machata <[email protected]>
The idea behind the warnings is that the user would get warned in case when more than one priority is configured for a given DSCP value on a netdevice.
The warning is currently wrong, because dcb_ieee_getapp_mask() returns the first matching entry, not all of them, and the warning will then claim that some priority is "current", when in fact it is not.
But more importantly, the warning is misleading in general. Consider the following commands:
# dcb app flush dev swp19 dscp-prio # dcb app add dev swp19 dscp-prio 24:3 # dcb app replace dev swp19 dscp-prio 24:2
The last command will issue the following warning:
mlxsw_spectrum3 0000:07:00.0 swp19: Ignoring new priority 2 for DSCP 24 in favor of current value of 3
The reason is that the "replace" command works by first adding the new value, and then removing all old values. This is the only way to make the replacement without causing the traffic to be prioritized to whatever the chip defaults to. The warning is issued in response to adding the new priority, and then no warning is shown when the old priority is removed. The upshot is that the canonical way to change traffic prioritization always produces a warning about ignoring the new priority, but what gets configured is in fact what the user intended.
An option to just emit warning every time that the prioritization changes just to make it clear that it happened is obviously unsatisfactory.
Therefore, in this patch, remove the warnings.
--- drivers/net/ethernet/mellanox/mlxsw/spectrum_dcb.c | 13 ------------- 1 file changed, 13 deletions(-)
drivers/net/ethernet/mellanox/mlxsw/spectrum_dcb.c +0 -13
@@ -168,8 +168,6 @@ static int mlxsw_sp_dcbnl_ieee_setets(struct net_device *dev,static int mlxsw_sp_dcbnl_app_validate (struct net_device *dev,struct dcb_app *app) { -int prio; -if (app->priority >=IEEE_8021QAZ_MAX_TCS ) {netdev_err (dev,"APP entry with priority value %u is invalid\n" , app->priority);
@@ -183,17 +181,6 @@ static int mlxsw_sp_dcbnl_app_validate(struct net_device *dev, app->protocol);return -EINVAL ; } - -/* Warn about any DSCP APP entries with the same PID. */ - prio =fls (dcb_ieee_getapp_mask (dev, app)); -if (prio--) { -if (prio < app->priority) -netdev_warn (dev,"Choosing priority %d for DSCP %d in favor of previously-active value of %d\n" , - app->priority, app->protocol, prio); -else if (prio > app->priority) -netdev_warn (dev,"Ignoring new priority %d for DSCP %d in favor of current value of %d\n" , - app->priority, app->protocol, prio); - }break ;case IEEE_8021QAZ_APP_SEL_ETHERTYPE :
Signature · 1 line
2.35.1
Ido Schimmel
maintainerPATCH 5/8 mlxsw: Treat LLDP packets as control
In reply to Ido SchimmelFrom: Petr Machata <[email protected]>
When trapping packets for on-CPU processing, Spectrum machines differentiate between control and non-control traps. Traffic trapped through non-control traps is treated as data and kept in shared buffer in pools 0-4. Traffic trapped through control traps is kept in the dedicated control buffer 9. The advantage of marking traps as control is that pressure in the data plane does not prevent the control traffic to be processed.
When the LLDP trap was introduced, it was marked as a control trap. But then in commit aed4b5721143 ("mlxsw: spectrum: PTP: Hook into packet receive path"), PTP traps were introduced. Because Ethernet-encapsulated PTP packets look to the Spectrum-1 ASIC as LLDP traffic and are trapped under the LLDP trap, this trap was reconfigured as non-control, in sync with the PTP traps.
There is however no requirement that PTP traffic be handled as data. Besides, the usual encapsulation for PTP traffic is UDP, not bare Ethernet, and that is in deployments that even need PTP, which is far less common than LLDP. This is reflected by the default policer, which was not bumped up to the 19Kpps / 24Kpps that is the expected load of a PTP-enabled Spectrum-1 switch.
Marking of LLDP trap as non-control was therefore probably misguided. In this patch, change it back to control.
--- drivers/net/ethernet/mellanox/mlxsw/spectrum_trap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlxsw/spectrum_trap.c +1 -1
@@ -864,7 +864,7 @@ static const struct mlxsw_sp_trap_item mlxsw_sp_trap_items_arr[] = { .trap = MLXSW_SP_TRAP_CONTROL(LLDP, LLDP, TRAP), .listeners_arr = {MLXSW_RXL (mlxsw_sp_rx_ptp_listener,LLDP ,TRAP_TO_CPU , - false, SP_LLDP, DISCARD), + true, SP_LLDP, DISCARD), }, }, {
Signature · 1 line
2.35.1
Ido Schimmel
maintainerPATCH 6/8 mlxsw: spectrum_acl: Do not report activity for multicast routes
In reply to Ido SchimmelThe driver periodically queries the device for activity of ACL rules in order to report it to tc upon 'FLOW_CLS_STATS'.
In Spectrum-2 and later ASICs, multicast routes are programmed as ACL rules, but unlike rules installed by tc, their activity is of no interest.
Avoid unnecessary activity query for such rules by always reporting them as inactive.
--- drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.c +2 -3
@@ -1827,10 +1827,9 @@ static int mlxsw_sp_acl_tcam_mr_rule_activity_get(struct mlxsw_sp *mlxsw_sp, void *rule_priv, bool *activity) { - struct mlxsw_sp_acl_tcam_mr_rule *rule = rule_priv; + *activity = false; -return mlxsw_sp_acl_tcam_ventry_activity_get (mlxsw_sp, &rule->ventry, - activity); +return 0 ; } static const struct mlxsw_sp_acl_profile_ops mlxsw_sp_acl_tcam_mr_ops = {
Signature · 1 line
2.35.1
Ido Schimmel
maintainerPATCH 7/8 mlxsw: spectrum_switchdev: Only query FDB notifications when necessary
In reply to Ido SchimmelThe driver periodically queries the device for FDB notifications (e.g., learned, aged-out) in order to update the bridge driver. These notifications can only be generated when bridges are offloaded to the device.
Avoid unnecessary queries by starting to query upon installation of the first bridge and stop querying upon removal of the last bridge.
--- .../mellanox/mlxsw/spectrum_switchdev.c | 31 ++++++++++++------- 1 file changed, 20 insertions(+), 11 deletions(-)
drivers/net/ethernet/mellanox/mlxsw/spectrum_switchdev.c +20 -11
@@ -207,6 +207,16 @@ static void mlxsw_sp_bridge_device_vxlan_fini(struct mlxsw_sp_bridge *bridge, } } +static void mlxsw_sp_fdb_notify_work_schedule (struct mlxsw_sp *mlxsw_sp, +bool no_delay) +{ +struct mlxsw_sp_bridge *bridge = mlxsw_sp->bridge; +unsigned int interval = no_delay ?0 : bridge->fdb_notify.interval; + +mlxsw_core_schedule_dw (&bridge->fdb_notify.dw, +msecs_to_jiffies (interval)); +} +static struct mlxsw_sp_bridge_device * mlxsw_sp_bridge_device_create(struct mlxsw_sp_bridge *bridge, struct net_device *br_dev,
@@ -245,6 +255,8 @@ mlxsw_sp_bridge_device_create(struct mlxsw_sp_bridge *bridge, bridge_device->ops = bridge->bridge_8021d_ops; }INIT_LIST_HEAD (&bridge_device->mids_list); +if (list_empty (&bridge->bridges_list)) +mlxsw_sp_fdb_notify_work_schedule (bridge->mlxsw_sp, false);list_add (&bridge_device->list, &bridge->bridges_list);/* It is possible we already have VXLAN devices enslaved to the bridge.
@@ -273,6 +285,8 @@ mlxsw_sp_bridge_device_destroy(struct mlxsw_sp_bridge *bridge,mlxsw_sp_bridge_device_rifs_destroy (bridge->mlxsw_sp, bridge_device->dev);list_del (&bridge_device->list); +if (list_empty (&bridge->bridges_list)) +cancel_delayed_work (&bridge->fdb_notify.dw);if (bridge_device->vlan_enabled) bridge->vlan_enabled_exists = false;WARN_ON (!list_empty (&bridge_device->ports_list));
@@ -2886,22 +2900,13 @@ static void mlxsw_sp_fdb_notify_rec_process(struct mlxsw_sp *mlxsw_sp, } } -static void mlxsw_sp_fdb_notify_work_schedule (struct mlxsw_sp *mlxsw_sp, -bool no_delay) -{ -struct mlxsw_sp_bridge *bridge = mlxsw_sp->bridge; -unsigned int interval = no_delay ?0 : bridge->fdb_notify.interval; - -mlxsw_core_schedule_dw (&bridge->fdb_notify.dw, -msecs_to_jiffies (interval)); -} -#define MLXSW_SP_FDB_SFN_QUERIES_PER_SESSION 10static void mlxsw_sp_fdb_notify_work (struct work_struct *work) {struct mlxsw_sp_bridge *bridge;struct mlxsw_sp *mlxsw_sp; +bool reschedule = false;char *sfn_pl;int queries;u8 num_rec;
@@ -2916,6 +2921,9 @@ static void mlxsw_sp_fdb_notify_work(struct work_struct *work) mlxsw_sp = bridge->mlxsw_sp;rtnl_lock (); +if (list_empty (&bridge->bridges_list)) + goto out; + reschedule = true; queries =MLXSW_SP_FDB_SFN_QUERIES_PER_SESSION ;while (queries >0 ) {mlxsw_reg_sfn_pack (sfn_pl);
@@ -2935,6 +2943,8 @@ static void mlxsw_sp_fdb_notify_work(struct work_struct *work) out:rtnl_unlock ();kfree (sfn_pl); +if (!reschedule) +return ;mlxsw_sp_fdb_notify_work_schedule (mlxsw_sp, !queries); }
@@ -3665,7 +3675,6 @@ static int mlxsw_sp_fdb_init(struct mlxsw_sp *mlxsw_sp)INIT_DELAYED_WORK (&bridge->fdb_notify.dw, mlxsw_sp_fdb_notify_work); bridge->fdb_notify.interval =MLXSW_SP_DEFAULT_LEARNING_INTERVAL ; -mlxsw_sp_fdb_notify_work_schedule (mlxsw_sp, false);return 0 ; err_register_switchdev_blocking_notifier:
Signature · 1 line
2.35.1
Ido Schimmel
maintainerPATCH 8/8 mlxsw: spectrum_router: Only query neighbour activity when necessary
In reply to Ido SchimmelThe driver periodically queries the device for activity of neighbour entries in order to report it to the kernel's neighbour table.
Avoid unnecessary activity query when no neighbours are installed. Use an atomic variable to track the number of neighbours, as it is read without any locks.
--- drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c | 6 ++++++ drivers/net/ethernet/mellanox/mlxsw/spectrum_router.h | 1 + 2 files changed, 7 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c +6 -0
@@ -2360,6 +2360,7 @@ mlxsw_sp_neigh_entry_create(struct mlxsw_sp *mlxsw_sp, struct neighbour *n) goto err_neigh_entry_insert;mlxsw_sp_neigh_counter_alloc (mlxsw_sp, neigh_entry); +atomic_inc (&mlxsw_sp->router->neighs_update.neigh_count);list_add (&neigh_entry->rif_list_node, &rif->neigh_list);return neigh_entry;
@@ -2374,6 +2375,7 @@ mlxsw_sp_neigh_entry_destroy(struct mlxsw_sp *mlxsw_sp, struct mlxsw_sp_neigh_entry *neigh_entry) {list_del (&neigh_entry->rif_list_node); +atomic_dec (&mlxsw_sp->router->neighs_update.neigh_count);mlxsw_sp_neigh_counter_free (mlxsw_sp, neigh_entry);mlxsw_sp_neigh_entry_remove (mlxsw_sp, neigh_entry);mlxsw_sp_neigh_entry_free (neigh_entry);
@@ -2571,6 +2573,9 @@ static int mlxsw_sp_router_neighs_update_rauhtd(struct mlxsw_sp *mlxsw_sp)char *rauhtd_pl;int err; +if (!atomic_read (&mlxsw_sp->router->neighs_update.neigh_count)) +return 0 ; + rauhtd_pl =kmalloc (MLXSW_REG_RAUHTD_LEN ,GFP_KERNEL );if (!rauhtd_pl)return -ENOMEM ;
@@ -2950,6 +2955,7 @@ static int mlxsw_sp_neigh_init(struct mlxsw_sp *mlxsw_sp) mlxsw_sp_router_neighs_update_work);INIT_DELAYED_WORK (&mlxsw_sp->router->nexthop_probe_dw, mlxsw_sp_router_probe_unresolved_nexthops); +atomic_set (&mlxsw_sp->router->neighs_update.neigh_count,0 );mlxsw_core_schedule_dw (&mlxsw_sp->router->neighs_update.dw,0 );mlxsw_core_schedule_dw (&mlxsw_sp->router->nexthop_probe_dw,0 );return 0 ;
drivers/net/ethernet/mellanox/mlxsw/spectrum_router.h +1 -0
@@ -56,6 +56,7 @@ struct mlxsw_sp_router {struct {struct delayed_work dw;unsigned long interval;/* ms */ +atomic_t neigh_count; } neighs_update;struct delayed_work nexthop_probe_dw;#define MLXSW_SP_UNRESOLVED_NH_PROBE_INTERVAL 5000/* ms */
Signature · 1 line
2.35.1
This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:
Quoted 6 lines · On Wed, 4 May 2022 09:29:01 +0300 you wrote:
Patches #1-#3 add missing topology diagrams in selftests and perform small cleanups.
Patches #4-#5 make small adjustments in QoS configuration. See detailed description in the commit messages.
Quoted 7 lines
Patches #6-#8 reduce the number of background EMAD transactions. The driver periodically queries the device (via EMAD transactions) about updates that cannot happen in certain situations. This can negatively impact the latency of time critical transactions, as the device is busy processing other transactions.
[...]
Here is the summary with links:
- [net-next,1/8] selftests: mlxsw: bail_on_lldpad before installing the cleanup trap
https://git.kernel.org/netdev/net-next/c/18d2c710e5df - [net-next,2/8] selftests: router_vid_1: Add a diagram, fix coding style https://git.kernel.org/netdev/net-next/c/5ade50e2df2b - [net-next,3/8] selftests: router.sh: Add a diagram https://git.kernel.org/netdev/net-next/c/faa7521add89 - [net-next,4/8] mlxsw: spectrum_dcb: Do not warn about priority changes https://git.kernel.org/netdev/net-next/c/b6b584562cbe - [net-next,5/8] mlxsw: Treat LLDP packets as control https://git.kernel.org/netdev/net-next/c/0106668cd2f9 - [net-next,6/8] mlxsw: spectrum_acl: Do not report activity for multicast routes https://git.kernel.org/netdev/net-next/c/d1314096fbe9 - [net-next,7/8] mlxsw: spectrum_switchdev: Only query FDB notifications when necessary https://git.kernel.org/netdev/net-next/c/b8950003849d - [net-next,8/8] mlxsw: spectrum_router: Only query neighbour activity when necessary https://git.kernel.org/netdev/net-next/c/cff9437605d5
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html