Support bandwidth clamping in mana using net shapers
Subsystems (5)
v2 cover Support bandwidth clamping in mana using net shapers
This patchset introduces hardware-backed bandwidth rate limiting for MANA NICs via the net_shaper_ops interface, enabling efficient and fine-grained traffic shaping directly on the device.
Previously, MANA lacked a mechanism for user-configurable bandwidth control. With this addition, users can now configure shaping parameters, allowing better traffic management and performance isolation.
The implementation includes the net_shaper_ops callbacks in the MANA driver and supports one shaper per vport. Add shaping support via mana_set_bw_clamp(), allowing the configuration of bandwidth rates in 100 Mbps increments (minimum 100 Mbps). The driver validates input and rejects unsupported values. On failure, it restores the previous configuration which is queried using mana_query_link_cfg() or retains the current state.
To prevent potential deadlocks introduced by net_shaper_ops, switch to _locked variants of NAPI APIs when netdevops_lock is held during VF setup and teardown.
Also, Add support for ethtool get_link_ksettings to report the maximum link speed supported by the SKU in mbps.
These APIs when invoked on hardware that are older or that do not support these APIs, the speed would be reported as UNKNOWN and the net-shaper calls to set speed would fail.
---
Changes in v2:
* Rebase to latest net-next branch.
* Edit commit message in
"net: mana: Fix potential deadlocks in mana napi ops".
* Use netdev_lock_ops_to_full() instead of explicit if..else statements for napi APIs.
* Define GDMA_STATUS_CMD_UNSUPPORTED for unsupported HWC status code.
---
Erni Sri Satya Vennela (4):
net: mana: Fix potential deadlocks in mana napi ops
net: mana: Add support for net_shaper_ops
net: mana: Add speed support in mana_get_link_ksettings
net: mana: Handle unsupported HWC commands
.../net/ethernet/microsoft/mana/hw_channel.c | 4 + drivers/net/ethernet/microsoft/mana/mana_en.c | 197 +++++++++++++++++- .../ethernet/microsoft/mana/mana_ethtool.c | 6 + include/net/mana/gdma.h | 1 + include/net/mana/mana.h | 42 ++++ 5 files changed, 241 insertions(+), 9 deletions(-)
Signature · 1 line
2.34.1
PATCH v2 1/4 net: mana: Fix potential deadlocks in mana napi ops
When net_shaper_ops are enabled for MANA, netdev_ops_lock becomes active.
MANA VF setup/teardown by netvsc follows this call chain:
netvsc_vf_setup()
dev_change_flags()
...
__dev_open() OR __dev_close()dev_change_flags() holds the netdev mutex via netdev_lock_ops.
Meanwhile, mana_create_txq() and mana_create_rxq() in mana_open() path call NAPI APIs (netif_napi_add_tx(), netif_napi_add_weight(), napi_enable()), which also try to acquire the same lock, risking deadlock.
Similarly in the teardown path (mana_close()), netif_napi_disable() and netif_napi_del(), contend for the same lock.
Switch to the _locked variants of these APIs to avoid deadlocks when the netdev_ops_lock is held.
---
Changes in v2:
* Use netdev_lock_ops_to_full() instead of if...else statements for napi APIs.
* Edit commit message.
--- drivers/net/ethernet/microsoft/mana/mana_en.c | 30 +++++++++++++------ 1 file changed, 21 insertions(+), 9 deletions(-)
drivers/net/ethernet/microsoft/mana/mana_en.c +19 -9
@@ -1912,8 +1912,11 @@ static void mana_destroy_txq(struct mana_port_context *apc) napi = &apc->tx_qp[i].tx_cq.napi;if (apc->tx_qp[i].txq.napi_initialized) {napi_synchronize (napi); -napi_disable (napi); -netif_napi_del (napi); +netdev_lock_ops_to_full (napi->dev); +napi_disable_locked (napi); +netif_napi_del_locked (napi); +netdev_unlock_full_to_ops (napi->dev); apc->tx_qp[i].txq.napi_initialized = false; }mana_destroy_wq_obj (apc,GDMA_SQ , apc->tx_qp[i].tx_object);
@@ -2065,8 +2068,11 @@ static int mana_create_txq(struct mana_port_context *apc,mana_create_txq_debugfs (apc, i); -netif_napi_add_tx (net, &cq->napi, mana_poll); -napi_enable (&cq->napi); +set_bit (NAPI_STATE_NO_BUSY_POLL , &cq->napi.state); +netdev_lock_ops_to_full (net); +netif_napi_add_locked (net, &cq->napi, mana_poll); +napi_enable_locked (&cq->napi); +netdev_unlock_full_to_ops (net); txq->napi_initialized = true;mana_gd_ring_cq (cq->gdma_cq,SET_ARM_BIT );
@@ -2102,9 +2108,11 @@ static void mana_destroy_rxq(struct mana_port_context *apc,if (napi_initialized) {napi_synchronize (napi); -napi_disable (napi); - -netif_napi_del (napi); +netdev_lock_ops_to_full (napi->dev); +napi_disable_locked (napi); +netif_napi_del_locked (napi); +netdev_unlock_full_to_ops (napi->dev); }xdp_rxq_info_unreg (&rxq->xdp_rxq);
@@ -2355,14 +2363,18 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc, gc->cq_table[cq->gdma_id] = cq->gdma_cq; -netif_napi_add_weight (ndev, &cq->napi, mana_poll,1 ); +netdev_lock_ops_to_full (ndev); +netif_napi_add_weight_locked (ndev, &cq->napi, mana_poll,1 ); +netdev_unlock_full_to_ops (ndev);WARN_ON (xdp_rxq_info_reg (&rxq->xdp_rxq, ndev, rxq_idx, cq->napi.napi_id));WARN_ON (xdp_rxq_info_reg_mem_model (&rxq->xdp_rxq,MEM_TYPE_PAGE_POOL , rxq->page_pool)); -napi_enable (&cq->napi); +netdev_lock_ops_to_full (ndev); +napi_enable_locked (&cq->napi); +netdev_unlock_full_to_ops (ndev);mana_gd_ring_cq (cq->gdma_cq,SET_ARM_BIT ); out:
Signature · 1 line
2.34.1
RE: [PATCH net-next v2 1/4] net: mana: Fix potential deadlocks in mana napi ops
Quoted 24 lines
Subject: [PATCH net-next v2 1/4] net: mana: Fix potential deadlocks in mana napi ops
When
net_shaper_opsare enabled for MANA,netdev_ops_lockbecomes active.MANA VF setup/teardown by netvsc follows this call chain:
netvsc_vf_setup()dev_change_flags() ... __dev_open() OR __dev_close()
dev_change_flags()holds the netdev mutex vianetdev_lock_ops.Meanwhile,
mana_create_txq()andmana_create_rxq()inmana_open()path call NAPI APIs (netif_napi_add_tx(),netif_napi_add_weight(),napi_enable()), which also try to acquire the same lock, risking deadlock.Similarly in the teardown path (
mana_close()),netif_napi_disable()andnetif_napi_del(), contend for the same lock.Switch to the
_lockedvariants of these APIs to avoid deadlocks when thenetdev_ops_lockis held.
Quoted 6 lines
Fixes d4c22ec680c8 ("net: hold netdev instance lock duringndo_open/ndo_stop")
Signed-off-by Erni Sri Satya Vennela <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>Reviewed-by Shradha Gupta <[email protected]>Reviewed-by Saurabh Singh Sengar <[email protected]>
Quoted 82 lines
---
Changes in v2:
* Usenetdev_lock_ops_to_full()instead of if...else statements for napi APIs.
* Edit commit message.--- drivers/net/ethernet/microsoft/mana/mana_en.c | 30 +++++++++++++------ 1 file changed, 21 insertions(+), 9 deletions(-)drivers/net/ethernet/microsoft/mana/mana_en.c +4
-2@@ -1912,8 +1912,11 @@ static void mana_destroy_txq(struct mana_port_context *apc) napi = &apc->tx_qp[i].tx_cq.napi;if (apc->tx_qp[i].txq.napi_initialized) {napi_synchronize (napi); -napi_disable (napi); -netif_napi_del (napi); +netdev_lock_ops_to_full (napi->dev); +napi_disable_locked (napi); +netif_napi_del_locked (napi); +netdev_unlock_full_to_ops (napi->dev); apc->tx_qp[i].txq.napi_initialized = false; } mana_destroy_wq_obj(apc, GDMA_SQ, apc-Quoted 1 line
tx_qp[i].tx_object); @@ -2065,8 +2068,11 @@ static int
mana_create_txq(struct
mana_port_context*apc,drivers/net/ethernet/microsoft/mana/mana_en.c +16
-7mana_create_txq_debugfs (apc, i); -netif_napi_add_tx (net, &cq->napi, mana_poll); -napi_enable (&cq->napi); +set_bit (NAPI_STATE_NO_BUSY_POLL , &cq->napi.state); +netdev_lock_ops_to_full (net); +netif_napi_add_locked (net, &cq->napi, mana_poll); +napi_enable_locked (&cq->napi); +netdev_unlock_full_to_ops (net); txq->napi_initialized = true; mana_gd_ring_cq(cq->gdma_cq, SET_ARM_BIT); @@ -2102,9 +2108,11 @@ static void mana_destroy_rxq(struct mana_port_context *apc, if (napi_initialized) {napi_synchronize (napi); -napi_disable (napi); - -netif_napi_del (napi); +netdev_lock_ops_to_full (napi->dev); +napi_disable_locked (napi); +netif_napi_del_locked (napi); +netdev_unlock_full_to_ops (napi->dev); }xdp_rxq_info_unreg (&rxq->xdp_rxq);@@ -2355,14 +2363,18 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc, gc->cq_table[cq->gdma_id] = cq->gdma_cq; -netif_napi_add_weight (ndev, &cq->napi, mana_poll,1 ); +netdev_lock_ops_to_full (ndev); +netif_napi_add_weight_locked (ndev, &cq->napi, mana_poll,1 ); +netdev_unlock_full_to_ops (ndev);WARN_ON (xdp_rxq_info_reg (&rxq->xdp_rxq, ndev, rxq_idx, cq->napi.napi_id));WARN_ON (xdp_rxq_info_reg_mem_model (&rxq->xdp_rxq,MEM_TYPE_PAGE_POOL , rxq->page_pool)); -napi_enable (&cq->napi); +netdev_lock_ops_to_full (ndev); +napi_enable_locked (&cq->napi); +netdev_unlock_full_to_ops (ndev);
mana_gd_ring_cq(cq->gdma_cq, SET_ARM_BIT);
out:
--
2.34.1
PATCH v2 2/4 net: mana: Add support for net_shaper_ops
In reply to Erni Sri Satya VennelaIntroduce support for net_shaper_ops in the MANA driver, enabling configuration of rate limiting on the MANA NIC.
To apply rate limiting, the driver issues a HWC command via mana_set_bw_clamp() and updates the corresponding shaper object in the net_shaper cache. If an error occurs during this process, the driver restores the previous speed by querying the current link configuration using mana_query_link_cfg().
The minimum supported bandwidth is 100 Mbps, and only values that are exact multiples of 100 Mbps are allowed. Any other values are rejected.
To remove a shaper, the driver resets the bandwidth to the maximum supported by the SKU using mana_set_bw_clamp() and clears the associated cache entry. If an error occurs during this process, the shaper details are retained.
On the hardware that does not support these APIs, the net-shaper calls to set speed would fail.
Set the speed:./tools/net/ynl/pyynl/cli.py \
--spec Documentation/netlink/specs/net_shaper.yaml \
--do set --json '{"ifindex":'$IFINDEX',
"handle":{"scope": "netdev", "id":'$ID' },
"bw-max": 200000000 }'Get the shaper details:./tools/net/ynl/pyynl/cli.py \
--spec Documentation/netlink/specs/net_shaper.yaml \
--do get --json '{"ifindex":'$IFINDEX',
"handle":{"scope": "netdev", "id":'$ID' }}'Quoted 4 lines
{'bw-max': 200000000,
'handle': {'scope': 'netdev'},
'ifindex': $IFINDEX,
'metric': 'bps'}
Delete the shaper object:./tools/net/ynl/pyynl/cli.py \
--spec Documentation/netlink/specs/net_shaper.yaml \
--do delete --json '{"ifindex":'$IFINDEX',
"handle":{"scope": "netdev","id":'$ID' }}'---
Changes in v2:
* No change.
--- drivers/net/ethernet/microsoft/mana/mana_en.c | 155 ++++++++++++++++++ include/net/mana/mana.h | 40 +++++ 2 files changed, 195 insertions(+)
drivers/net/ethernet/microsoft/mana/mana_en.c +155 -0
@@ -719,6 +719,78 @@ static int mana_change_mtu(struct net_device *ndev, int new_mtu)return err; } +static int mana_shaper_set (struct net_shaper_binding *binding, +const struct net_shaper *shaper, +struct netlink_ext_ack *extack) +{ +struct mana_port_context *apc =netdev_priv (binding->netdev); +u32 old_speed, rate; +int err; + +if (shaper->handle.scope !=NET_SHAPER_SCOPE_NETDEV ) { +NL_SET_ERR_MSG_MOD (extack,"net shaper scope should be netdev" ); +return -EINVAL ; + } + +if (apc->handle.id && shaper->handle.id != apc->handle.id) { +NL_SET_ERR_MSG_MOD (extack,"Cannot create multiple shapers" ); +return -EOPNOTSUPP ; + } + +if (!shaper->bw_max || (shaper->bw_max %100000000 )) { +NL_SET_ERR_MSG_MOD (extack,"Please use multiples of 100Mbps for bandwidth" ); +return -EINVAL ; + } + + rate =div_u64 (shaper->bw_max,1000 );/* Convert bps to Kbps */ + rate =div_u64 (rate,1000 );/* Convert Kbps to Mbps */ + +/* Get current speed */ + err =mana_query_link_cfg (apc); + old_speed = (err) ?SPEED_UNKNOWN : apc->speed; + +if (!err) { + err =mana_set_bw_clamp (apc, rate,TRI_STATE_TRUE ); + apc->speed = (err) ? old_speed : rate; + apc->handle = (err) ? apc->handle : shaper->handle; + } + +return err; +} + +static int mana_shaper_del (struct net_shaper_binding *binding, +const struct net_shaper_handle *handle, +struct netlink_ext_ack *extack) +{ +struct mana_port_context *apc =netdev_priv (binding->netdev); +int err; + + err =mana_set_bw_clamp (apc,0 ,TRI_STATE_FALSE ); + +if (!err) { +/* Reset mana port context parameters */ + apc->handle.id =0 ; + apc->handle.scope =NET_SHAPER_SCOPE_UNSPEC ; + apc->speed =0 ; + } + +return err; +} + +static void mana_shaper_cap (struct net_shaper_binding *binding, +enum net_shaper_scope scope, +unsigned long *flags) +{ + *flags =BIT (NET_SHAPER_A_CAPS_SUPPORT_BW_MAX ) | +BIT (NET_SHAPER_A_CAPS_SUPPORT_METRIC_BPS ); +} + +static const struct net_shaper_ops mana_shaper_ops = { + .set = mana_shaper_set, + .delete = mana_shaper_del, + .capabilities = mana_shaper_cap, +}; +static const struct net_device_ops mana_devops = { .ndo_open = mana_open, .ndo_stop = mana_close,
@@ -729,6 +801,7 @@ static const struct net_device_ops mana_devops = { .ndo_bpf = mana_bpf, .ndo_xdp_xmit = mana_xdp_xmit, .ndo_change_mtu = mana_change_mtu, + .net_shaper_ops = &mana_shaper_ops, };static void mana_cleanup_port_context (struct mana_port_context *apc)
@@ -1162,6 +1235,86 @@ static int mana_cfg_vport_steering(struct mana_port_context *apc,return err; } +int mana_query_link_cfg (struct mana_port_context *apc) +{ +struct net_device *ndev = apc->ndev; +struct mana_query_link_config_resp resp = {}; +struct mana_query_link_config_req req = {}; +int err; + +mana_gd_init_req_hdr (&req.hdr,MANA_QUERY_LINK_CONFIG , +sizeof (req),sizeof (resp)); + + req.vport = apc->port_handle; + req.hdr.resp.msg_version =GDMA_MESSAGE_V2 ; + + err =mana_send_request (apc->ac, &req,sizeof (req), &resp, +sizeof (resp)); + +if (err) { +netdev_err (ndev,"Failed to query link config: %d\n" , err); +return err; + } + + err =mana_verify_resp_hdr (&resp.hdr,MANA_QUERY_LINK_CONFIG , +sizeof (resp)); + +if (err || resp.hdr.status) { +netdev_err (ndev,"Failed to query link config: %d, 0x%x\n" , err, + resp.hdr.status); +if (!err) + err = -EOPNOTSUPP ; +return err; + } + +if (resp.qos_unconfigured) { + err = -EINVAL ; +return err; + } + apc->speed = resp.link_speed_mbps; +return 0 ; +} + +int mana_set_bw_clamp (struct mana_port_context *apc,u32 speed, +int enable_clamping) +{ +struct mana_set_bw_clamp_resp resp = {}; +struct mana_set_bw_clamp_req req = {}; +struct net_device *ndev = apc->ndev; +int err; + +mana_gd_init_req_hdr (&req.hdr,MANA_SET_BW_CLAMP , +sizeof (req),sizeof (resp)); + req.vport = apc->port_handle; + req.link_speed_mbps = speed; + req.enable_clamping = enable_clamping; + + err =mana_send_request (apc->ac, &req,sizeof (req), &resp, +sizeof (resp)); + +if (err) { +netdev_err (ndev,"Failed to set bandwidth clamp for speed %u, err = %d" , + speed, err); +return err; + } + + err =mana_verify_resp_hdr (&resp.hdr,MANA_SET_BW_CLAMP , +sizeof (resp)); + +if (err || resp.hdr.status) { +netdev_err (ndev,"Failed to set bandwidth clamp: %d, 0x%x\n" , err, + resp.hdr.status); +if (!err) + err = -EOPNOTSUPP ; +return err; + } + +if (resp.qos_unconfigured) +netdev_info (ndev,"QoS is unconfigured\n" ); + +return 0 ; +} + int mana_create_wq_obj(struct mana_port_context *apc, mana_handle_t vport, u32 wq_type, struct mana_obj_spec *wq_spec,
@@ -3013,6 +3166,8 @@ static int mana_probe_port(struct mana_context *ac, int port_idx, goto free_indir; } +debugfs_create_u32 ("current_speed" ,0400 , apc->mana_port_debugfs, &apc->speed); +return 0 ; free_indir:
include/net/mana/mana.h +40 -0
@@ -5,6 +5,7 @@#define _MANA_H#include <net/xdp.h> +#include <net/net_shaper.h> #include "gdma.h" #include "hw_channel.h"
@@ -526,7 +527,12 @@ struct mana_port_context {struct mutex vport_mutex;int vport_use_count; +/* Net shaper handle*/ +struct net_shaper_handle handle; +u16 port_idx; +/* Currently configured speed (mbps) */ +u32 speed;bool port_is_up;bool port_st_save;/* Saved port state */
@@ -562,6 +568,9 @@ struct bpf_prog *mana_xdp_get(struct mana_port_context *apc);void mana_chn_setxdp (struct mana_port_context *apc,struct bpf_prog *prog);int mana_bpf (struct net_device *ndev,struct netdev_bpf *bpf);void mana_query_gf_stats (struct mana_port_context *apc); +int mana_query_link_cfg (struct mana_port_context *apc); +int mana_set_bw_clamp (struct mana_port_context *apc,u32 speed, +int enable_clamping);void mana_query_phy_stats (struct mana_port_context *apc);int mana_pre_alloc_rxbufs (struct mana_port_context *apc,int mtu,int num_queues);void mana_pre_dealloc_rxbufs (struct mana_port_context *apc);
@@ -589,6 +598,8 @@ enum mana_command_code {MANA_FENCE_RQ =0x20006 ,MANA_CONFIG_VPORT_RX =0x20007 ,MANA_QUERY_VPORT_CONFIG =0x20008 , +MANA_QUERY_LINK_CONFIG =0x2000A , +MANA_SET_BW_CLAMP =0x2000B , MANA_QUERY_PHY_STAT = 0x2000c,/* Privileged commands for the PF mode */
@@ -598,6 +609,35 @@ enum mana_command_code { MANA_DEREGISTER_HW_PORT = 0x28004, }; +/* Query Link Configuration*/ +struct mana_query_link_config_req { +struct gdma_req_hdr hdr; +mana_handle_t vport; +};/* HW DATA */ + +struct mana_query_link_config_resp { +struct gdma_resp_hdr hdr; +u32 qos_speed_mbps; +u8 qos_unconfigured; +u8 reserved1[3 ]; +u32 link_speed_mbps; +u8 reserved2[4 ]; +};/* HW DATA */ + +/* Set Bandwidth Clamp*/ +struct mana_set_bw_clamp_req { +struct gdma_req_hdr hdr; +mana_handle_t vport; +enum TRI_STATE enable_clamping; +u32 link_speed_mbps; +};/* HW DATA */ + +struct mana_set_bw_clamp_resp { +struct gdma_resp_hdr hdr; +u8 qos_unconfigured; +u8 reserved[7 ]; +};/* HW DATA */ +/* Query Device Configuration */ struct mana_query_device_cfg_req {struct gdma_req_hdr hdr;
Signature · 1 line
2.34.1
RE: [PATCH net-next v2 2/4] net: mana: Add support for net_shaper_ops
Quoted 43 lines
Subject: [PATCH net-next v2 2/4] net: mana: Add support for
net_shaper_opsIntroduce support for
net_shaper_opsin the MANA driver, enabling configuration of rate limiting on the MANA NIC.To apply rate limiting, the driver issues a HWC command via
mana_set_bw_clamp()and updates the corresponding shaper object in thenet_shapercache. If an error occurs during this process, the driver restores the previous speed by querying the current link configuration usingmana_query_link_cfg().The minimum supported bandwidth is 100 Mbps, and only values that are exact multiples of 100 Mbps are allowed. Any other values are rejected.
To remove a shaper, the driver resets the bandwidth to the maximum supported by the SKU using
mana_set_bw_clamp()and clears the associated cache entry. If an error occurs during this process, the shaper details are retained.On the hardware that does not support these APIs, the net-shaper calls to set speed would fail.
Set the speed:
./tools/net/ynl/pyynl/cli.py\
--specDocumentation/netlink/specs/net_shaper.yaml\ --do set --json
'{"ifindex":'$IFINDEX',"handle":{"scope": "netdev", "id":'$ID' }, "bw-max": 200000000 }'Get the shaper details:
./tools/net/ynl/pyynl/cli.py\
--specDocumentation/netlink/specs/net_shaper.yaml\ --do get --json
'{"ifindex":'$IFINDEX',"handle":{"scope": "netdev", "id":'$ID' }}'Quoted 4 lines
{'bw-max': 200000000,
'handle': {'scope': 'netdev'},
'ifindex': $IFINDEX,
'metric': 'bps'}Delete the shaper object:
./tools/net/ynl/pyynl/cli.py\
--specDocumentation/netlink/specs/net_shaper.yaml\ --do delete --json
'{"ifindex":'$IFINDEX',
Quoted 6 lines
"handle":{"scope": "netdev","id":'$ID' }}'Signed-off-by Erni Sri Satya Vennela <[email protected]>Reviewed-by Haiyang Zhang <[email protected]>Reviewed-by Shradha Gupta <[email protected]>Reviewed-by Saurabh Singh Sengar <[email protected]>
Quoted 285 lines
---
Changes in v2:
* No change.--- drivers/net/ethernet/microsoft/mana/mana_en.c | 155 ++++++++++++++++++ include/net/mana/mana.h | 40 +++++ 2 files changed, 195 insertions(+)drivers/net/ethernet/microsoft/mana/mana_en.c +20
-0@@ -719,6 +719,78 @@ static int mana_change_mtu(struct net_device *ndev, int new_mtu)return err; } +static int mana_shaper_set (struct net_shaper_binding *binding, +const struct net_shaper *shaper, +struct netlink_ext_ack *extack) +{ +struct mana_port_context *apc =netdev_priv (binding->netdev); +u32 old_speed, rate; +int err; + +if (shaper->handle.scope !=NET_SHAPER_SCOPE_NETDEV ) { +NL_SET_ERR_MSG_MOD (extack,"net shaper scope should be netdev"); +return -EINVAL ; + } + +if (apc->handle.id && shaper->handle.id != apc->handle.id) { +NL_SET_ERR_MSG_MOD (extack,"Cannot create multiple shapers"); +return -EOPNOTSUPP ; + } + +if (!shaper->bw_max || (shaper->bw_max %100000000 )) { + NL_SET_ERR_MSG_MOD(extack, "Please use multiples of100Mbps for bandwidth");
drivers/net/ethernet/microsoft/mana/mana_en.c +53
-0+return -EINVAL ; + } + + rate =div_u64 (shaper->bw_max,1000 );/* Convert bps to Kbps */ + rate =div_u64 (rate,1000 );/* Convert Kbps to Mbps */ + +/* Get current speed */ + err =mana_query_link_cfg (apc); + old_speed = (err) ?SPEED_UNKNOWN : apc->speed; + +if (!err) { + err =mana_set_bw_clamp (apc, rate,TRI_STATE_TRUE ); + apc->speed = (err) ? old_speed : rate; + apc->handle = (err) ? apc->handle : shaper->handle; + } + +return err; +} + +static int mana_shaper_del (struct net_shaper_binding *binding, +const struct net_shaper_handle *handle, +struct netlink_ext_ack *extack) +{ +struct mana_port_context *apc =netdev_priv (binding->netdev); +int err; + + err =mana_set_bw_clamp (apc,0 ,TRI_STATE_FALSE ); + +if (!err) { +/* Reset mana port context parameters */ + apc->handle.id =0 ; + apc->handle.scope =NET_SHAPER_SCOPE_UNSPEC ; + apc->speed =0 ; + } + +return err; +} + +static void mana_shaper_cap (struct net_shaper_binding *binding, +enum net_shaper_scope scope, +unsigned long *flags) +{ + *flags =BIT (NET_SHAPER_A_CAPS_SUPPORT_BW_MAX ) | +BIT (NET_SHAPER_A_CAPS_SUPPORT_METRIC_BPS ); +} + +static const struct net_shaper_ops mana_shaper_ops = { + .set = mana_shaper_set, + .delete = mana_shaper_del, + .capabilities = mana_shaper_cap, +}; +static const struct net_device_ops mana_devops = { .ndo_open = mana_open, .ndo_stop = mana_close,@@ -729,6 +801,7 @@ static const struct net_device_ops mana_devops = { .ndo_bpf = mana_bpf, .ndo_xdp_xmit = mana_xdp_xmit, .ndo_change_mtu = mana_change_mtu, + .net_shaper_ops = &mana_shaper_ops, }; static void mana_cleanup_port_context(struct mana_port_context *apc) @@ -1162,6 +1235,86 @@ static int mana_cfg_vport_steering(struct
mana_port_context*apc,drivers/net/ethernet/microsoft/mana/mana_en.c +58
-0return err; } +int mana_query_link_cfg (struct mana_port_context *apc) { +struct net_device *ndev = apc->ndev; +struct mana_query_link_config_resp resp = {}; +struct mana_query_link_config_req req = {}; +int err; + +mana_gd_init_req_hdr (&req.hdr,MANA_QUERY_LINK_CONFIG , +sizeof (req),sizeof (resp)); + + req.vport = apc->port_handle; + req.hdr.resp.msg_version =GDMA_MESSAGE_V2 ; + + err =mana_send_request (apc->ac, &req,sizeof (req), &resp, +sizeof (resp)); + +if (err) { +netdev_err (ndev,"Failed to query link config: %d\n" , err); +return err; + } + + err =mana_verify_resp_hdr (&resp.hdr,MANA_QUERY_LINK_CONFIG , +sizeof (resp)); + +if (err || resp.hdr.status) { +netdev_err (ndev,"Failed to query link config: %d, 0x%x\n" , err, + resp.hdr.status); +if (!err) + err = -EOPNOTSUPP ; +return err; + } + +if (resp.qos_unconfigured) { + err = -EINVAL ; +return err; + } + apc->speed = resp.link_speed_mbps; +return 0 ; +} + +int mana_set_bw_clamp (struct mana_port_context *apc,u32 speed, +int enable_clamping) +{ +struct mana_set_bw_clamp_resp resp = {}; +struct mana_set_bw_clamp_req req = {}; +struct net_device *ndev = apc->ndev; +int err; + +mana_gd_init_req_hdr (&req.hdr,MANA_SET_BW_CLAMP , +sizeof (req),sizeof (resp)); + req.vport = apc->port_handle; + req.link_speed_mbps = speed; + req.enable_clamping = enable_clamping; + + err =mana_send_request (apc->ac, &req,sizeof (req), &resp, +sizeof (resp)); + +if (err) { + netdev_err(ndev, "Failed to set bandwidth clamp for speed %u,err = %d",
drivers/net/ethernet/microsoft/mana/mana_en.c +9
-0+ speed, err); +return err; + } + + err =mana_verify_resp_hdr (&resp.hdr,MANA_SET_BW_CLAMP , +sizeof (resp)); + +if (err || resp.hdr.status) { + netdev_err(ndev, "Failed to set bandwidth clamp: %d, 0x%x\n",err,
drivers/net/ethernet/microsoft/mana/mana_en.c +16
-0+ resp.hdr.status); +if (!err) + err = -EOPNOTSUPP ; +return err; + } + +if (resp.qos_unconfigured) +netdev_info (ndev,"QoS is unconfigured\n" ); + +return 0 ; +} + int mana_create_wq_obj(struct mana_port_context *apc, mana_handle_t vport, u32 wq_type, struct mana_obj_spec *wq_spec, @@ -3013,6 +3166,8 @@ static int mana_probe_port(struct mana_context *ac, int port_idx, goto free_indir; } + debugfs_create_u32("current_speed", 0400, apc->mana_port_debugfs, +&apc->speed); + return 0; free_indir:include/net/mana/mana.h index renamed from include/net/mana/mana.h +0
-0
4176edf1be71..038b18340e51100644include/net/mana/mana.h +6
-0@@ -5,6 +5,7 @@#define _MANA_H#include <net/xdp.h> +#include <net/net_shaper.h> #include "gdma.h" #include "hw_channel.h" @@ -526,7 +527,12 @@ struct mana_port_context {struct mutex vport_mutex;int vport_use_count; +/* Net shaper handle*/ +struct net_shaper_handle handle; +u16 port_idx; +/* Currently configured speed (mbps) */ +u32 speed;bool port_is_up; bool port_st_save; /* Saved port state */ @@ -562,6 +568,9 @@ struct
bpf_prog*mana_xdp_get(structmana_port_context*apc); voidmana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog); intmana_bpf(struct net_device *ndev, struct netdev_bpf *bpf); voidmana_query_gf_stats(struct mana_port_context *apc);include/net/mana/mana.h +3
-0+int mana_query_link_cfg (struct mana_port_context *apc);int +mana_set_bw_clamp (struct mana_port_context *apc,u32 speed, +int enable_clamping);void mana_query_phy_stats (struct mana_port_context *apc);int
mana_pre_alloc_rxbufs(struct mana_port_context *apc, int mtu, int num_queues); voidmana_pre_dealloc_rxbufs(struct mana_port_context *apc);include/net/mana/mana.h +2
-0@@ -589,6 +598,8 @@ enum mana_command_code {MANA_FENCE_RQ =0x20006 ,MANA_CONFIG_VPORT_RX =0x20007 ,MANA_QUERY_VPORT_CONFIG =0x20008 , +MANA_QUERY_LINK_CONFIG =0x2000A , +MANA_SET_BW_CLAMP =0x2000B ,MANA_QUERY_PHY_STAT =0x2000c , /* Privileged commands for the PF mode */ @@ -598,6 +609,35 @@enum
mana_command_code{include/net/mana/mana.h +29
-1MANA_DEREGISTER_HW_PORT = 0x28004, }; +/* Query Link Configuration*/ +struct mana_query_link_config_req { +struct gdma_req_hdr hdr; +mana_handle_t vport; +};/* HW DATA */ + +struct mana_query_link_config_resp { +struct gdma_resp_hdr hdr; +u32 qos_speed_mbps; +u8 qos_unconfigured; +u8 reserved1[3 ]; +u32 link_speed_mbps; +u8 reserved2[4 ]; +};/* HW DATA */ + +/* Set Bandwidth Clamp*/ +struct mana_set_bw_clamp_req { +struct gdma_req_hdr hdr; +mana_handle_t vport; +enum TRI_STATE enable_clamping; +u32 link_speed_mbps; +};/* HW DATA */ + +struct mana_set_bw_clamp_resp { +struct gdma_resp_hdr hdr; +u8 qos_unconfigured; +u8 reserved[7 ]; +};/* HW DATA */ +/* Query Device Configuration */ struct mana_query_device_cfg_req {struct gdma_req_hdr hdr; --2.34.1
PATCH v2 3/4 net: mana: Add speed support in mana_get_link_ksettings
In reply to Erni Sri Satya VennelaAllow mana ethtool get_link_ksettings operation to report the maximum speed supported by the SKU in mbps.
The driver retrieves this information by issuing a HWC command to the hardware via mana_query_link_cfg(), which retrieves the SKU's maximum supported speed.
These APIs when invoked on hardware that are older/do not support these APIs, the speed would be reported as UNKNOWN.
Before:
$ethtool enP30832s1
Quoted 1 line
Settings for enP30832s1:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Full
Auto-negotiation: off
Port: Other
PHYAD: 0
Transceiver: internal
Link detected: yes
After:
$ethtool enP30832s1
Quoted 1 line
Settings for enP30832s1:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 16000Mb/s
Duplex: Full
Auto-negotiation: off
Port: Other
PHYAD: 0
Transceiver: internal
Link detected: yes
---
Changes in v2:
* No change.
--- drivers/net/ethernet/microsoft/mana/mana_en.c | 1 + drivers/net/ethernet/microsoft/mana/mana_ethtool.c | 6 ++++++ include/net/mana/mana.h | 2 ++ 3 files changed, 9 insertions(+)
drivers/net/ethernet/microsoft/mana/mana_en.c +1 -0
@@ -1272,6 +1272,7 @@ int mana_query_link_cfg(struct mana_port_context *apc)return err; } apc->speed = resp.link_speed_mbps; + apc->max_speed = resp.qos_speed_mbps;return 0 ; }
drivers/net/ethernet/microsoft/mana/mana_ethtool.c +6 -0
@@ -495,6 +495,12 @@ static int mana_set_ringparam(struct net_device *ndev,static int mana_get_link_ksettings (struct net_device *ndev,struct ethtool_link_ksettings *cmd) { +struct mana_port_context *apc =netdev_priv (ndev); +int err; + + err =mana_query_link_cfg (apc); + cmd->base.speed = (err) ?SPEED_UNKNOWN : apc->max_speed; + cmd->base.duplex =DUPLEX_FULL ; cmd->base.port =PORT_OTHER ;
include/net/mana/mana.h +2 -0
@@ -533,6 +533,8 @@ struct mana_port_context {u16 port_idx;/* Currently configured speed (mbps) */ u32 speed; +/* Maximum speed supported by the SKU (mbps) */ +u32 max_speed;bool port_is_up;bool port_st_save;/* Saved port state */
Signature · 1 line
2.34.1
PATCH v2 4/4 net: mana: Handle unsupported HWC commands
In reply to Erni Sri Satya VennelaIf any of the HWC commands are not recognized by the underlying hardware, the hardware returns the response header status of -1. Log the information using netdev_info_once to avoid multiple error logs in dmesg.
---
Changes in v2:
* Define GDMA_STATUS_CMD_UNSUPPORTED for unsupported HWC status code instead of using -1.
--- drivers/net/ethernet/microsoft/mana/hw_channel.c | 4 ++++ drivers/net/ethernet/microsoft/mana/mana_en.c | 11 +++++++++++ include/net/mana/gdma.h | 1 + 3 files changed, 16 insertions(+)
drivers/net/ethernet/microsoft/mana/hw_channel.c +4 -0
@@ -891,6 +891,10 @@ int mana_hwc_send_request(struct hw_channel_context *hwc, u32 req_len, }if (ctx->status_code && ctx->status_code !=GDMA_STATUS_MORE_ENTRIES ) { +if (ctx->status_code ==GDMA_STATUS_CMD_UNSUPPORTED ) { + err = -EOPNOTSUPP ; + goto out; + }if (req_msg->req.msg_type !=MANA_QUERY_PHY_STAT )dev_err (hwc->dev,"HWC: Failed hw_channel req: 0x%x\n" , ctx->status_code);
drivers/net/ethernet/microsoft/mana/mana_en.c +11 -0
@@ -847,6 +847,9 @@ static int mana_send_request(struct mana_context *ac, void *in_buf, err =mana_gd_send_request (gc, in_len, in_buf, out_len, out_buf);if (err || resp->status) { +if (err == -EOPNOTSUPP ) +return err; +if (req->req.msg_type !=MANA_QUERY_PHY_STAT )dev_err (dev,"Failed to send mana message: %d, 0x%x\n" , err, resp->status);
@@ -1252,6 +1255,10 @@ int mana_query_link_cfg(struct mana_port_context *apc) sizeof(resp));if (err) { +if (err == -EOPNOTSUPP ) { +netdev_info_once (ndev,"MANA_QUERY_LINK_CONFIG not supported\n" ); +return err; + }netdev_err (ndev,"Failed to query link config: %d\n" , err);return err; }
@@ -1294,6 +1301,10 @@ int mana_set_bw_clamp(struct mana_port_context *apc, u32 speed, sizeof(resp));if (err) { +if (err == -EOPNOTSUPP ) { +netdev_info_once (ndev,"MANA_SET_BW_CLAMP not supported\n" ); +return err; + }netdev_err (ndev,"Failed to set bandwidth clamp for speed %u, err = %d" , speed, err);return err;
include/net/mana/gdma.h +1 -0
@@ -10,6 +10,7 @@#include "shm_channel.h" #define GDMA_STATUS_MORE_ENTRIES 0x00000105 +#define GDMA_STATUS_CMD_UNSUPPORTED 0xffffffff/* Structures labeled with "HW DATA" are exchanged with the hardware. All of * them are naturally aligned and hence don't need __packed.
Signature · 1 line
2.34.1
Jakub Kicinski
maintainerQuoted 20 lines · On Fri, 13 Jun 2025 04:20:23 -0700 Erni Sri Satya Vennela wrote:
This patchset introduces hardware-backed bandwidth rate limiting for MANA NICs via the
net_shaper_opsinterface, enabling efficient and fine-grained traffic shaping directly on the device.Previously, MANA lacked a mechanism for user-configurable bandwidth control. With this addition, users can now configure shaping parameters, allowing better traffic management and performance isolation.
The implementation includes the
net_shaper_opscallbacks in the MANA driver and supports one shaper per vport. Add shaping support viamana_set_bw_clamp(), allowing the configuration of bandwidth rates in 100 Mbps increments (minimum 100 Mbps). The driver validates input and rejects unsupported values. On failure, it restores the previous configuration which is queried usingmana_query_link_cfg()or retains the current state.To prevent potential deadlocks introduced by
net_shaper_ops, switch to_lockedvariants of NAPI APIs whennetdevops_lockis held during VF setup and teardown.
Quoted 6 lines
Also, Add support for ethtool
get_link_ksettingsto report the maximum link speed supported by the SKU in mbps.These APIs when invoked on hardware that are older or that do not support these APIs, the speed would be reported as UNKNOWN and the net-shaper calls to set speed would fail.
Failed to apply patch:
Applying: net: mana: Fix potential deadlocks in mana napi ops
error: patch fragment without header at line 23: @@ -2102,9 +2108,11 @@ static void mana_destroy_rxq(struct mana_port_context *apc,
error: could not build fake ancestor
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 net: mana: Fix potential deadlocks in mana napi ops
please rebase
Signature · 1 line
pw-bot: cr
Quoted 39 lines · Jakub Kicinski
Quoted 20 lines · On Fri, 13 Jun 2025 04:20:23 -0700 Erni Sri Satya Vennela wrote:
This patchset introduces hardware-backed bandwidth rate limiting for MANA NICs via the
net_shaper_opsinterface, enabling efficient and fine-grained traffic shaping directly on the device.Previously, MANA lacked a mechanism for user-configurable bandwidth control. With this addition, users can now configure shaping parameters, allowing better traffic management and performance isolation.
The implementation includes the
net_shaper_opscallbacks in the MANA driver and supports one shaper per vport. Add shaping support viamana_set_bw_clamp(), allowing the configuration of bandwidth rates in 100 Mbps increments (minimum 100 Mbps). The driver validates input and rejects unsupported values. On failure, it restores the previous configuration which is queried usingmana_query_link_cfg()or retains the current state.To prevent potential deadlocks introduced by
net_shaper_ops, switch to_lockedvariants of NAPI APIs whennetdevops_lockis held during VF setup and teardown.Quoted 6 lines
Also, Add support for ethtool
get_link_ksettingsto report the maximum link speed supported by the SKU in mbps.These APIs when invoked on hardware that are older or that do not support these APIs, the speed would be reported as UNKNOWN and the net-shaper calls to set speed would fail.
Failed to apply patch:
Applying: net: mana: Fix potential deadlocks in mana napi ops
error: patch fragment without header at line 23: @@ -2102,9 +2108,11 @@ static void mana_destroy_rxq(structmana_port_context*apc,
error: could not build fake ancestor
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 net: mana: Fix potential deadlocks in mana napi ops
Quoted 1 line
please rebase
Hi Jakub,
Thank you for your reply. I will rebase and repost.
- Vennela
Quoted 2 lines
Signature · 1 line
pw-bot: cr