net: introduce TX H/W shaping API
Subsystems (11)
Paolo Abeni
maintainerv6 cover net: introduce TX H/W shaping API
We have a plurality of shaping-related drivers API, but none flexible enough to meet existing demand from vendors[1].
This series introduces new device APIs to configure in a flexible way TX H/W shaping. The new functionalities are exposed via a newly defined generic netlink interface and include introspection capabilities. Some self-tests are included, on top of a dummy netdevsim implementation. Finally a basic implementation for the iavf driver is provided.
Some usage examples:
* Configure shaping on a given queue:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do set --json '{"ifindex": '$IFINDEX',
"shaper": {"handle":
{"scope": "queue", "id":'$QUEUEID'},
"bw-max": 2000000}}'* Container B/W sharing
The orchestration infrastructure wants to group the container-related queues under a RR scheduling and limit the aggregate bandwidth:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do group --json '{"ifindex": '$IFINDEX',
"leaves": [
{"handle": {"scope": "queue", "id":'$QID1'},
"weight": '$W1'},
{"handle": {"scope": "queue", "id":'$QID2'},
"weight": '$W2'}],
{"handle": {"scope": "queue", "id":'$QID3'},
"weight": '$W3'}],
"node": {"handle": {"scope":"node"},
"bw-max": 10000000}}'{'ifindex': $IFINDEX, 'handle': {'scope': 'node', 'id': 0}}
Q1 \
\
Q2 -- node 0 ------- netdev
/ (bw-max: 10M)Q3 /
* Delegation
A containers wants to limit the aggregate B/W bandwidth of 2 of the 3 queues it owns - the starting configuration is the one from the previous point:
SPEC=Documentation/netlink/specs/net_shaper.yaml
./tools/net/ynl/cli.py --spec $SPEC \
--do group --json '{"ifindex": '$IFINDEX',
"leaves": [
{"handle": {"scope": "queue", "id":'$QID1'},
"weight": '$W1'},
{"handle": {"scope": "queue", "id":'$QID2'},
"weight": '$W2'}],
"node": {"handle": {"scope": "node"},
"bw-max": 5000000 }}'{'ifindex': $IFINDEX, 'handle': {'scope': 'node', 'id': 1}}
Q1 -- node 1 --------\
/ (bw-max: 5M) \
Q2 / node 0 ------- netdev
/ (bw-max: 10M)Q3 ------------------
* Cleanup:
To delete a single queue shaper:
./tools/net/ynl/cli.py --spec $SPEC --do delete --json \
'{"ifindex": '$IFINDEX',
"handle": {"scope": "queue", "id":'$QID3'}}'Q1 -- node 1 --------\
/ (bw-max: 5M) \
Q2 / node 0 ------- netdev
(bw-max: 10M)Deleting a node shaper relinks all its leaves to the node's parent:
./tools/net/ynl/cli.py --spec $SPEC --do delete --json \
'{"ifindex": '$IFINDEX',
"handle": {"scope": "node", "id":1}}'Q1 ------\
\
node 0 ------- netdev
/ (bw-max: 10M)Q2 -----
Deleting the last shaper under a node shaper deletes the node, too:
./tools/net/ynl/cli.py --spec $SPEC --do delete --json \
'{"ifindex": '$IFINDEX',
"handle": {"scope": "queue", "id":'$QID1'}}'
./tools/net/ynl/cli.py --spec $SPEC --do delete --json \
'{"ifindex": '$IFINDEX',
"handle": {"scope": "queue", "id":'$QID2'}}'
./tools/net/ynl/cli.py --spec $SPEC --do get --json \
'{"ifindex": '$IFINDEX',
"handle": {"scope": "node", "id": 0}}'
Netlink error: No such file or directory
nl_len = 44 (28) nl_flags = 0x300 nl_type = 2
error: -2
extack: {'bad-attr': '.handle'}[1] https://lore.kernel.org/netdev/[email protected]/
---
Changes from v5:
- handle/shaper unsplit
- removed 'binding' from NL api
- several helper renames
- added rcu protection to shaper pointer
- implemented value checking into the core
- handle device channels reconf
- consolidate NL ctx handling
- dropped idr allocator, leverage the existing xarray
- caps names shrinking
- group() op is optional
- cap callback can't fail
- more self-tests
v5: https://lore.kernel.org/netdev/[email protected]/
Changes from v4:
- ops operate on struct binding
- 'root' -> 'node' rename
- most core function/helpers operate on 'binding'
- use NL_SET_BAD_ATTR where possible
- some code deduplication
v4: https://lore.kernel.org/netdev/[email protected]/
Changes from v3:
- rename
- locking
- delete operates on node, too
v3: https://lore.kernel.org/netdev/[email protected]/
Changes from RFC v2:
- added patch 1
- fixed deprecated API usage
RFC v2: https://lore.kernel.org/netdev/[email protected]/
Changes from RFC v1:
- set() and delete() ops operate on a single shaper
- added group() op to allow grouping and nesting
- split the NL implementation into multiple patches to help reviewing
RFC v1: https://lore.kernel.org/netdev/[email protected]/
Paolo Abeni (11):
genetlink: extend info user-storage to match NL cb ctx
netlink: spec: add shaper YAML spec
net-shapers: implement NL get operation
net-shapers: implement NL set and delete operations
net-shapers: implement NL group operation
net-shapers: implement delete support for NODE scope shaper
net-shapers: implement shaper cleanup on queue deletion
netlink: spec: add shaper introspection support
net: shaper: implement introspection support
net-shapers: implement cap validation in the core
testing: net-drv: add basic shaper test
Sudheer Mogilappagari (2):
iavf: Add net_shaper_ops support
iavf: add support to exchange qos capabilities
Wenjun Wu (2):
virtchnl: support queue rate limit and quanta size configuration
ice: Support VF queue rate limit and quanta size configuration
Documentation/netlink/specs/net_shaper.yaml | 401 +++++ Documentation/networking/kapi.rst | 3 + MAINTAINERS | 1 + drivers/net/Kconfig | 1 + drivers/net/ethernet/intel/Kconfig | 1 + drivers/net/ethernet/intel/iavf/iavf.h | 13 + drivers/net/ethernet/intel/iavf/iavf_main.c | 159 +- drivers/net/ethernet/intel/iavf/iavf_txrx.h | 2 + .../net/ethernet/intel/iavf/iavf_virtchnl.c | 157 +- drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_base.c | 2 + drivers/net/ethernet/intel/ice/ice_common.c | 21 + .../net/ethernet/intel/ice/ice_hw_autogen.h | 8 + drivers/net/ethernet/intel/ice/ice_txrx.h | 1 + drivers/net/ethernet/intel/ice/ice_type.h | 1 + drivers/net/ethernet/intel/ice/ice_vf_lib.h | 8 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 335 ++++ drivers/net/ethernet/intel/ice/ice_virtchnl.h | 11 + .../intel/ice/ice_virtchnl_allowlist.c | 6 + drivers/net/netdevsim/netdev.c | 39 + drivers/net/vxlan/vxlan_mdb.c | 2 +- include/linux/avf/virtchnl.h | 119 ++ include/linux/netdevice.h | 21 + include/linux/netlink.h | 5 +- include/net/genetlink.h | 8 +- include/net/net_shaper.h | 120 ++ include/uapi/linux/net_shaper.h | 97 ++ net/Kconfig | 3 + net/Makefile | 1 + net/core/dev.c | 8 + net/core/dev.h | 10 + net/core/netdev-genl.c | 2 +- net/core/rtnetlink.c | 2 +- net/devlink/devl_internal.h | 2 +- net/ethtool/rss.c | 2 +- net/netfilter/nf_conntrack_netlink.c | 2 +- net/netlink/genetlink.c | 4 +- net/shaper/Makefile | 8 + net/shaper/shaper.c | 1460 +++++++++++++++++ net/shaper/shaper_nl_gen.c | 162 ++ net/shaper/shaper_nl_gen.h | 46 + tools/testing/selftests/drivers/net/Makefile | 1 + tools/testing/selftests/drivers/net/shaper.py | 458 ++++++ .../testing/selftests/net/lib/py/__init__.py | 1 + tools/testing/selftests/net/lib/py/ynl.py | 5 + 45 files changed, 3707 insertions(+), 14 deletions(-) create mode 100644 Documentation/netlink/specs/net_shaper.yaml create mode 100644 include/net/net_shaper.h create mode 100644 include/uapi/linux/net_shaper.h create mode 100644 net/shaper/Makefile create mode 100644 net/shaper/shaper.c create mode 100644 net/shaper/shaper_nl_gen.c create mode 100644 net/shaper/shaper_nl_gen.h create mode 100755 tools/testing/selftests/drivers/net/shaper.py
Signature · 1 line
2.45.2
Paolo Abeni
maintainerPATCH v6 1/15 genetlink: extend info user-storage to match NL cb ctx
This allows a more uniform implementation of non-dump and dump operations, and will be used later in the series to avoid some per-operation allocation.
Additionally rename the NL_ASSERT_DUMP_CTX_FITS macro, to fit a more extended usage.
--- drivers/net/vxlan/vxlan_mdb.c | 2 +- include/linux/netlink.h | 5 +++-- include/net/genetlink.h | 8 ++++++-- net/core/netdev-genl.c | 2 +- net/core/rtnetlink.c | 2 +- net/devlink/devl_internal.h | 2 +- net/ethtool/rss.c | 2 +- net/netfilter/nf_conntrack_netlink.c | 2 +- net/netlink/genetlink.c | 4 ++-- 9 files changed, 17 insertions(+), 12 deletions(-)
drivers/net/vxlan/vxlan_mdb.c +1 -1
@@ -284,7 +284,7 @@ int vxlan_mdb_dump(struct net_device *dev, struct sk_buff *skb,ASSERT_RTNL (); -NL_ASSERT_DUMP_CTX_FITS (struct vxlan_mdb_dump_ctx ); +NL_ASSERT_CTX_FITS (struct vxlan_mdb_dump_ctx ); nlh = nlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq, RTM_NEWMDB, sizeof(*bpm),
include/linux/netlink.h +3 -2
@@ -34,6 +34,7 @@ struct netlink_skb_parms {#define NETLINK_CB (skb) (*(struct netlink_skb_parms*)&((skb)->cb))#define NETLINK_CREDS (skb) (&NETLINK_CB((skb)).creds) +#define NETLINK_CTX_SIZE 48void netlink_table_grab (void );
@@ -293,7 +294,7 @@ struct netlink_callback {int flags;bool strict_check;union { -u8 ctx[48 ]; +u8 ctx[NETLINK_CTX_SIZE ];/* args is deprecated. Cast a struct over ctx instead * for proper type safety.
@@ -302,7 +303,7 @@ struct netlink_callback { }; }; -#define NL_ASSERT_DUMP_CTX_FITS (type_name) \ +#define NL_ASSERT_CTX_FITS (type_name) \ BUILD_BUG_ON(sizeof(type_name) > \ sizeof_field(struct netlink_callback, ctx))
include/net/genetlink.h +6 -2
@@ -124,7 +124,8 @@ struct genl_family {* @genlhdr: generic netlink message header * @attrs: netlink attributes * @_net: network namespace -* @user_ptr: user pointers +* @ctx: storage space for the use by the family +* @user_ptr: user pointers (deprecated, use ctx instead) * @extack: extended ACK report struct */ struct genl_info {
@@ -135,7 +136,10 @@ struct genl_info {struct genlmsghdr * genlhdr;struct nlattr ** attrs;possible_net_t _net; -void * user_ptr[2 ]; +union { +u8 ctx[NETLINK_CTX_SIZE ]; +void * user_ptr[2 ]; + };struct netlink_ext_ack *extack; };
net/core/netdev-genl.c +1 -1
@@ -23,7 +23,7 @@ struct netdev_nl_dump_ctx {static struct netdev_nl_dump_ctx *netdev_dump_ctx (struct netlink_callback *cb) { -NL_ASSERT_DUMP_CTX_FITS (struct netdev_nl_dump_ctx ); +NL_ASSERT_CTX_FITS (struct netdev_nl_dump_ctx );return (struct netdev_nl_dump_ctx *)cb->ctx; }
net/core/rtnetlink.c +1 -1
@@ -6169,7 +6169,7 @@ static int rtnl_mdb_dump(struct sk_buff *skb, struct netlink_callback *cb)int idx, s_idx;int err; -NL_ASSERT_DUMP_CTX_FITS (struct rtnl_mdb_dump_ctx ); +NL_ASSERT_CTX_FITS (struct rtnl_mdb_dump_ctx );if (cb->strict_check) { err =rtnl_mdb_valid_dump_req (cb->nlh, cb->extack);
net/devlink/devl_internal.h +1 -1
@@ -166,7 +166,7 @@ int devlink_nl_dumpit(struct sk_buff *msg, struct netlink_callback *cb,static inline struct devlink_nl_dump_state *devlink_dump_state (struct netlink_callback *cb) { -NL_ASSERT_DUMP_CTX_FITS (struct devlink_nl_dump_state ); +NL_ASSERT_CTX_FITS (struct devlink_nl_dump_state );return (struct devlink_nl_dump_state *)cb->ctx; }
net/ethtool/rss.c +1 -1
@@ -224,7 +224,7 @@ struct rss_nl_dump_ctx {static struct rss_nl_dump_ctx *rss_dump_ctx (struct netlink_callback *cb) { -NL_ASSERT_DUMP_CTX_FITS (struct rss_nl_dump_ctx ); +NL_ASSERT_CTX_FITS (struct rss_nl_dump_ctx );return (struct rss_nl_dump_ctx *)cb->ctx; }
net/netfilter/nf_conntrack_netlink.c +1 -1
@@ -3876,7 +3876,7 @@ static int __init ctnetlink_init(void) {int ret; -NL_ASSERT_DUMP_CTX_FITS (struct ctnetlink_list_dump_ctx ); +NL_ASSERT_CTX_FITS (struct ctnetlink_list_dump_ctx ); ret =nfnetlink_subsys_register (&ctnl_subsys);if (ret <0 ) {
net/netlink/genetlink.c +2 -2
@@ -997,7 +997,7 @@ static int genl_start(struct netlink_callback *cb) info->info.attrs = attrs;genl_info_net_set (&info->info,sock_net (cb->skb->sk)); info->info.extack = cb->extack; -memset (&info->info.user_ptr,0 ,sizeof (info->info.user_ptr)); +memset (&info->info.ctx,0 ,sizeof (info->info.ctx)); cb->data = info;if (ops->start) {
@@ -1104,7 +1104,7 @@ static int genl_family_rcv_msg_doit(const struct genl_family *family, info.attrs = attrbuf; info.extack = extack;genl_info_net_set (&info, net); -memset (&info.user_ptr,0 ,sizeof (info.user_ptr)); +memset (&info.ctx,0 ,sizeof (info.ctx));if (ops->pre_doit) { err = ops->pre_doit (ops, skb, &info);
Signature · 1 line
2.45.2
Re: [PATCH v6 net-next 01/15] genetlink: extend info user-storage to match NL cb ctx
Quoted 9 lines · On Wed, 4 Sep 2024 15:53:33 +0200 Paolo Abeni wrote:
This allows a more uniform implementation of non-dump and dump operations, and will be used later in the series to avoid some per-operation allocation.
Additionally rename the
NL_ASSERT_DUMP_CTX_FITSmacro, to fit a more extended usage.Suggested-by Jakub Kicinski <[email protected]>Signed-off-by Paolo Abeni <[email protected]>
Thank you!
Paolo Abeni
maintainerPATCH v6 2/15 netlink: spec: add shaper YAML spec
In reply to Paolo AbeniDefine the user-space visible interface to query, configure and delete network shapers via yaml definition.
Add dummy implementations for the relevant NL callbacks.
set() and delete() operations touch a single shaper creating/updating or deleting it.
The group() operation creates a shaper's group, nesting multiple input shapers under the specified output shaper.
---
v5 -> v6:
- moved back ifindex out of binding attr, drop the latter
- restrict leaves attributes to scheduling-related ones
v4 -> v5:
- moved ifindex under the binding attr
- moved id, scope to new attr set
- rename 'root' as 'node'
- deleted unused 'info' subset
- a lot of doc update and fixup
- removed empty black line at MAKEFILE eof
v3 -> v4:
- spec file rename
- always use '@' for references
- detached scope -> node scope
- inputs/output -> leaves/root
- deduplicate leaves/root policy
- get/dump/group return ifindex, too
- added some general introduction to the doc
RFC v1 -> RFC v2:
- u64 -> uint
- net_shapers -> net-shapers
- documented all the attributes
- dropped [ admin-perm ] for get() op
- group op
- set/delete touch a single shaper
--- Documentation/netlink/specs/net_shaper.yaml | 313 ++++++++++++++++++++ MAINTAINERS | 1 + include/uapi/linux/net_shaper.h | 80 +++++ net/Kconfig | 3 + net/Makefile | 1 + net/shaper/Makefile | 8 + net/shaper/shaper.c | 55 ++++ net/shaper/shaper_nl_gen.c | 133 +++++++++ net/shaper/shaper_nl_gen.h | 36 +++ 9 files changed, 630 insertions(+) create mode 100644 Documentation/netlink/specs/net_shaper.yaml create mode 100644 include/uapi/linux/net_shaper.h create mode 100644 net/shaper/Makefile create mode 100644 net/shaper/shaper.c create mode 100644 net/shaper/shaper_nl_gen.c create mode 100644 net/shaper/shaper_nl_gen.h
Documentation/netlink/specs/net_shaper.yaml new file +313 -0
@@ -0,0 +1,313 @@ +# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) + +name: net-shaper + +doc: | + Networking HW rate limiting configuration. + + This API allows configuring HW shapers available on the network + devices at different levels (queues, network device) and allows + arbitrary manipulation of the scheduling tree of the involved + shapers. + + Each @shaper is identified within the given device, by an @handle, + comprising both a @scope and an @id. + + Depending on the @scope value, the shapers are attached to specific + HW objects (queues, devices) or, for @node scope, represent a + scheduling group, that can be placed in an arbitrary location of + the scheduling tree. + + Shapers can be created with two different operations: the @set + operation, to create and update a single "attached" shaper, and + the @group operation, to create and update a scheduling + group. Only the @group operation can create @node scope shapers + + Existing shapers can be deleted /reset via the @delete operation. + + The user can query the running configuration via the @get operation. + +definitions: + - + type: enum + name: scope + doc: Defines the shaper @id interpretation. + render-max: true + entries: + - name: unspec + doc: The scope is not specified. + - + name: netdev + doc: The main shaper for the given network device. + - + name: queue + doc: | + The shaper is attached to the given device queue, + the @id represents the queue number. + - + name: node + doc: | + The shaper allows grouping of queues or other + node shapers; can be nested in either @netdev + shapers or other @node shapers, allowing placement + in any location of the scheduling tree, except + leaves and root. + - + type: enum + name: metric + doc: Different metric supported by the shaper. + entries: + - + name: bps + doc: Shaper operates on a bits per second basis. + - + name: pps + doc: Shaper operates on a packets per second basis. + +attribute-sets: + - + name: net-shaper + attributes: + - + name: handle + type: nest + nested-attributes: handle + doc: Unique identifier for the given shaper inside the owning device. + - + name: metric + type: u32 + enum: metric + doc: Metric used by the given shaper for bw-min, bw-max and burst. + - + name: bw-min + type: uint + doc: Minimum Guaranteed bandwidth for the given shaper. + - + name: bw-max + type: uint + doc: Maximum bandwidth for the given shaper or 0 when unlimited. + - + name: burst + type: uint + doc: | + Maximum burst-size for shaping. Should not be interpreted + as a quantum. + - + name: priority + type: u32 + doc: | + Scheduling priority for the given shaper. The priority + scheduling is applied to sibling shapers. + - + name: weight + type: u32 + doc: | + Relative weight for round robin scheduling of the + given shaper. + The scheduling is applied to all sibling shapers + with the same priority. + - + name: ifindex + type: u32 + doc: Interface index owning the specified shaper. + - + name: parent + type: nest + nested-attributes: handle + doc: | + Identifier for the parent of the affected shaper. + Only needed for @group operation. + - + name: leaves + type: nest + multi-attr: true + nested-attributes: leaf-info + doc: | + Describes a set of leaves shapers for a @group operation. + - + name: node + type: nest + nested-attributes: node-info + doc: | + Describes the node shaper for a @group operation. + Differently from @leaves and @shaper allow specifying + the shaper parent handle, too. + - + name: shaper + type: nest + nested-attributes: info + doc: | + Describes a single shaper for a @set operation. + - + name: handle + attributes: + - + name: scope + type: u32 + enum: scope + doc: Defines the shaper @id interpretation. + - + name: id + type: u32 + doc: | + Numeric identifier of a shaper. The id semantic depends on + the scope. For @queue scope it's the queue id and for @node + scope it's the node identifier. + - + name: info + subset-of: net-shaper + attributes: + - + name: handle + - + name: metric + - + name: bw-min + - + name: bw-max + - + name: burst + - + name: priority + - + name: weight + - + name: leaf-info + subset-of: net-shaper + attributes: + - + name: handle + - + name: priority + - + name: weight + - + name: node-info + subset-of: net-shaper + attributes: + - + name: parent + - + name: handle + - + name: metric + - + name: bw-min + - + name: bw-max + - + name: burst + - + name: priority + - + name: weight + +operations: + list: + - + name: get + doc: | + Get information about a shaper for a given device. + attribute-set: net-shaper + + do: + pre: net-shaper-nl-pre-doit + post: net-shaper-nl-post-doit + request: + attributes: &ns-binding + - ifindex + - handle + reply: + attributes: &ns-attrs + - ifindex + - parent + - handle + - metric + - bw-min + - bw-max + - burst + - priority + - weight + + dump: + pre: net-shaper-nl-pre-dumpit + post: net-shaper-nl-post-dumpit + request: + attributes: + - ifindex + reply: + attributes: *ns-attrs + - + name: set + doc: | + Create or update the specified shaper. + The set operation can't be used to create a @node scope shaper, + use the @group operation instead. + attribute-set: net-shaper + flags: [ admin-perm ] + + do: + pre: net-shaper-nl-pre-doit + post: net-shaper-nl-post-doit + request: + attributes: + - ifindex + - shaper + + - + name: delete + doc: | + Clear (remove) the specified shaper. When deleting + a @node shaper, reattach all the node's leaves to the + deleted node's parent. + If, after the removal, the parent shaper has no more + leaves and the parent shaper scope is @node, the parent + node is deleted, recursively. + When deleting a @queue shaper or a @netdev shaper, + the shaper disappears from the hierarchy, but the + queue/device can still send traffic: it has an implicit + node with infinite bandwidth. Queue's implicit node + feeds an implicit RR node at the root of the hierarchy. + attribute-set: net-shaper + flags: [ admin-perm ] + + do: + pre: net-shaper-nl-pre-doit + post: net-shaper-nl-post-doit + request: + attributes: *ns-binding + + - + name: group + doc: | + Create or update a scheduling group, attaching the specified + @leaves shapers under the specified @node, creating the latter, + if needed. + The @leaves shapers scope must be @queue and the @node shaper + scope must be either @node or @netdev. + When the @node shaper has @node scope, if the @handle @id is not + specified, a new shaper of such scope is created, otherwise the + specified @node must already exist. + The @parent handle for the @node shaper is optional in most cases. + For newly created @node scope shaper, the @parent is set by + default to the parent linked to the @leaves before the @group + operation. If, prior to the grouping operation, the @leaves + have different parents, the @node shaper parent must be explicitly + set. + The operation is atomic, on failure no change is applied to + the device shaping configuration, otherwise the @node shaper + full identifier, comprising @binding and @handle, is provided + as the reply. + attribute-set: net-shaper + flags: [ admin-perm ] + + do: + pre: net-shaper-nl-pre-doit + post: net-shaper-nl-post-doit + request: + attributes: + - ifindex + - leaves + - node + reply: + attributes: *ns-binding
MAINTAINERS +1 -0
@@ -15901,6 +15901,7 @@ F: include/linux/platform_data/wiznet.h F: include/uapi/linux/cn_proc.h F: include/uapi/linux/ethtool_netlink.h F: include/uapi/linux/if_* +F: include/uapi/linux/net_shaper.h F: include/uapi/linux/netdev* F: tools/testing/selftests/drivers/net/ X: drivers/net/wireless/
include/uapi/linux/net_shaper.h new file +80 -0
80 lines not on this page · raw message
net/Kconfig +3 -0
@@ -66,6 +66,9 @@ config SKB_DECRYPTED config SKB_EXTENSIONS bool +config NET_SHAPER + bool + menu "Networking options" source "net/packet/Kconfig"
net/Makefile +1 -0
@@ -79,3 +79,4 @@ obj-$(CONFIG_XDP_SOCKETS) += xdp/ obj-$(CONFIG_MPTCP) += mptcp/ obj-$(CONFIG_MCTP) += mctp/ obj-$(CONFIG_NET_HANDSHAKE) += handshake/ +obj-$(CONFIG_NET_SHAPER) += shaper/
net/shaper/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Makefile for the net shaper infrastructure. +# +# Copyright (c) 2024, Red Hat, Inc. +# + +obj-y += shaper.o shaper_nl_gen.o
net/shaper/shaper.c new file +55 -0
55 lines not on this page · raw message
net/shaper/shaper_nl_gen.c new file +133 -0
133 lines not on this page · raw message
net/shaper/shaper_nl_gen.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ +/* Do not edit directly, auto-generated from: */ +/* Documentation/netlink/specs/net_shaper.yaml */ +/* YNL-GEN kernel header */ + +#ifndef _LINUX_NET_SHAPER_GEN_H +#define _LINUX_NET_SHAPER_GEN_H + +#include <net/netlink.h> +#include <net/genetlink.h> + +#include <uapi/linux/net_shaper.h> + +/* Common nested types */ +extern const struct nla_policy net_shaper_handle_nl_policy[NET_SHAPER_A_HANDLE_ID +1 ]; +extern const struct nla_policy net_shaper_info_nl_policy[NET_SHAPER_A_WEIGHT +1 ]; +extern const struct nla_policy net_shaper_leaf_info_nl_policy[NET_SHAPER_A_WEIGHT +1 ]; +extern const struct nla_policy net_shaper_node_info_nl_policy[NET_SHAPER_A_PARENT +1 ]; + +int net_shaper_nl_pre_doit (const struct genl_split_ops *ops, +struct sk_buff *skb,struct genl_info *info); +void +net_shaper_nl_post_doit (const struct genl_split_ops *ops,struct sk_buff *skb, +struct genl_info *info); +int net_shaper_nl_pre_dumpit (struct netlink_callback *cb); +int net_shaper_nl_post_dumpit (struct netlink_callback *cb); + +int net_shaper_nl_get_doit (struct sk_buff *skb,struct genl_info *info); +int net_shaper_nl_get_dumpit (struct sk_buff *skb,struct netlink_callback *cb); +int net_shaper_nl_set_doit (struct sk_buff *skb,struct genl_info *info); +int net_shaper_nl_delete_doit (struct sk_buff *skb,struct genl_info *info); +int net_shaper_nl_group_doit (struct sk_buff *skb,struct genl_info *info); + +extern struct genl_family net_shaper_nl_family; + +#endif /* _LINUX_NET_SHAPER_GEN_H */
Signature · 1 line
2.45.2
Jakub Kicinski
maintainerRe: [PATCH v6 net-next 02/15] netlink: spec: add shaper YAML spec
Quoted 12 lines · On Wed, 4 Sep 2024 15:53:34 +0200 Paolo Abeni wrote:
Documentation/netlink/specs/net_shaper.yaml new file +12
-0+doc: | + Networking HW rate limiting configuration. + + This API allows configuring HW shapers available on the network + devices at different levels (queues, network device) and allows + arbitrary manipulation of the scheduling tree of the involved + shapers. + + Each @shaper is identified within the given device, by an @handle, + comprising both a @scope and an @id. + + Depending on the @scope value, the shapers are attached to specific
Documentation/netlink/specs/net_shaper.yaml line 26
Quoted 10 lines
Documentation/netlink/specs/net_shaper.yaml new file +10
-0+ HW objects (queues, devices) or, for @node scope, represent a + scheduling group, that can be placed in an arbitrary location of + the scheduling tree. + + Shapers can be created with two different operations: the @set + operation, to create and update a single "attached" shaper, and + the @group operation, to create and update a scheduling + group. Only the @group operation can create @node scope shapers + + Existing shapers can be deleted /reset via the @delete operation.
nit: space before the / ?
Documentation/netlink/specs/net_shaper.yaml line 84
Quoted 3 lines
Documentation/netlink/specs/net_shaper.yaml new file +3
-0+ name: bw-min + type: uint + doc: Minimum Guaranteed bandwidth for the given shaper.
I think I asked to remove "Minimum"? Both "guaranteed" and "minimum" express the fact that we can't go lower, so it's a bit of a pleonasm.
Documentation/netlink/specs/net_shaper.yaml line 134
Quoted 8 lines
Documentation/netlink/specs/net_shaper.yaml new file +8
-0+ - + name: node + type: nest + nested-attributes: node-info + doc: | + Describes the node shaper for a @group operation. + Differently from @leaves and @shaper allow specifying + the shaper parent handle, too.
Parent handle is inside node scope? Why are leaves outside and parent inside? Both should be at the same scope, preferably main scope.
Documentation/netlink/specs/net_shaper.yaml line 140
Quoted 6 lines
Documentation/netlink/specs/net_shaper.yaml new file +6
-0+ - + name: shaper + type: nest + nested-attributes: info + doc: | + Describes a single shaper for a @set operation.
Why does this level of nesting exist? With the exception of ifindex all attributes for SET are nested inside this..
Paolo Abeni
maintainerRe: [PATCH v6 net-next 02/15] netlink: spec: add shaper YAML spec
oops,
I unintentionally stripped the recipients list, in my previous reply. Re-adding all of them. I'm sorry for the duplicates.
Quoted 10 lines · Jakub Kicinski
Documentation/netlink/specs/net_shaper.yaml line 134
Quoted 8 lines · On Wed, 4 Sep 2024 15:53:34 +0200 Paolo Abeni wrote:
Documentation/netlink/specs/net_shaper.yaml new file +8
-0+ - + name: node + type: nest + nested-attributes: node-info + doc: | + Describes the node shaper for a @group operation. + Differently from @leaves and @shaper allow specifying + the shaper parent handle, too.
Quoted 2 lines
Parent handle is inside node scope? Why are leaves outside and parent inside? Both should be at the same scope, preferably main scope.
The group() op receives as arguments, in the main scope:
ifindex
node
leaves
'parent' is a nested attribute for 'node', exactly as 'handle'. We need to specify both to identify the 'node' itself (via the 'handle') and to specify where in the hierarchy the 'node' will be located (via the 'parent'). Do I read correctly that you would prefer:
ifindexnode_handlenode_parent
leaves
?
I think the former is more clean/clear.
Quoted 9 lines
Documentation/netlink/specs/net_shaper.yaml line 140
Quoted 6 lines
Documentation/netlink/specs/net_shaper.yaml new file +6
-0+ - + name: shaper + type: nest + nested-attributes: info + doc: | + Describes a single shaper for a @set operation.Why does this level of nesting exist? With the exception of ifindex all attributes for SET are nested inside this..
Yep, we can drop the nesting level, I think. I used the nesting to be have a syntax similar to the with group() operation.
Thanks,
Paolo
Jakub Kicinski
maintainerRe: [PATCH v6 net-next 02/15] netlink: spec: add shaper YAML spec
Quoted 25 lines · On Thu, 5 Sep 2024 16:51:00 +0200 Paolo Abeni wrote:
Quoted 10 lines · Jakub Kicinski
Documentation/netlink/specs/net_shaper.yaml line 134
Quoted 8 lines · On Wed, 4 Sep 2024 15:53:34 +0200 Paolo Abeni wrote:
Documentation/netlink/specs/net_shaper.yaml new file +8
-0+ - + name: node + type: nest + nested-attributes: node-info + doc: | + Describes the node shaper for a @group operation. + Differently from @leaves and @shaper allow specifying + the shaper parent handle, too.Quoted 2 lines
Parent handle is inside node scope? Why are leaves outside and parent inside? Both should be at the same scope, preferably main scope.
The
group()op receives as arguments, in the main scope:ifindex
node
leaves'parent' is a nested attribute for 'node', exactly as 'handle'. We need to specify both to identify the 'node' itself (via the 'handle') and to specify where in the hierarchy the 'node' will be located (via the 'parent'). Do I read correctly that you would prefer:
Quoted 4 lines
ifindex
node_handlenode_parent
leaves
I don't see example uses in the cover letter or the test so there's a good chance I'm missing something, but... why node_parent? The only thing you need to know about the parent is its handle, so just "parent", right?
Also why node_handle? Just "handle", and other attrs of the node can live in the main scope.
Unless you have a strong reason to do this to simplify the code - "from netlink perspective" it looks like unnecessary nesting. The operation arguments describe the node, there's no need to nest things in another layer.
Paolo Abeni
maintainerRe: [PATCH v6 net-next 02/15] netlink: spec: add shaper YAML spec
Quoted 31 lines · Jakub Kicinski
Quoted 25 lines · On Thu, 5 Sep 2024 16:51:00 +0200 Paolo Abeni wrote:
Quoted 10 lines · Jakub Kicinski
Documentation/netlink/specs/net_shaper.yaml line 134
Quoted 8 lines · On Wed, 4 Sep 2024 15:53:34 +0200 Paolo Abeni wrote:
Documentation/netlink/specs/net_shaper.yaml new file +8
-0+ - + name: node + type: nest + nested-attributes: node-info + doc: | + Describes the node shaper for a @group operation. + Differently from @leaves and @shaper allow specifying + the shaper parent handle, too.Quoted 2 lines
Parent handle is inside node scope? Why are leaves outside and parent inside? Both should be at the same scope, preferably main scope.
The
group()op receives as arguments, in the main scope:ifindex
node
leaves'parent' is a nested attribute for 'node', exactly as 'handle'. We need to specify both to identify the 'node' itself (via the 'handle') and to specify where in the hierarchy the 'node' will be located (via the 'parent'). Do I read correctly that you would prefer:
Quoted 4 lines
ifindex
node_handlenode_parent
leaves
Quoted 7 lines
I don't see example uses in the cover letter or the test so there's a good chance I'm missing something, but... why
node_parent? The only thing you need to know about the parent is its handle, so just "parent", right?Also why
node_handle? Just "handle", and other attrs of the node can live in the main scope.
I added the 'node_' prefix in the list to stress that such attributes belong to the node.
In the yaml/command line will be only 'handle', 'parent'.
Quoted 4 lines
Unless you have a strong reason to do this to simplify the code - "from netlink perspective" it looks like unnecessary nesting. The operation arguments describe the node, there's no need to nest things in another layer.
Ok, the code complexity should not change much. Side question: currently the node() operation allows specifying all the b/w related attributes for the 'node' shaper, should I keep them? (and move them in the main yaml scope)
Thanks,
Paolo
Jakub Kicinski
maintainerRe: [PATCH v6 net-next 02/15] netlink: spec: add shaper YAML spec
Quoted 5 lines · On Thu, 5 Sep 2024 18:17:42 +0200 Paolo Abeni wrote:
Quoted 5 lines
I don't see example uses in the cover letter or the test so there's a good chance I'm missing something, but... why
node_parent? The only thing you need to know about the parent is its handle, so just "parent", right?
Quoted 7 lines
Quoted 2 lines
Also why
node_handle? Just "handle", and other attrs of the node can live in the main scope.I added the '
node_' prefix in the list to stress that such attributes belong to the node.In the yaml/command line will be only 'handle', 'parent'.
And the scope inside parent is 'handle', not subset of 'net-shaper'? Just to be 100% sure :)
Quoted 9 lines
Quoted 4 lines
Unless you have a strong reason to do this to simplify the code - "from netlink perspective" it looks like unnecessary nesting. The operation arguments describe the node, there's no need to nest things in another layer.
Ok, the code complexity should not change much. Side question: currently the
node()operation allows specifying all the b/w related attributes for the 'node' shaper, should I keep them? (and move them in the main yaml scope)
Up to you, I was surprised they were there (I expected @group to be solely about creation of the RR node, and rate limit would have to be set via a separate @set). But I don't expect providing rate limit params in @group to be problematic and user space may find it convenient. So I'm neutral.
And yes, they should sit directly at the message level, not in any nest.
Paolo Abeni
maintainerPATCH v6 3/15 net-shapers: implement NL get operation
In reply to Paolo AbeniIntroduce the basic infrastructure to implement the net-shaper core functionality. Each network devices carries a net-shaper cache, the NL get() operation fetches the data from such cache.
The cache is initially empty, will be fill by the set()/group() operation implemented later and is destroyed at device cleanup time.
The net_shaper_fill_handle(), net_shaper_ctx_init(), and net_shaper_generic_pre() implementations handle generic index type attributes, despite the current caller always pass a constant value to avoid more noise in later patches using them with different attributes.
---
v5 -> v6:
- move shaper lock into dev
- add rcu protection to shaper get/dump
- net_shaper_cache_lookup -> net_shaper_lookup
- net_shaper_ctx_init -> net_shaper_ctx_setup
- net_shaper_info -> net_shaper
- net_shaper_data -> net_shaper_hierarcy
- use NL_REQ_ATTR_CHECK where possible
- nla_nest_start_noflag -> nla_parse_nested
- consolidate ctx usage
- capabilities() op returns void
v4 -> v5:
- ops operate on struct binding
- replace net_device * with binding* in most helpers
- include 'ifindex' in get/dump output
- use dev_tracker for real
- user pre/post for dump op, too
- use NL_SET_BAD_ATTR where applicable
- drop redundant/useless kdoc documentation
- add type arg to net_shaper_ctx_init() (moved from later patch)
- factor out generic pre/post helper for later usage in the series
- remove unneeded forward declaration from netdevice.h
- dropped 'inline' modifier in .c file
- dropped black line at net_shaper.h EoF
v3 -> v4:
- add scope prefix
- use forward declaration in the include
- move the handle out of shaper_info
RFC v2 -> RFC v3:
- dev_put() -> netdev_put()
--- Documentation/networking/kapi.rst | 3 + include/linux/netdevice.h | 21 ++ include/net/net_shaper.h | 120 +++++++++++ net/core/dev.c | 6 + net/core/dev.h | 6 + net/shaper/shaper.c | 341 +++++++++++++++++++++++++++++- 6 files changed, 490 insertions(+), 7 deletions(-) create mode 100644 include/net/net_shaper.h
Documentation/networking/kapi.rst +3 -0
@@ -104,6 +104,9 @@ Driver Support .. kernel-doc:: include/linux/netdevice.h :internal: +.. kernel-doc:: include/net/net_shaper.h + :internal: + PHY Support -----------
include/linux/netdevice.h +21 -0
@@ -1603,6 +1603,14 @@ struct net_device_ops {int (*ndo_hwtstamp_set)(struct net_device *dev,struct kernel_hwtstamp_config *kernel_config,struct netlink_ext_ack *extack); + +#if IS_ENABLED (CONFIG_NET_SHAPER ) +/** +* @net_shaper_ops: Device shaping offload operations +* see include/net/net_shapers.h +*/ +const struct net_shaper_ops *net_shaper_ops; +#endif };/**
@@ -2399,6 +2407,19 @@ struct net_device {/** @irq_moder: dim parameters used if IS_ENABLED(CONFIG_DIMLIB). */ struct dim_irq_moder *irq_moder; +/** +* @lock: protects @net_shaper_hierarchy, feel free to use for other +* netdev-scope protection. Ordering: take after rtnl_lock. +*/ +struct mutex lock; + +#if IS_ENABLED (CONFIG_NET_SHAPER ) +/** +* @net_shaper_hierarchy: data tracking the current shaper status +* see include/net/net_shapers.h +*/ +struct net_shaper_hierarchy *net_shaper_hierarchy; +#endif u8 priv[] ____cacheline_aligned__counted_by (priv_len ); }____cacheline_aligned ;
include/net/net_shaper.h new file +120 -0
120 lines not on this page · raw message
net/core/dev.c +6 -0
@@ -10376,6 +10376,8 @@ int register_netdevice(struct net_device *dev)xa_init_flags (&dev->ethtool->rss_ctx,XA_FLAGS_ALLOC1 );mutex_init (&dev->ethtool->rss_lock); +mutex_init (&dev->lock); +spin_lock_init (&dev->addr_list_lock);netdev_set_addr_lockdep_class (dev);
@@ -11400,6 +11402,10 @@ void unregister_netdevice_many_notify(struct list_head *head,mutex_destroy (&dev->ethtool->rss_lock); +net_shaper_flush_netdev (dev); + +mutex_destroy (&dev->lock); +if (skb)rtmsg_ifinfo_send (skb, dev,GFP_KERNEL , portid, nlh);
net/core/dev.h +6 -0
@@ -35,6 +35,12 @@ void dev_addr_flush(struct net_device *dev);int dev_addr_init (struct net_device *dev);void dev_addr_check (struct net_device *dev); +#if IS_ENABLED (CONFIG_NET_SHAPER ) +void net_shaper_flush_netdev (struct net_device *dev); +#else +static inline void net_shaper_flush_netdev (struct net_device *dev) {} +#endif +/* sysctls not referred to from outside net/core/ */ extern int netdev_unregister_timeout_secs;extern int weight_p;
net/shaper/shaper.c +334 -7
378 lines not on this page · raw message
Signature · 1 line
2.45.2
Re: [PATCH v6 net-next 03/15] net-shapers: implement NL get operation
net/shaper/shaper.c line 140
Quoted 8 lines · On Wed, 4 Sep 2024 15:53:35 +0200 Paolo Abeni wrote:
net/shaper/shaper.c +8
-0+static int net_shaper_ctx_setup (const struct genl_info *info,int type, +struct net_shaper_nl_ctx *ctx) +{ +struct net *ns =genl_info_net (info); +struct net_device *dev; +int ifindex; + +memset (ctx,0 ,sizeof (*ctx));
Don't think you need to memset() this? Patch 1 touches the relevant memset()s in the core.
net/shaper/shaper.c line 230
Quoted 12 lines
net/shaper/shaper.c +12
-0+static int net_shaper_generic_pre (struct genl_info *info,int type) +{ +struct net_shaper_nl_ctx *ctx = (struct net_shaper_nl_ctx *)info->ctx; +int ret; + +BUILD_BUG_ON (sizeof (*ctx) >sizeof (info->ctx)); + + ret =net_shaper_ctx_setup (info, type, ctx); +if (ret) +return ret; + +return 0 ;
There seems to be no extra code here at the end of the series so:
return net_shaper_ctx_setup(info, type, ctx);
With those nits addressed:
Paolo Abeni
maintainerPATCH v6 4/15 net-shapers: implement NL set and delete operations
In reply to Paolo AbeniBoth NL operations directly map on the homonymous device shaper callbacks, update accordingly the shapers cache and are serialized via a per device lock.
Implement the cache modification helpers to additionally deal with NODE scope shaper. That will be needed by the group() operation implemented in the next patch.
The delete implementation is partial: does not handle NODE scope shaper yet. Such support will require infrastructure from the next patch and will be implemented later in the series.
---
v5 -> v6:
- use xa_alloc() instead idr
- rcu protection on shaper free
- drop NL_SET_ERR_MSG() on allocation failures
- net_shaper_cache_init() -> net_shaper_hierarchy_setup()
- net_shaper_cache_{pre_insert,commit} -> net_shaper_{pre_insert,commit}
- fix BAD_ADDR handle in net_shaper_parse_info()
v4 -> v5:
- replace net_device * with binding* in most helpers
- move check for scope NONE handle at parse time and leverage NL_SET_BAD_ATTR()
- move the default parent initialization to net_shaper_parse_info_nest()
v3 -> v4:
- add locking
- helper rename
RFC v2 -> RFC v3:
- dev_put() -> netdev_put()
--- net/shaper/shaper.c | 403 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 401 insertions(+), 2 deletions(-)
net/shaper/shaper.c +401 -2
@@ -36,6 +36,24 @@ static struct net_shaper_binding *net_shaper_binding_from_ctx(void *ctx)return &((struct net_shaper_nl_ctx *)ctx)->binding; } +static void net_shaper_lock (struct net_shaper_binding *binding) +{ +switch (binding->type) { +case NET_SHAPER_BINDING_TYPE_NETDEV : +mutex_lock (&binding->netdev->lock); +break ; + } +} + +static void net_shaper_unlock (struct net_shaper_binding *binding) +{ +switch (binding->type) { +case NET_SHAPER_BINDING_TYPE_NETDEV : +mutex_unlock (&binding->netdev->lock); +break ; + } +} +static struct net_shaper_hierarchy *net_shaper_hierarchy (struct net_shaper_binding *binding) {
@@ -47,6 +65,16 @@ net_shaper_hierarchy(struct net_shaper_binding *binding)return NULL ; } +static const struct net_shaper_ops * +net_shaper_ops (struct net_shaper_binding *binding) +{ +if (binding->type ==NET_SHAPER_BINDING_TYPE_NETDEV ) +return binding->netdev->netdev_ops->net_shaper_ops; + +/* No other type supported yet. */ +return NULL ; +} +static int net_shaper_fill_binding (struct sk_buff *msg,const struct net_shaper_binding *binding,u32 type)
@@ -171,6 +199,37 @@ static u32 net_shaper_handle_to_index(const struct net_shaper_handle *handle)FIELD_PREP (NET_SHAPER_ID_MASK , handle->id); } +static void net_shaper_index_to_handle (u32 index, +struct net_shaper_handle *handle) +{ + handle->scope =FIELD_GET (NET_SHAPER_SCOPE_MASK , index); + handle->id =FIELD_GET (NET_SHAPER_ID_MASK , index); +} + +static void net_shaper_default_parent (const struct net_shaper_handle *handle, +struct net_shaper_handle *parent) +{ +switch (handle->scope) { +case NET_SHAPER_SCOPE_UNSPEC : +case NET_SHAPER_SCOPE_NETDEV : +case __NET_SHAPER_SCOPE_MAX: + parent->scope =NET_SHAPER_SCOPE_UNSPEC ; +break ; + +case NET_SHAPER_SCOPE_QUEUE : +case NET_SHAPER_SCOPE_NODE : + parent->scope =NET_SHAPER_SCOPE_NETDEV ; +break ; + } + parent->id =0 ; +} + +/* +* MARK_0 is already in use due to XA_FLAGS_ALLOC, can't reuse such flag as +* it's cleared by xa_store(). +*/ +#define NET_SHAPER_NOT_VALID XA_MARK_1 +static struct net_shaper *net_shaper_lookup (struct net_shaper_binding *binding,const struct net_shaper_handle *handle)
@@ -179,6 +238,135 @@ net_shaper_lookup(struct net_shaper_binding *binding,u32 index =net_shaper_handle_to_index (handle);return hierarchy ?xa_load (&hierarchy->shapers, index) :NULL ; +if (!hierarchy ||xa_get_mark (&hierarchy->shapers, index, +NET_SHAPER_NOT_VALID )) +return NULL ; + +return xa_load (&hierarchy->shapers, index); +} + +/* Allocate on demand the per device shaper's hierarchy container. +* Called under the net shaper lock +*/ +static struct net_shaper_hierarchy * +net_shaper_hierarchy_setup (struct net_shaper_binding *binding) +{ +struct net_shaper_hierarchy *hierarchy =net_shaper_hierarchy (binding); + +if (hierarchy) +return hierarchy; + + hierarchy =kmalloc (sizeof (*hierarchy),GFP_KERNEL ); +if (!hierarchy) +return NULL ; + +/* The flag is required for ID allocation */ +xa_init_flags (&hierarchy->shapers,XA_FLAGS_ALLOC ); + +switch (binding->type) { +case NET_SHAPER_BINDING_TYPE_NETDEV : +/* Pairs with READ_ONCE in net_shaper_hierarchy. */ +WRITE_ONCE (binding->netdev->net_shaper_hierarchy, hierarchy); +break ; + } +return hierarchy; +} + +/* Prepare the hierarchy container to actually insert the given shaper, doing +* in advance the needed allocations. +*/ +static int net_shaper_pre_insert (struct net_shaper_binding *binding, +struct net_shaper_handle *handle, +struct netlink_ext_ack *extack) +{ +struct net_shaper_hierarchy *hierarchy =net_shaper_hierarchy (binding); +struct net_shaper *prev, *cur; +bool id_allocated = false; +int ret, index; + +if (!hierarchy) +return -ENOMEM ; + + index =net_shaper_handle_to_index (handle); + cur =xa_load (&hierarchy->shapers, index); +if (cur) +return 0 ; + +/* Allocated a new id, if needed. */ +if (handle->scope ==NET_SHAPER_SCOPE_NODE && + handle->id ==NET_SHAPER_ID_UNSPEC ) { +u32 min, max; + + handle->id =NET_SHAPER_ID_MASK -1 ; + max =net_shaper_handle_to_index (handle); + handle->id =0 ; + min =net_shaper_handle_to_index (handle); + + ret =xa_alloc (&hierarchy->shapers, &index,NULL , +XA_LIMIT (min, max),GFP_KERNEL ); +if (ret <0 ) { +NL_SET_ERR_MSG (extack,"Can't allocate new id for NODE shaper" ); +return ret; + } + +net_shaper_index_to_handle (index, handle); + id_allocated = true; + } + + cur =kzalloc (sizeof (*cur),GFP_KERNEL ); +if (!cur) { + ret = -ENOMEM ; + goto free_id; + } + +/* Mark 'tentative' shaper inside the hierarchy container. +* xa_set_mark is a no-op if the previous store fails. +*/ +xa_lock (&hierarchy->shapers); + prev =__xa_store (&hierarchy->shapers, index, cur,GFP_KERNEL ); +__xa_set_mark (&hierarchy->shapers, index,NET_SHAPER_NOT_VALID ); +xa_unlock (&hierarchy->shapers); +if (xa_err (prev)) { +NL_SET_ERR_MSG (extack,"Can't insert shaper into device store" ); +kfree_rcu (cur, rcu); + ret =xa_err (prev); + goto free_id; + } +return 0 ; + +free_id: +if (id_allocated) +xa_erase (&hierarchy->shapers, index); +return ret; +} + +/* Commit the tentative insert with the actual values. +* Must be called only after a successful net_shaper_pre_insert(). +*/ +static void net_shaper_commit (struct net_shaper_binding *binding, +int nr_shapers,const struct net_shaper *shapers) +{ +struct net_shaper_hierarchy *hierarchy =net_shaper_hierarchy (binding); +struct net_shaper *cur; +int index; +int i; + +xa_lock (&hierarchy->shapers); +for (i =0 ; i < nr_shapers; ++i) { + index =net_shaper_handle_to_index (&shapers[i].handle); + + cur =xa_load (&hierarchy->shapers, index); +if (WARN_ON_ONCE (!cur)) +continue ; + +/* Successful update: drop the tentative mark +* and update the hierarchy container. +*/ +__xa_clear_mark (&hierarchy->shapers, index, +NET_SHAPER_NOT_VALID ); + *cur = shapers[i]; + } +xa_unlock (&hierarchy->shapers); } static int net_shaper_parse_handle(const struct nlattr *attr,
@@ -216,6 +404,85 @@ static int net_shaper_parse_handle(const struct nlattr *attr,return 0 ; } +static int net_shaper_parse_info (struct net_shaper_binding *binding, +struct nlattr **tb, +const struct genl_info *info, +struct net_shaper *shaper, +bool *exists) +{ +struct net_shaper *old; +int ret; + +/* The shaper handle is the only mandatory attribute. */ +if (NL_REQ_ATTR_CHECK (info->extack,NULL , tb,NET_SHAPER_A_HANDLE )) +return -EINVAL ; + + ret =net_shaper_parse_handle (tb[NET_SHAPER_A_HANDLE ], info, + &shaper->handle); +if (ret) +return ret; + +if (shaper->handle.scope ==NET_SHAPER_SCOPE_UNSPEC ) { +NL_SET_BAD_ATTR (info->extack, tb[NET_SHAPER_A_HANDLE ]); +return -EINVAL ; + } + +/* Fetch existing hierarchy, if any, so that user provide info will +* incrementally update the existing shaper configuration. +*/ +rcu_read_lock (); + old =net_shaper_lookup (binding, &shaper->handle); +if (old) + *shaper = *old; + *exists = !!old; +rcu_read_unlock (); + +if (tb[NET_SHAPER_A_METRIC ]) + shaper->metric =nla_get_u32 (tb[NET_SHAPER_A_METRIC ]); + +if (tb[NET_SHAPER_A_BW_MIN ]) + shaper->bw_min =nla_get_uint (tb[NET_SHAPER_A_BW_MIN ]); + +if (tb[NET_SHAPER_A_BW_MAX ]) + shaper->bw_max =nla_get_uint (tb[NET_SHAPER_A_BW_MAX ]); + +if (tb[NET_SHAPER_A_BURST ]) + shaper->burst =nla_get_uint (tb[NET_SHAPER_A_BURST ]); + +if (tb[NET_SHAPER_A_PRIORITY ]) + shaper->priority =nla_get_u32 (tb[NET_SHAPER_A_PRIORITY ]); + +if (tb[NET_SHAPER_A_WEIGHT ]) + shaper->weight =nla_get_u32 (tb[NET_SHAPER_A_WEIGHT ]); +return 0 ; +} + +/* Fetch the existing shaper info and update them with the user-provided +* attributes. +*/ +static int net_shaper_parse_info_nest (struct net_shaper_binding *binding, +const struct nlattr *attr, +const struct genl_info *info, +struct net_shaper *shaper) +{ +struct nlattr *tb[NET_SHAPER_A_WEIGHT +1 ]; +bool exists; +int ret; + + ret =nla_parse_nested (tb,NET_SHAPER_A_WEIGHT , attr, + net_shaper_info_nl_policy, info->extack); +if (ret <0 ) +return ret; + + ret =net_shaper_parse_info (binding, tb, info, shaper, &exists); +if (ret <0 ) +return ret; + +if (!exists) +net_shaper_default_parent (&shaper->handle, &shaper->parent); +return 0 ; +} +static int net_shaper_generic_pre (struct genl_info *info,int type) {struct net_shaper_nl_ctx *ctx = (struct net_shaper_nl_ctx *)info->ctx;
@@ -336,14 +603,143 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb,return ret; } +/* Update the H/W and on success update the hierarchy container, too. */ +static int net_shaper_set (struct net_shaper_binding *binding, +const struct net_shaper *shaper, +struct netlink_ext_ack *extack) +{ +const struct net_shaper_ops *ops =net_shaper_ops (binding); +struct net_shaper_handle handle = shaper->handle; +struct net_shaper_hierarchy *hierarchy; +int ret; + +net_shaper_lock (binding); + + hierarchy =net_shaper_hierarchy_setup (binding); +if (!hierarchy) { + ret = -ENOMEM ; + goto unlock; + } + +/* The 'set' can't create node-scope shapers. */ +if (handle.scope ==NET_SHAPER_SCOPE_NODE && + !net_shaper_lookup (binding, &handle)) { + ret = -ENOENT ; + goto unlock; + } + + ret =net_shaper_pre_insert (binding, &handle, extack); +if (ret) + goto unlock; + + ret = ops->set (binding, shaper, extack); +net_shaper_commit (binding,1 , shaper); + +unlock: +net_shaper_unlock (binding); +return ret; +} +int net_shaper_nl_set_doit (struct sk_buff *skb,struct genl_info *info) { -return -EOPNOTSUPP ; +struct net_shaper_binding *binding; +struct net_shaper shaper; +struct nlattr *attr; +int ret; + +if (GENL_REQ_ATTR_CHECK (info,NET_SHAPER_A_SHAPER )) +return -EINVAL ; + + binding =net_shaper_binding_from_ctx (info->ctx); + attr = info->attrs[NET_SHAPER_A_SHAPER ]; + ret =net_shaper_parse_info_nest (binding, attr, info, &shaper); +if (ret) +return ret; + +return net_shaper_set (binding, &shaper, info->extack); +} + +static int __net_shaper_delete (struct net_shaper_binding *binding, +struct net_shaper *shaper, +struct netlink_ext_ack *extack) +{ +struct net_shaper_hierarchy *hierarchy =net_shaper_hierarchy (binding); +struct net_shaper_handle parent_handle, handle = shaper->handle; +const struct net_shaper_ops *ops =net_shaper_ops (binding); +int ret; + +again: + parent_handle = shaper->parent; + + ret = ops->delete (binding, &handle, extack); +if (ret <0 ) +return ret; + +xa_erase (&hierarchy->shapers,net_shaper_handle_to_index (&handle)); +kfree_rcu (shaper, rcu); + +/* Eventually delete the parent, if it is left over with no leaves. */ +if (parent_handle.scope ==NET_SHAPER_SCOPE_NODE ) { + shaper =net_shaper_lookup (binding, &parent_handle); +if (shaper && !--shaper->leaves) { + handle = parent_handle; + goto again; + } + } +return 0 ; +} + +static int net_shaper_delete (struct net_shaper_binding *binding, +const struct net_shaper_handle *handle, +struct netlink_ext_ack *extack) +{ +struct net_shaper_hierarchy *hierarchy; +struct net_shaper *shaper; +int ret; + +net_shaper_lock (binding); + + hierarchy =net_shaper_hierarchy (binding); +if (!hierarchy) { + ret = -ENOENT ; + goto unlock; + } + + shaper =net_shaper_lookup (binding, handle); +if (!shaper) { + ret = -ENOENT ; + goto unlock; + } + +if (handle->scope ==NET_SHAPER_SCOPE_NODE ) { +/* TODO: implement support for scope NODE delete. */ + ret = -EINVAL ; + goto unlock; + } + + ret =__net_shaper_delete (binding, shaper, extack); + +unlock: +net_shaper_unlock (binding); +return ret; }int net_shaper_nl_delete_doit (struct sk_buff *skb,struct genl_info *info) { -return -EOPNOTSUPP ; +struct net_shaper_binding *binding; +struct net_shaper_handle handle; +int ret; + +if (GENL_REQ_ATTR_CHECK (info,NET_SHAPER_A_HANDLE )) +return -EINVAL ; + + binding =net_shaper_binding_from_ctx (info->ctx); + ret =net_shaper_parse_handle (info->attrs[NET_SHAPER_A_HANDLE ], info, + &handle); +if (ret) +return ret; + +return net_shaper_delete (binding, &handle, info->extack); }static void net_shaper_flush (struct net_shaper_binding *binding)
@@ -355,12 +751,15 @@ static void net_shaper_flush(struct net_shaper_binding *binding)if (!hierarchy)return ; +net_shaper_lock (binding);xa_lock (&hierarchy->shapers);xa_for_each (&hierarchy->shapers, index, cur) {__xa_erase (&hierarchy->shapers, index);kfree (cur); }xa_unlock (&hierarchy->shapers); +net_shaper_unlock (binding); +kfree (hierarchy); }
Signature · 1 line
2.45.2
Paolo Abeni
maintainerPATCH v6 5/15 net-shapers: implement NL group operation
In reply to Paolo AbeniAllow grouping multiple leaves shaper under the given root. The node and the leaves shapers are created, if needed, otherwise the existing shapers are re-linked as requested.
Try hard to pre-allocated the needed resources, to avoid non trivial H/W configuration rollbacks in case of any failure.
---
v5 -> v6:
- pre-allocate msg in net_shaper_nl_group_doit()
- factor out handle_cmp() and parent_from_leaves() helpers
- driver group() implementation is optional
- move the 0 leaves check earlier
- net_shaper_cache_rollback ->net_shaper_rollback
- report the failure on reply generation error (no rollback)
- several net_shaper_group_send_reply() fixup
v4 -> v5:
- replace net_device* with binding* in most helpers
- factor out net_shaper_fill_binding() helper for re-use in later patch
- move most sanity check at parse time and use NL_SET_BAD_ATTR
- reused net_shaper_fill_handle() in net_shaper_group_send_reply() instead of open-coding it.
v3 -> v4:
- cleanup left-over scope node shaper after re-link, as needed
- add locking
- separate arguments for shaper handle
RFC v2 -> RFC v3:
- dev_put() -> netdev_put()
--- net/shaper/shaper.c | 367 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 366 insertions(+), 1 deletion(-)
net/shaper/shaper.c +366 -1
@@ -75,6 +75,24 @@ net_shaper_ops(struct net_shaper_binding *binding)return NULL ; } +/* Count the number of [multi] attributes of the given type. */ +static int net_shaper_list_len (struct genl_info *info,int type) +{ +struct nlattr *attr; +int rem, cnt =0 ; + +nla_for_each_attr_type (attr, type,genlmsg_data (info->genlhdr), +genlmsg_len (info->genlhdr), rem) + cnt++; +return cnt; +} + +static int net_shaper_handle_size (void ) +{ +return nla_total_size (nla_total_size (sizeof (u32)) + +nla_total_size (sizeof (u32))); +} +static int net_shaper_fill_binding (struct sk_buff *msg,const struct net_shaper_binding *binding,u32 type)
@@ -369,6 +387,25 @@ static void net_shaper_commit(struct net_shaper_binding *binding,xa_unlock (&hierarchy->shapers); } +/* Rollback all the tentative inserts from the data container. */ +static void net_shaper_rollback (struct net_shaper_binding *binding) +{ +struct net_shaper_hierarchy *hierarchy =net_shaper_hierarchy (binding); +struct net_shaper *cur; +unsigned long index; + +if (!hierarchy) +return ; + +xa_lock (&hierarchy->shapers); +xa_for_each_marked (&hierarchy->shapers, index, cur, +NET_SHAPER_NOT_VALID ) { +__xa_erase (&hierarchy->shapers, index); +kfree (cur); + } +xa_unlock (&hierarchy->shapers); +} +static int net_shaper_parse_handle (const struct nlattr *attr,const struct genl_info *info,struct net_shaper_handle *handle)
@@ -463,6 +500,7 @@ static int net_shaper_parse_info(struct net_shaper_binding *binding,static int net_shaper_parse_info_nest (struct net_shaper_binding *binding,const struct nlattr *attr,const struct genl_info *info, +const struct net_shaper *node,struct net_shaper *shaper) {struct nlattr *tb[NET_SHAPER_A_WEIGHT +1 ];
@@ -478,11 +516,59 @@ static int net_shaper_parse_info_nest(struct net_shaper_binding *binding,if (ret <0 )return ret; +if (node && shaper->handle.scope !=NET_SHAPER_SCOPE_QUEUE ) { +NL_SET_BAD_ATTR (info->extack, tb[NET_SHAPER_A_HANDLE ]); +return -EINVAL ; + } +if (!exists)net_shaper_default_parent (&shaper->handle, &shaper->parent);return 0 ; } +/* Alike net_parse_shaper_info(), but additionally allow the user specifying +* the shaper's parent handle. +*/ +static int net_shaper_parse_node (struct net_shaper_binding *binding, +const struct nlattr *attr, +const struct genl_info *info, +struct net_shaper *shaper) +{ +struct nlattr *tb[NET_SHAPER_A_PARENT +1 ]; +bool exists; +int ret; + + ret =nla_parse_nested (tb,NET_SHAPER_A_PARENT , attr, + net_shaper_node_info_nl_policy, + info->extack); +if (ret <0 ) +return ret; + + ret =net_shaper_parse_info (binding, tb, info, shaper, &exists); +if (ret) +return ret; + +if (shaper->handle.scope !=NET_SHAPER_SCOPE_NODE && + shaper->handle.scope !=NET_SHAPER_SCOPE_NETDEV ) { +NL_SET_BAD_ATTR (info->extack, tb[NET_SHAPER_A_HANDLE ]); +return -EINVAL ; + } + +if (tb[NET_SHAPER_A_PARENT ]) { + ret =net_shaper_parse_handle (tb[NET_SHAPER_A_PARENT ], info, + &shaper->parent); +if (ret) +return ret; + +if (shaper->parent.scope !=NET_SHAPER_SCOPE_NODE && + shaper->parent.scope !=NET_SHAPER_SCOPE_NETDEV ) { +NL_SET_BAD_ATTR (info->extack, tb[NET_SHAPER_A_PARENT ]); +return -EINVAL ; + } + } +return 0 ; +} +static int net_shaper_generic_pre (struct genl_info *info,int type) {struct net_shaper_nl_ctx *ctx = (struct net_shaper_nl_ctx *)info->ctx;
@@ -652,7 +738,7 @@ int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info) binding =net_shaper_binding_from_ctx (info->ctx); attr = info->attrs[NET_SHAPER_A_SHAPER ]; - ret =net_shaper_parse_info_nest (binding, attr, info, &shaper); + ret =net_shaper_parse_info_nest (binding, attr, info,NULL , &shaper);if (ret)return ret;
@@ -689,6 +775,118 @@ static int __net_shaper_delete(struct net_shaper_binding *binding,return 0 ; } +static int net_shaper_handle_cmp (const struct net_shaper_handle *a, +const struct net_shaper_handle *b) +{ +/* Must avoid holes in struct net_shaper_handle. */ +BUILD_BUG_ON (sizeof (*a) !=8 ); + +return memcmp (a, b,sizeof (*a)); +} + +static int net_shaper_parent_from_leaves (int leaves_count, +const struct net_shaper *leaves, +struct net_shaper *node, +struct netlink_ext_ack *extack) +{ +struct net_shaper_handle parent = leaves[0 ].parent; +int i; + +for (i =1 ; i < leaves_count; ++i) { +if (net_shaper_handle_cmp (&leaves[i].parent, &parent)) { +NL_SET_ERR_MSG_FMT (extack,"All the leaves shapers must have the same old parent" ); +return -EINVAL ; + } + } + + node->parent = parent; +return 0 ; +} + +static int __net_shaper_group (struct net_shaper_binding *binding, +int leaves_count,struct net_shaper *leaves, +struct net_shaper *node, +struct netlink_ext_ack *extack) +{ +const struct net_shaper_ops *ops =net_shaper_ops (binding); +struct net_shaper_handle leaf_handle; +struct net_shaper *parent =NULL ; +int i, ret; + +if (node->handle.scope ==NET_SHAPER_SCOPE_NODE ) { +if (node->handle.id !=NET_SHAPER_ID_UNSPEC && + !net_shaper_lookup (binding, &node->handle)) { +/* The related attribute is not available when +* reaching here from the delete() op. +*/ +NL_SET_ERR_MSG_FMT (extack,"Node shaper %d:%d does not exists" , + node->handle.scope, node->handle.id); +return -ENOENT ; + } + +/* When unspecified, the node parent scope is inherited from +* the leaves. +*/ +if (node->parent.scope ==NET_SHAPER_SCOPE_UNSPEC ) { + ret =net_shaper_parent_from_leaves (leaves_count, + leaves, node, + extack); +if (ret) +return ret; + } + + }else { +net_shaper_default_parent (&node->handle, &node->parent); + } + +if (node->parent.scope ==NET_SHAPER_SCOPE_NODE ) { + parent =net_shaper_lookup (binding, &node->parent); +if (!parent) { +NL_SET_ERR_MSG_FMT (extack,"Node parent shaper %d:%d does not exists" , + node->parent.scope, node->parent.id); +return -ENOENT ; + } + } + +/* For newly created node scope shaper, the following will update +* the handle, due to id allocation. +*/ + ret =net_shaper_pre_insert (binding, &node->handle, extack); +if (ret) +return ret; + +for (i =0 ; i < leaves_count; ++i) { + leaf_handle = leaves[i].handle; + + ret =net_shaper_pre_insert (binding, &leaf_handle, extack); +if (ret) + goto rollback; + +if (!net_shaper_handle_cmp (&leaves[i].parent, &node->handle)) +continue ; + +/* The leaves shapers will be nested to the node, update the +* linking accordingly. +*/ + leaves[i].parent = node->handle; + node->leaves++; + } + + ret = ops->group (binding, leaves_count, leaves, node, extack); +if (ret <0 ) + goto rollback; + +if (parent) + parent->leaves++; +net_shaper_commit (binding,1 , node); +net_shaper_commit (binding, leaves_count, leaves); +return 0 ; + +rollback: +net_shaper_rollback (binding); +return ret; +} +static int net_shaper_delete (struct net_shaper_binding *binding,const struct net_shaper_handle *handle,struct netlink_ext_ack *extack)
@@ -742,6 +940,173 @@ int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)return net_shaper_delete (binding, &handle, info->extack); } +/* Update the H/W and on success update the data container, too. */ +static int net_shaper_group (struct net_shaper_binding *binding, +int leaves_count,struct net_shaper *leaves, +struct net_shaper *node, +struct netlink_ext_ack *extack) +{ +struct net_shaper_hierarchy *hierarchy; +struct net_shaper **old_nodes, *tmp; +int i, ret, old_nodes_count =0 ; + + old_nodes =kcalloc (leaves_count,sizeof (struct net_shaper *), +GFP_KERNEL ); +if (!old_nodes) +return -ENOMEM ; + +net_shaper_lock (binding); + + hierarchy =net_shaper_hierarchy_setup (binding); +if (!hierarchy) { + ret = -ENOMEM ; + goto unlock; + } + +/* Record the nodes shaper that this group() operation can make +* childless for later cleanup. +*/ +for (i =0 ; i < leaves_count; i++) { +if (leaves[i].parent.scope ==NET_SHAPER_SCOPE_NODE && +net_shaper_handle_cmp (&leaves[i].parent, &node->handle)) { + tmp =net_shaper_lookup (binding, &leaves[i].parent); +if (!tmp) +continue ; + + old_nodes[old_nodes_count++] = tmp; + } + } + + ret =__net_shaper_group (binding, leaves_count, leaves, node, extack); + +/* Check if we need to delete any nde left alone by the new leaves +* linkage. +*/ +for (i =0 ; i < old_nodes_count; ++i) { + tmp = old_nodes[i]; + +if (--tmp->leaves >0 ) +continue ; + +/* Errors here are not fatal: the grouping operation is +* completed, and user-space can still explicitly clean-up +* left-over nodes. +*/ +__net_shaper_delete (binding, tmp, extack); + } + +unlock: +net_shaper_unlock (binding); + +kfree (old_nodes); +return ret; +} + +static int net_shaper_group_send_reply (struct net_shaper_binding *binding, +const struct net_shaper_handle *handle, +struct genl_info *info, +struct sk_buff *msg) +{ +void *hdr; + + hdr =genlmsg_iput (msg, info); +if (!hdr) + goto free_msg; + +if (net_shaper_fill_binding (msg, binding,NET_SHAPER_A_IFINDEX ) || +net_shaper_fill_handle (msg, handle,NET_SHAPER_A_HANDLE )) + goto free_msg; + +genlmsg_end (msg, hdr); + +return genlmsg_reply (msg, info); + +free_msg: +/* Should never happen as msg is pre-allocated with enough space. */ +WARN_ONCE (true,"calculated message payload length (%d)" , +net_shaper_handle_size ()); +nlmsg_free (msg); +return -EMSGSIZE ; +} + +int net_shaper_nl_group_doit (struct sk_buff *skb,struct genl_info *info) +{ +struct net_shaper_binding *binding; +struct net_shaper *leaves, node; +int i, ret, rem, leaves_count; +struct sk_buff *msg; +struct nlattr *attr; + +if (GENL_REQ_ATTR_CHECK (info,NET_SHAPER_A_LEAVES ) || +GENL_REQ_ATTR_CHECK (info,NET_SHAPER_A_NODE )) +return -EINVAL ; + + binding =net_shaper_binding_from_ctx (info->ctx); + +/* The group operation is optional. */ +if (!net_shaper_ops (binding)->group) +return -EOPNOTSUPP ; + + leaves_count =net_shaper_list_len (info,NET_SHAPER_A_LEAVES ); +if (!leaves_count) { +NL_SET_BAD_ATTR (info->extack, + info->attrs[NET_SHAPER_A_LEAVES ]); +return -EINVAL ; + } + + leaves =kcalloc (leaves_count,sizeof (struct net_shaper ),GFP_KERNEL ); +if (!leaves) +return -ENOMEM ; + + ret =net_shaper_parse_node (binding, info->attrs[NET_SHAPER_A_NODE ], + info, &node); +if (ret) + goto free_shapers; + + i =0 ; +nla_for_each_attr_type (attr,NET_SHAPER_A_LEAVES , +genlmsg_data (info->genlhdr), +genlmsg_len (info->genlhdr), rem) { +if (WARN_ON_ONCE (i >= leaves_count)) + goto free_shapers; + +/* Leaves parsing uses a broader attribute set than what is +* enforced by the op policy. That is safe, as the NL core +* should parse recursively and reject bad commands long +* before it calls the family handler. +*/ + ret =net_shaper_parse_info_nest (binding, attr, info, + &node, &leaves[i]); +if (ret) + goto free_shapers; + i++; + } + +/* Prepare the msg reply in advance, to avoid device operation +* rollback on allocation failure. +*/ + msg =genlmsg_new (net_shaper_handle_size (),GFP_KERNEL ); +if (!msg) + goto free_shapers; + + ret =net_shaper_group (binding, leaves_count, leaves, &node, + info->extack); +if (ret <0 ) + goto free_msg; + + ret =net_shaper_group_send_reply (binding, &node.handle, info, msg); +if (ret) +GENL_SET_ERR_MSG_FMT (info,"Can't send reply" ); + +free_shapers: +kfree (leaves); +return ret; + +free_msg: +kfree_skb (msg); + goto free_shapers; +} +static void net_shaper_flush (struct net_shaper_binding *binding) {struct net_shaper_hierarchy *hierarchy =net_shaper_hierarchy (binding);
Signature · 1 line
2.45.2
Paolo Abeni
maintainerPATCH v6 6/15 net-shapers: implement delete support for NODE scope shaper
In reply to Paolo AbeniLeverage the previously introduced group operation to implement the removal of NODE scope shaper, re-linking its leaves under the the parent node before actually deleting the specified NODE scope shaper.
---
v4 -> v5:
- replace net_device* with binding* in most helpers
--- net/shaper/shaper.c | 87 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 75 insertions(+), 12 deletions(-)
net/shaper/shaper.c +75 -12
@@ -804,7 +804,8 @@ static int net_shaper_parent_from_leaves(int leaves_count, }static int __net_shaper_group (struct net_shaper_binding *binding, -int leaves_count,struct net_shaper *leaves, +bool update_node,int leaves_count, +struct net_shaper *leaves,struct net_shaper *node,struct netlink_ext_ack *extack) {
@@ -848,12 +849,14 @@ static int __net_shaper_group(struct net_shaper_binding *binding, } } -/* For newly created node scope shaper, the following will update -* the handle, due to id allocation. -*/ - ret =net_shaper_pre_insert (binding, &node->handle, extack); -if (ret) -return ret; +if (update_node) { +/* For newly created node scope shaper, the following will +* update the handle, due to id allocation. +*/ + ret =net_shaper_pre_insert (binding, &node->handle, extack); +if (ret) +return ret; + }for (i =0 ; i < leaves_count; ++i) { leaf_handle = leaves[i].handle;
@@ -878,7 +881,8 @@ static int __net_shaper_group(struct net_shaper_binding *binding,if (parent) parent->leaves++; -net_shaper_commit (binding,1 , node); +if (update_node) +net_shaper_commit (binding,1 , node);net_shaper_commit (binding, leaves_count, leaves);return 0 ;
@@ -887,6 +891,64 @@ static int __net_shaper_group(struct net_shaper_binding *binding,return ret; } +static int net_shaper_pre_del_node (struct net_shaper_binding *binding, +const struct net_shaper *shaper, +struct netlink_ext_ack *extack) +{ +struct net_shaper_hierarchy *hierarchy =net_shaper_hierarchy (binding); +struct net_shaper *cur, *leaves, node = {}; +int ret, leaves_count =0 ; +unsigned long index; +bool update_node; + +if (!shaper->leaves) +return 0 ; + +/* Fetch the new node information. */ + node.handle = shaper->parent; + cur =net_shaper_lookup (binding, &node.handle); +if (cur) { + node = *cur; + }else { +/* A scope NODE shaper can be nested only to the NETDEV scope +* shaper without creating the latter, this check may fail only +* if the data is in inconsistent status. +*/ +if (WARN_ON_ONCE (node.handle.scope !=NET_SHAPER_SCOPE_NETDEV )) +return -EINVAL ; + } + + leaves =kcalloc (shaper->leaves,sizeof (struct net_shaper ), +GFP_KERNEL ); +if (!leaves) +return -ENOMEM ; + +/* Build the leaves arrays. */ +xa_for_each (&hierarchy->shapers, index, cur) { +if (net_shaper_handle_cmp (&cur->parent, &shaper->handle)) +continue ; + +if (WARN_ON_ONCE (leaves_count == shaper->leaves)) { + ret = -EINVAL ; + goto free; + } + + leaves[leaves_count++] = *cur; + } + +/* When re-linking to the netdev shaper, avoid the eventual, implicit, +* creation of the new node, would be surprising since the user is +* doing a delete operation. +*/ + update_node = node.handle.scope !=NET_SHAPER_SCOPE_NETDEV ; + ret =__net_shaper_group (binding, update_node, leaves_count, + leaves, &node, extack); + +free: +kfree (leaves); +return ret; +} +static int net_shaper_delete (struct net_shaper_binding *binding,const struct net_shaper_handle *handle,struct netlink_ext_ack *extack)
@@ -910,9 +972,9 @@ static int net_shaper_delete(struct net_shaper_binding *binding, }if (handle->scope ==NET_SHAPER_SCOPE_NODE ) { -/* TODO: implement support for scope NODE delete. */ - ret = -EINVAL ; - goto unlock; + ret =net_shaper_pre_del_node (binding, shaper, extack); +if (ret) + goto unlock; } ret =__net_shaper_delete (binding, shaper, extack);
@@ -977,7 +1039,8 @@ static int net_shaper_group(struct net_shaper_binding *binding, } } - ret =__net_shaper_group (binding, leaves_count, leaves, node, extack); + ret =__net_shaper_group (binding, true, leaves_count, leaves, node, + extack);/* Check if we need to delete any nde left alone by the new leaves * linkage.
Signature · 1 line
2.45.2
Paolo Abeni
maintainerPATCH v6 7/15 net-shapers: implement shaper cleanup on queue deletion
In reply to Paolo Abenihook into netif_set_real_num_tx_queues() to cleanup any shaper configured on top of the to-be-destroyed TX queues.
--- net/core/dev.c | 2 ++ net/core/dev.h | 4 ++++ net/shaper/shaper.c | 31 +++++++++++++++++++++++++++++++ 3 files changed, 37 insertions(+)
net/core/dev.c +2 -0
@@ -2948,6 +2948,8 @@ int netif_set_real_num_tx_queues(struct net_device *dev, unsigned int txq)if (dev->num_tc)netif_setup_tc (dev, txq); +net_shaper_set_real_num_tx_queues (dev, txq); +dev_qdisc_change_real_num_tx (dev, txq); dev->real_num_tx_queues = txq;
net/core/dev.h +4 -0
@@ -37,8 +37,12 @@ void dev_addr_check(struct net_device *dev);#if IS_ENABLED (CONFIG_NET_SHAPER )void net_shaper_flush_netdev (struct net_device *dev); +void net_shaper_set_real_num_tx_queues (struct net_device *dev, +unsigned int txq);#else static inline void net_shaper_flush_netdev (struct net_device *dev) {} +static inline void net_shaper_set_real_num_tx_queues (struct net_device *dev, +unsigned int txq) {}#endif /* sysctls not referred to from outside net/core/ */
net/shaper/shaper.c +31 -0
@@ -1201,6 +1201,37 @@ void net_shaper_flush_netdev(struct net_device *dev)net_shaper_flush (&binding); } +void net_shaper_set_real_num_tx_queues (struct net_device *dev, +unsigned int txq) +{ +struct net_shaper_hierarchy *hierarchy; +struct net_shaper_binding binding; +int i; + + binding.type =NET_SHAPER_BINDING_TYPE_NETDEV ; + binding.netdev = dev; + hierarchy =net_shaper_hierarchy (&binding); +if (!hierarchy) +return ; + +net_shaper_lock (&binding); + +/* Take action only when decreasing the tx queue number. */ +for (i = txq; i < dev->real_num_tx_queues; ++i) { +struct net_shaper_handle handle; +struct net_shaper *shaper; + + handle.scope =NET_SHAPER_SCOPE_QUEUE ; + handle.id = i; + shaper =net_shaper_lookup (&binding, &handle); +if (!shaper) +continue ; + +__net_shaper_delete (&binding, shaper,NULL ); + } +net_shaper_unlock (&binding); +} +static int __init shaper_init (void ) {return genl_register_family (&net_shaper_nl_family);
Signature · 1 line
2.45.2
Jakub Kicinski
maintainerRe: [PATCH v6 net-next 07/15] net-shapers: implement shaper cleanup on queue deletion
net/core/dev.c int netif_set_real_num_tx_queues(struct net_device *dev, unsigned int txq) · line 2955
Quoted 5 lines · On Wed, 4 Sep 2024 15:53:39 +0200 Paolo Abeni wrote:
net/core/dev.c +2
-0+net_shaper_set_real_num_tx_queues (dev, txq); +dev_qdisc_change_real_num_tx (dev, txq); dev->real_num_tx_queues = txq;
The dev->lock has to be taken here, around those three lines, and then set / group must check QUEUE ids against dev->real_num_tx_queues, no? Otherwise the work net_shaper_set_real_num_tx_queues() does is prone to races?
Paolo Abeni
maintainerRe: [PATCH v6 net-next 07/15] net-shapers: implement shaper cleanup on queue deletion
Quoted 11 lines · Jakub Kicinski
net/core/dev.c int netif_set_real_num_tx_queues(struct net_device *dev, unsigned int txq) · line 2955
Quoted 5 lines · On Wed, 4 Sep 2024 15:53:39 +0200 Paolo Abeni wrote:
net/core/dev.c +2
-0+net_shaper_set_real_num_tx_queues (dev, txq); +dev_qdisc_change_real_num_tx (dev, txq); dev->real_num_tx_queues = txq;The
dev->lockhas to be taken here, around those three lines, and then set / group must check QUEUE ids againstdev->real_num_tx_queues, no? Otherwise the worknet_shaper_set_real_num_tx_queues()does is prone to races?
Yes, I think such race exists, but I'm unsure that tacking the lock around the above code will be enough.
i.e. if the relevant devices has 16 channel queues the set() races with a channel reconf on different CPUs:
CPU 1 CPU 2
set_channels(8)
driver_set_channel()
// actually change the number of queues to
// 8, dev->real_num_tx_queues is still 16
// dev->lock is not held yet because the
// driver still has to call
// netif_set_real_num_tx_queues()
set(QUEUE_15,...) // will pass validation // but queue 15 does not // exist anymore
Acquiring dev->lock around set_channel() will not be enough: some driver change the channels number i.e. when enabling XDP.
I think/fear we need to replace the dev->lock with the rtnl lock to solve the race for good.
/P
Jakub Kicinski
maintainerRe: [PATCH v6 net-next 07/15] net-shapers: implement shaper cleanup on queue deletion
Quoted 7 lines · On Thu, 5 Sep 2024 20:02:38 +0200 Paolo Abeni wrote:
Quoted 4 lines
The
dev->lockhas to be taken here, around those three lines, and then set / group must check QUEUE ids againstdev->real_num_tx_queues, no? Otherwise the worknet_shaper_set_real_num_tx_queues()does is prone to races?Yes, I think such race exists, but I'm unsure that tacking the lock around the above code will be enough.
I think "enough" will be subjective. Right now patch 7 provides no real guarantee.
Quoted 13 lines
i.e. if the relevant devices has 16 channel queues the
set()races with a channel reconf on different CPUs:CPU 1 CPU 2
set_channels(8)
driver_set_channel()
// actually change the number of queues to
// 8,dev->real_num_tx_queuesis still 16
//dev->lockis not held yet because the
// driver still has to call
//netif_set_real_num_tx_queues()
Quoted 4 lines
set(QUEUE_15,...) // will pass validation // but queue 15 does not // exist anymore
That may be true - in my proposal the driver can only expect that once netif_set_real_num_tx_queues() returns core will not issue rate limit ops on disabled queues. Driver has to make sure rate limit ops for old queues are accepted all the way up to the call to set_real and ops for new queues are accepted immediately after.
Importantly, the core's state is always consistent - given both the flushing inside net_shaper_set_real_num_tx_queues() and proposed check would be under netdev->lock.
For the driver -- let me flip the question around -- what do you expect the locking scheme to be in case of channel count change? Alternatively we could just expect the driver to take netdev->lock around the appropriate section of code and we'd do:
void net_shaper_set_real_num_tx_queues(struct net_device *dev, ...)
{
...
if (!READ_ONCE(dev->net_shaper_hierarchy))
return;
lockdep_assert_held(dev->lock);
...
}I had a look at iavf, and there is no relevant locking around the queue count check at all, so that doesn't help..
Quoted 2 lines
Acquiring
dev->lockaroundset_channel()will not be enough: some driver change the channels number i.e. when enabling XDP.
Indeed, trying to lock before calling the driver would be both a huge job and destined to fail.
Quoted 2 lines
I think/fear we need to replace the
dev->lockwith the rtnl lock to solve the race for good.
Maybe :( I think we need *an* answer for:
- how we expect the driver to protect itself (assuming that the racy check in iavf_verify_handle() actually serves some purpose, which may not be true);
- how we ensure consistency of core state (no shapers for queues which don't exist, assuming we agree having shapers for queues which don't exist is counter productive).
Reverting back to rtnl_lock for all would be sad, the scheme of expecting the driver to take netdev->lock could work? It's the model we effectively settled on in devlink. Core->driver callbacks are always locked by the core, for driver->core calls driver should explicitly take the lock (some wrappers for lock+op+unlock are provided).
Paolo Abeni
maintainerRe: [PATCH v6 net-next 07/15] net-shapers: implement shaper cleanup on queue deletion
Quoted 6 lines · Jakub Kicinski
For the driver -- let me flip the question around -- what do you expect the locking scheme to be in case of channel count change? Alternatively we could just expect the driver to take
netdev->lockaround the appropriate section of code and we'd do:void
net_shaper_set_real_num_tx_queues(struct net_device *dev, ...)
Quoted 8 lines
{
...
if (!READ_ONCE(dev->net_shaper_hierarchy))
return;
lockdep_assert_held(dev->lock);
...
}In the IAVF case that will be problematic, as AFAICS the channel reconf is done by 2 consecutive async task, the first task - iavf_reset_task - changes the actual number of channels freeing/allocating the driver resources and the 2nd one - iavf_finish_config - notify the stack issuing netif_set_real_num_tx_queues(). iavf_reset_task can't easily wait for iavf_finish_config due to locking order.
Quoted 2 lines
I had a look at iavf, and there is no relevant locking around the queue count check at all, so that doesn't help.
Yep, that is racy.
Quoted 9 lines
Quoted 2 lines
Acquiring
dev->lockaroundset_channel()will not be enough: some driver change the channels number i.e. when enabling XDP.Indeed, trying to lock before calling the driver would be both a huge job and destined to fail.
Quoted 2 lines
I think/fear we need to replace the
dev->lockwith the rtnl lock to solve the race for good.
Quoted 7 lines
Maybe :( I think we need *an* answer for:
- how we expect the driver to protect itself (assuming that the racy check iniavf_verify_handle()actually serves some purpose, which may not be true);
- how we ensure consistency of core state (no shapers for queues which don't exist, assuming we agree having shapers for queues which don't exist is counter productive).
I agree we must delete shapers on removed/deleted queues. The driver/firmware could reuse the same H/W resources for a different VF and such queue must start in the new VF with a default (no shaping) config.
Quoted 6 lines
Reverting back to
rtnl_lockfor all would be sad, the scheme of expecting the driver to takenetdev->lockcould work? It's the model we effectively settled on in devlink.Core->drivercallbacks are always locked by the core, fordriver->corecalls driver should explicitly take the lock (some wrappers for lock+op+unlock are provided).
I think/guess/hope the following could work:
- the driver wraps the h/w resources reconfiguration and netif_set_real_num_tx_queues() with dev->lock. In the iavf case, that means 2 separate critical sections: in iavf_reset_task() and in iavf_finish_config().
- the core, under dev->lock, checks vs real_num_tx_queues and call the shaper ops
- the iavf shaper callbacks would still need to check the queue id vs the current allocated hw resource number as the shapers ops could run in-between the 2 mentioned critical sections. The iavf driver could still act consistently with the core:
- if real_num_tx_queues < qid < current_allocated_hw_resources set the shaper,
- if current_allocated_hw_resources < qid < real_num_tx_queues do nothing and return success
In both the above scenarios, real_num_tx_queues will be set to current_allocated_hw_resources soon by the upcoming iavf_finish_config(), the core will update the hierarchy accordingly, the status will be consistent.
I think the code should be more clear, let me try to share it ASAP (or please block me soon ;)
Thanks,
Paolo
Jakub Kicinski
maintainerRe: [PATCH v6 net-next 07/15] net-shapers: implement shaper cleanup on queue deletion
Quoted 2 lines · On Fri, 6 Sep 2024 16:25:25 +0200 Paolo Abeni wrote:
I think the code should be more clear, let me try to share it ASAP (or please block me soon ;)
No block, AFAIU - sounds great.
Paolo Abeni
maintainerRe: [PATCH v6 net-next 07/15] net-shapers: implement shaper cleanup on queue deletion
In reply to Paolo AbeniQuoted 34 lines · Paolo Abeni
Quoted 11 lines · Jakub Kicinski
net/core/dev.c int netif_set_real_num_tx_queues(struct net_device *dev, unsigned int txq) · line 2955
Quoted 5 lines · On Wed, 4 Sep 2024 15:53:39 +0200 Paolo Abeni wrote:
net/core/dev.c +2
-0+net_shaper_set_real_num_tx_queues (dev, txq); +dev_qdisc_change_real_num_tx (dev, txq); dev->real_num_tx_queues = txq;The
dev->lockhas to be taken here, around those three lines, and then set / group must check QUEUE ids againstdev->real_num_tx_queues, no? Otherwise the worknet_shaper_set_real_num_tx_queues()does is prone to races?Yes, I think such race exists, but I'm unsure that tacking the lock around the above code will be enough.
i.e. if the relevant devices has 16 channel queues the
set()races with a channel reconf on different CPUs:CPU 1 CPU 2
set_channels(8)
driver_set_channel()
// actually change the number of queues to
// 8,dev->real_num_tx_queuesis still 16
//dev->lockis not held yet because the
// driver still has to call
//netif_set_real_num_tx_queues()set(QUEUE_15,...) // will pass validation // but queue 15 does not // exist anymore
Quoted 5 lines
Acquiring
dev->lockaroundset_channel()will not be enough: some driver change the channels number i.e. when enabling XDP.I think/fear we need to replace the
dev->lockwith the rtnl lock to solve the race for good.
I forgot to mention there is another, easier, alternative: keep the max queue id check in the drivers. The driver will have to acquire and held in the shaper callbacks the relevant driver-specific lock - 'crit_lock', in the iavf case.
Would you be ok with such 2nd option?
Side note: I think the iavf should have to acquire such lock in the callbacks no matter what or access/write to the rings info could be racy.
Thanks,
Paolo
Jakub Kicinski
maintainerRe: [PATCH v6 net-next 07/15] net-shapers: implement shaper cleanup on queue deletion
Quoted 6 lines · On Fri, 6 Sep 2024 16:49:32 +0200 Paolo Abeni wrote:
I forgot to mention there is another, easier, alternative: keep the max queue id check in the drivers. The driver will have to acquire and held in the shaper callbacks the relevant driver-specific lock - '
crit_lock', in the iavf case.Would you be ok with such 2nd option?
I'd strongly prefer if you implemented what was suggested.
Quoted 2 lines
Side note: I think the iavf should have to acquire such lock in the callbacks no matter what or access/write to the rings info could be racy.
Paolo Abeni
maintainerPATCH v6 8/15 netlink: spec: add shaper introspection support
In reply to Paolo AbeniAllow the user-space to fine-grain query the shaping features supported by the NIC on each domain.
---
v5 -> v6:
- shorter cap-related names
v4 -> v5:
- added pre/post helpers for dump op
--- Documentation/netlink/specs/net_shaper.yaml | 88 +++++++++++++++++++++ include/uapi/linux/net_shaper.h | 17 ++++ net/shaper/shaper.c | 32 ++++++++ net/shaper/shaper_nl_gen.c | 29 +++++++ net/shaper/shaper_nl_gen.h | 10 +++ 5 files changed, 176 insertions(+)
Documentation/netlink/specs/net_shaper.yaml +88 -0
@@ -27,6 +27,11 @@ doc: | The user can query the running configuration via the @get operation. + Different devices can provide different feature sets, e.g. with no + support for complex scheduling hierarchy, or for some shaping + parameters. The user can introspect the HW capabilities via the + @cap-get operation. + definitions: - type: enum
@@ -201,6 +206,53 @@ attribute-sets: name: priority - name: weight + - + name: caps + attributes: + - + name: ifindex + type: u32 + doc: Interface index queried for shapers capabilities. + - + name: scope + type: u32 + enum: scope + doc: The scope to which the queried capabilities apply. + - + name: support-metric-bps + type: flag + doc: The device accepts 'bps' metric for bw-min, bw-max and burst. + - + name: support-metric-pps + type: flag + doc: The device accepts 'pps' metric for bw-min, bw-max and burst. + - + name: support-nesting + type: flag + doc: | + The device supports nesting shaper belonging to this scope + below 'node' scoped shapers. Only 'queue' and 'node' + scope can have flag 'support-nesting'. + - + name: support-bw-min + type: flag + doc: The device supports a minimum guaranteed B/W. + - + name: support-bw-max + type: flag + doc: The device supports maximum B/W shaping. + - + name: support-burst + type: flag + doc: The device supports a maximum burst size. + - + name: support-priority + type: flag + doc: The device supports priority scheduling. + - + name: support-weight + type: flag + doc: The device supports weighted round robin scheduling. operations: list:
@@ -311,3 +363,39 @@ operations: - node reply: attributes: *ns-binding + + - + name: cap-get + doc: | + Get the shaper capabilities supported by the given device + for the specified scope. + attribute-set: caps + + do: + pre: net-shaper-nl-cap-pre-doit + post: net-shaper-nl-cap-post-doit + request: + attributes: + - ifindex + - scope + reply: + attributes: &cap-attrs + - ifindex + - scope + - support-metric-bps + - support-metric-pps + - support-nesting + - support-bw-min + - support-bw-max + - support-burst + - support-priority + - support-weight + + dump: + pre: net-shaper-nl-cap-pre-dumpit + post: net-shaper-nl-cap-post-dumpit + request: + attributes: + - ifindex + reply: + attributes: *cap-attrs
include/uapi/linux/net_shaper.h +17 -0
@@ -67,11 +67,28 @@ enum {NET_SHAPER_A_HANDLE_MAX = (__NET_SHAPER_A_HANDLE_MAX -1 ) }; +enum { +NET_SHAPER_A_CAPS_IFINDEX =1 , +NET_SHAPER_A_CAPS_SCOPE , +NET_SHAPER_A_CAPS_SUPPORT_METRIC_BPS , +NET_SHAPER_A_CAPS_SUPPORT_METRIC_PPS , +NET_SHAPER_A_CAPS_SUPPORT_NESTING , +NET_SHAPER_A_CAPS_SUPPORT_BW_MIN , +NET_SHAPER_A_CAPS_SUPPORT_BW_MAX , +NET_SHAPER_A_CAPS_SUPPORT_BURST , +NET_SHAPER_A_CAPS_SUPPORT_PRIORITY , +NET_SHAPER_A_CAPS_SUPPORT_WEIGHT , + + __NET_SHAPER_A_CAPS_MAX, +NET_SHAPER_A_CAPS_MAX = (__NET_SHAPER_A_CAPS_MAX -1 ) +}; +enum {NET_SHAPER_CMD_GET =1 ,NET_SHAPER_CMD_SET ,NET_SHAPER_CMD_DELETE ,NET_SHAPER_CMD_GROUP , +NET_SHAPER_CMD_CAP_GET , __NET_SHAPER_CMD_MAX,NET_SHAPER_CMD_MAX = (__NET_SHAPER_CMD_MAX -1 )
net/shaper/shaper.c +32 -0
@@ -614,6 +614,27 @@ int net_shaper_nl_post_dumpit(struct netlink_callback *cb)return 0 ; } +int net_shaper_nl_cap_pre_doit (const struct genl_split_ops *ops, +struct sk_buff *skb,struct genl_info *info) +{ +return -EOPNOTSUPP ; +} + +void net_shaper_nl_cap_post_doit (const struct genl_split_ops *ops, +struct sk_buff *skb,struct genl_info *info) +{ +} + +int net_shaper_nl_cap_pre_dumpit (struct netlink_callback *cb) +{ +return -EOPNOTSUPP ; +} + +int net_shaper_nl_cap_post_dumpit (struct netlink_callback *cb) +{ +return -EOPNOTSUPP ; +} +int net_shaper_nl_get_doit (struct sk_buff *skb,struct genl_info *info) {struct net_shaper_binding *binding;
@@ -1170,6 +1191,17 @@ int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info) goto free_shapers; } +int net_shaper_nl_cap_get_doit (struct sk_buff *skb,struct genl_info *info) +{ +return 0 ; +} + +int net_shaper_nl_cap_get_dumpit (struct sk_buff *skb, +struct netlink_callback *cb) +{ +return 0 ; +} +static void net_shaper_flush (struct net_shaper_binding *binding) {struct net_shaper_hierarchy *hierarchy =net_shaper_hierarchy (binding);
net/shaper/shaper_nl_gen.c +29 -0
@@ -73,6 +73,17 @@ static const struct nla_policy net_shaper_group_nl_policy[NET_SHAPER_A_NODE + 1] [NET_SHAPER_A_NODE] = NLA_POLICY_NESTED(net_shaper_node_info_nl_policy), }; +/* NET_SHAPER_CMD_CAP_GET - do */ +static const struct nla_policy net_shaper_cap_get_do_nl_policy[NET_SHAPER_A_CAPS_SCOPE +1 ] = { + [NET_SHAPER_A_CAPS_IFINDEX ] = { .type =NLA_U32 , }, + [NET_SHAPER_A_CAPS_SCOPE ] =NLA_POLICY_MAX (NLA_U32 ,3 ), +}; + +/* NET_SHAPER_CMD_CAP_GET - dump */ +static const struct nla_policy net_shaper_cap_get_dump_nl_policy[NET_SHAPER_A_CAPS_IFINDEX +1 ] = { + [NET_SHAPER_A_CAPS_IFINDEX ] = { .type =NLA_U32 , }, +}; +/* Ops table for net_shaper */ static const struct genl_split_ops net_shaper_nl_ops[] = { {
@@ -120,6 +131,24 @@ static const struct genl_split_ops net_shaper_nl_ops[] = { .maxattr = NET_SHAPER_A_NODE, .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO, }, + { + .cmd = NET_SHAPER_CMD_CAP_GET, + .pre_doit = net_shaper_nl_cap_pre_doit, + .doit = net_shaper_nl_cap_get_doit, + .post_doit = net_shaper_nl_cap_post_doit, + .policy = net_shaper_cap_get_do_nl_policy, + .maxattr = NET_SHAPER_A_CAPS_SCOPE, + .flags = GENL_CMD_CAP_DO, + }, + { + .cmd = NET_SHAPER_CMD_CAP_GET, + .start = net_shaper_nl_cap_pre_dumpit, + .dumpit = net_shaper_nl_cap_get_dumpit, + .done = net_shaper_nl_cap_post_dumpit, + .policy = net_shaper_cap_get_dump_nl_policy, + .maxattr = NET_SHAPER_A_CAPS_IFINDEX, + .flags = GENL_CMD_CAP_DUMP, + }, }; struct genl_family net_shaper_nl_family __ro_after_init = {
net/shaper/shaper_nl_gen.h +10 -0
@@ -19,17 +19,27 @@ extern const struct nla_policy net_shaper_node_info_nl_policy[NET_SHAPER_A_PARENint net_shaper_nl_pre_doit (const struct genl_split_ops *ops,struct sk_buff *skb,struct genl_info *info); +int net_shaper_nl_cap_pre_doit (const struct genl_split_ops *ops, +struct sk_buff *skb,struct genl_info *info);void net_shaper_nl_post_doit (const struct genl_split_ops *ops,struct sk_buff *skb,struct genl_info *info); +void +net_shaper_nl_cap_post_doit (const struct genl_split_ops *ops, +struct sk_buff *skb,struct genl_info *info);int net_shaper_nl_pre_dumpit (struct netlink_callback *cb); +int net_shaper_nl_cap_pre_dumpit (struct netlink_callback *cb);int net_shaper_nl_post_dumpit (struct netlink_callback *cb); +int net_shaper_nl_cap_post_dumpit (struct netlink_callback *cb);int net_shaper_nl_get_doit (struct sk_buff *skb,struct genl_info *info);int net_shaper_nl_get_dumpit (struct sk_buff *skb,struct netlink_callback *cb);int net_shaper_nl_set_doit (struct sk_buff *skb,struct genl_info *info);int net_shaper_nl_delete_doit (struct sk_buff *skb,struct genl_info *info);int net_shaper_nl_group_doit (struct sk_buff *skb,struct genl_info *info); +int net_shaper_nl_cap_get_doit (struct sk_buff *skb,struct genl_info *info); +int net_shaper_nl_cap_get_dumpit (struct sk_buff *skb, +struct netlink_callback *cb);extern struct genl_family net_shaper_nl_family;
Signature · 1 line
2.45.2
Paolo Abeni
maintainerPATCH v6 9/15 net: shaper: implement introspection support
In reply to Paolo AbeniThe netlink op is a simple wrapper around the device callback.
Extend the existing fetch_dev() helper adding an attribute argument for the requested device. Reuse such helper in the newly implemented operation.
---
v5 -> v6:
- update to new API
v4 -> v5:
- replace net_device* with binding* in most functions
- de-deplicate some code thanks to more generic helpers in previous patches
v3 -> v4:
- another dev_put() -> netdev_put() conversion, missed in previous iteration
RFC v2 -> v3:
- dev_put() -> netdev_put()
--- net/shaper/shaper.c | 98 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 95 insertions(+), 3 deletions(-)
net/shaper/shaper.c +95 -3
@@ -617,22 +617,29 @@ int net_shaper_nl_post_dumpit(struct netlink_callback *cb)int net_shaper_nl_cap_pre_doit (const struct genl_split_ops *ops,struct sk_buff *skb,struct genl_info *info) { -return -EOPNOTSUPP ; +return net_shaper_generic_pre (info,NET_SHAPER_A_CAPS_IFINDEX ); }void net_shaper_nl_cap_post_doit (const struct genl_split_ops *ops,struct sk_buff *skb,struct genl_info *info) { +net_shaper_generic_post (info); }int net_shaper_nl_cap_pre_dumpit (struct netlink_callback *cb) { -return -EOPNOTSUPP ; +struct net_shaper_nl_ctx *ctx = (struct net_shaper_nl_ctx *)cb->ctx; + +return net_shaper_ctx_setup (genl_info_dump (cb), +NET_SHAPER_A_CAPS_IFINDEX , ctx); }int net_shaper_nl_cap_post_dumpit (struct netlink_callback *cb) { -return -EOPNOTSUPP ; +struct net_shaper_nl_ctx *ctx = (struct net_shaper_nl_ctx *)cb->ctx; + +net_shaper_ctx_cleanup (ctx); +return 0 ; }int net_shaper_nl_get_doit (struct sk_buff *skb,struct genl_info *info)
@@ -1191,14 +1198,99 @@ int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info) goto free_shapers; } +static int +net_shaper_cap_fill_one (struct sk_buff *msg, +struct net_shaper_binding *binding, +enum net_shaper_scope scope,unsigned long flags, +const struct genl_info *info) +{ +unsigned long cur; +void *hdr; + + hdr =genlmsg_iput (msg, info); +if (!hdr) +return -EMSGSIZE ; + +if (net_shaper_fill_binding (msg, binding,NET_SHAPER_A_CAPS_IFINDEX ) || +nla_put_u32 (msg,NET_SHAPER_A_CAPS_SCOPE , scope)) + goto nla_put_failure; + +for (cur =NET_SHAPER_A_CAPS_SUPPORT_METRIC_BPS ; + cur <=NET_SHAPER_A_CAPS_MAX ; ++cur) { +if (flags &BIT (cur) &&nla_put_flag (msg, cur)) + goto nla_put_failure; + } + +genlmsg_end (msg, hdr); + +return 0 ; + +nla_put_failure: +genlmsg_cancel (msg, hdr); +return -EMSGSIZE ; +} +int net_shaper_nl_cap_get_doit (struct sk_buff *skb,struct genl_info *info) { +struct net_shaper_binding *binding; +const struct net_shaper_ops *ops; +enum net_shaper_scope scope; +unsigned long flags =0 ; +struct sk_buff *msg; +int ret; + +if (GENL_REQ_ATTR_CHECK (info,NET_SHAPER_A_CAPS_SCOPE )) +return -EINVAL ; + + binding =net_shaper_binding_from_ctx (info->ctx); + scope =nla_get_u32 (info->attrs[NET_SHAPER_A_CAPS_SCOPE ]); + ops =net_shaper_ops (binding); + ops->capabilities (binding, scope, &flags); +if (!flags) +return -EOPNOTSUPP ; + + msg =genlmsg_new (NLMSG_DEFAULT_SIZE ,GFP_KERNEL ); +if (!msg) +return -ENOMEM ; + + ret =net_shaper_cap_fill_one (msg, binding, scope, flags, info); +if (ret) + goto free_msg; + + ret =genlmsg_reply (msg, info); +if (ret) + goto free_msg;return 0 ; + +free_msg: +nlmsg_free (msg); +return ret; }int net_shaper_nl_cap_get_dumpit (struct sk_buff *skb,struct netlink_callback *cb) { +const struct genl_info *info =genl_info_dump (cb); +struct net_shaper_binding *binding; +const struct net_shaper_ops *ops; +enum net_shaper_scope scope; +int ret; + + binding =net_shaper_binding_from_ctx (cb->ctx); + ops =net_shaper_ops (binding); +for (scope =0 ; scope <=NET_SHAPER_SCOPE_MAX ; ++scope) { +unsigned long flags =0 ; + + ops->capabilities (binding, scope, &flags); +if (!flags) +continue ; + + ret =net_shaper_cap_fill_one (skb, binding, scope, flags, + info); +if (ret) +return ret; + } +return 0 ; }
Signature · 1 line
2.45.2
Paolo Abeni
maintainerPATCH v6 10/15 net-shapers: implement cap validation in the core
In reply to Paolo AbeniUse the device capabilities to reject invalid attribute values before pushing them to the H/W.
Note that validating the metric explicitly avoids NL_SET_BAD_ATTR() usage, to provide unambiguous error messages to the user.
Validating the nesting requires the knowledge of the new parent for the given shaper; as such is a chicken-egg problem: to validate the leaf nesting we need to know the node scope, to validate the node nesting we need to know the leafs parent scope.
To break the circular dependency, place the leafs nesting validation after the parsing.
--- net/shaper/shaper.c | 102 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 99 insertions(+), 3 deletions(-)
net/shaper/shaper.c +99 -3
@@ -441,6 +441,64 @@ static int net_shaper_parse_handle(const struct nlattr *attr,return 0 ; } +static int net_shaper_validate_caps (struct net_shaper_binding *binding, +struct nlattr **tb, +const struct genl_info *info, +struct net_shaper *shaper) +{ +const struct net_shaper_ops *ops =net_shaper_ops (binding); +struct nlattr *bad =NULL ; +unsigned long caps =0 ; + + ops->capabilities (binding, shaper->handle.scope, &caps); + +if (tb[NET_SHAPER_A_PRIORITY ] && + !(caps &BIT (NET_SHAPER_A_CAPS_SUPPORT_PRIORITY ))) + bad = tb[NET_SHAPER_A_PRIORITY ]; +if (tb[NET_SHAPER_A_WEIGHT ] && + !(caps &BIT (NET_SHAPER_A_CAPS_SUPPORT_WEIGHT ))) + bad = tb[NET_SHAPER_A_WEIGHT ]; +if (tb[NET_SHAPER_A_BW_MIN ] && + !(caps &BIT (NET_SHAPER_A_CAPS_SUPPORT_BW_MIN ))) + bad = tb[NET_SHAPER_A_BW_MIN ]; +if (tb[NET_SHAPER_A_BW_MAX ] && + !(caps &BIT (NET_SHAPER_A_CAPS_SUPPORT_BW_MAX ))) + bad = tb[NET_SHAPER_A_BW_MAX ]; +if (tb[NET_SHAPER_A_BURST ] && + !(caps &BIT (NET_SHAPER_A_CAPS_SUPPORT_BURST ))) + bad = tb[NET_SHAPER_A_BURST ]; + +if (!caps) + bad = tb[NET_SHAPER_A_HANDLE ]; + +if (bad) { +NL_SET_BAD_ATTR (info->extack, bad); +return -EOPNOTSUPP ; + } + +/* The metric is really used only if there is *any* rate-related +* setting, either in current attributes set or in pre-existing +* values. +*/ +if (shaper->burst || shaper->bw_min || shaper->bw_max) { +u32 metric_cap =NET_SHAPER_A_CAPS_SUPPORT_METRIC_BPS + + shaper->metric; + +/* The metric test can fail even when the user did not +* specify the METRIC attribute. Pointing to rate related +* attribute will be confusing, as the attribute itself +* could be indeed supported, with a different metric. +* Be more specific. +*/ +if (!(caps &BIT (metric_cap))) { +NL_SET_ERR_MSG_FMT (info->extack,"Bad metric %d" , + shaper->metric); +return -EOPNOTSUPP ; + } + } +return 0 ; +} + static int net_shaper_parse_info(struct net_shaper_binding *binding, struct nlattr **tb, const struct genl_info *info,
@@ -491,6 +549,28 @@ static int net_shaper_parse_info(struct net_shaper_binding *binding,if (tb[NET_SHAPER_A_WEIGHT ]) shaper->weight =nla_get_u32 (tb[NET_SHAPER_A_WEIGHT ]); + + ret =net_shaper_validate_caps (binding, tb, info, shaper); +if (ret <0 ) +return ret; + +return 0 ; +} + +static int net_shaper_validate_nesting (struct net_shaper_binding *binding, +const struct net_shaper *shaper, +struct netlink_ext_ack *extack) +{ +const struct net_shaper_ops *ops =net_shaper_ops (binding); +unsigned long caps =0 ; + + ops->capabilities (binding, shaper->handle.scope, &caps); +if (!(caps &BIT (NET_SHAPER_A_CAPS_SUPPORT_NESTING ))) { +NL_SET_ERR_MSG_FMT (extack, +"Nesting not supported for scope %d" , + shaper->handle.scope); +return -EOPNOTSUPP ; + }return 0 ; }
@@ -516,9 +596,21 @@ static int net_shaper_parse_info_nest(struct net_shaper_binding *binding,if (ret <0 )return ret; -if (node && shaper->handle.scope !=NET_SHAPER_SCOPE_QUEUE ) { -NL_SET_BAD_ATTR (info->extack, tb[NET_SHAPER_A_HANDLE ]); -return -EINVAL ; +/* When node is specified, the shaper is actually a leaf for a +* group() operation. +*/ +if (node) { +if (shaper->handle.scope !=NET_SHAPER_SCOPE_QUEUE ) { +NL_SET_BAD_ATTR (info->extack, tb[NET_SHAPER_A_HANDLE ]); +return -EINVAL ; + } + +if (node->handle.scope ==NET_SHAPER_SCOPE_NODE ) { + ret =net_shaper_validate_nesting (binding, shaper, + info->extack); +if (ret <0 ) +return ret; + } }if (!exists)
@@ -875,6 +967,10 @@ static int __net_shaper_group(struct net_shaper_binding *binding, node->parent.scope, node->parent.id);return -ENOENT ; } + + ret =net_shaper_validate_nesting (binding, node, extack); +if (ret <0 ) +return ret; }if (update_node) {
Signature · 1 line
2.45.2
Re: [PATCH v6 net-next 10/15] net-shapers: implement cap validation in the core
Quoted 11 lines · On Wed, 4 Sep 2024 15:53:42 +0200 Paolo Abeni wrote:
Use the device capabilities to reject invalid attribute values before pushing them to the H/W.
Note that validating the metric explicitly avoids
NL_SET_BAD_ATTR()usage, to provide unambiguous error messages to the user.Validating the nesting requires the knowledge of the new parent for the given shaper; as such is a chicken-egg problem: to validate the leaf nesting we need to know the node scope, to validate the node nesting we need to know the leafs parent scope.
Quoted 5 lines
To break the circular dependency, place the leafs nesting validation after the parsing.
Suggested-by Jakub Kicinski <[email protected]>Signed-off-by Paolo Abeni <[email protected]>
Paolo Abeni
maintainerPATCH v6 11/15 testing: net-drv: add basic shaper test
In reply to Paolo AbeniLeverage a basic/dummy netdevsim implementation to do functional coverage for NL interface.
---
v5 -> v6:
- additional test-cases for delegation and queue reconf
v4 -> v5:
- updated to new driver API
- more consistent indentation
rfc v1 -> v2:
- added more test-cases WRT nesting and grouping
--- drivers/net/Kconfig | 1 + drivers/net/netdevsim/netdev.c | 39 ++ tools/testing/selftests/drivers/net/Makefile | 1 + tools/testing/selftests/drivers/net/shaper.py | 458 ++++++++++++++++++ .../testing/selftests/net/lib/py/__init__.py | 1 + tools/testing/selftests/net/lib/py/ynl.py | 5 + 6 files changed, 505 insertions(+) create mode 100755 tools/testing/selftests/drivers/net/shaper.py
drivers/net/Kconfig +1 -0
@@ -641,6 +641,7 @@ config NETDEVSIM depends on PTP_1588_CLOCK_MOCK || PTP_1588_CLOCK_MOCK=n select NET_DEVLINK select PAGE_POOL + select NET_SHAPER help This driver is a developer testing tool and software model that can be used to test various control path networking APIs, especially
drivers/net/netdevsim/netdev.c +39 -0
@@ -22,6 +22,7 @@#include <net/netdev_queues.h> #include <net/page_pool/helpers.h> #include <net/netlink.h> +#include <net/net_shaper.h> #include <net/pkt_cls.h> #include <net/rtnetlink.h> #include <net/udp_tunnel.h>
@@ -475,6 +476,43 @@ static int nsim_stop(struct net_device *dev)return 0 ; } +static int nsim_shaper_set (struct net_shaper_binding *binding, +const struct net_shaper *shaper, +struct netlink_ext_ack *extack) +{ +return 0 ; +} + +static int nsim_shaper_del (struct net_shaper_binding *binding, +const struct net_shaper_handle *handle, +struct netlink_ext_ack *extack) +{ +return 0 ; +} + +static int nsim_shaper_group (struct net_shaper_binding *binding, +int leaves_count, +const struct net_shaper *leaves, +const struct net_shaper *root, +struct netlink_ext_ack *extack) +{ +return 0 ; +} + +static void nsim_shaper_cap (struct net_shaper_binding *binding, +enum net_shaper_scope scope, +unsigned long *flags) +{ + *flags =ULONG_MAX ; +} + +static const struct net_shaper_ops nsim_shaper_ops = { + .set = nsim_shaper_set, + .delete = nsim_shaper_del, + .group = nsim_shaper_group, + .capabilities = nsim_shaper_cap, +}; +static const struct net_device_ops nsim_netdev_ops = { .ndo_start_xmit = nsim_start_xmit, .ndo_set_rx_mode = nsim_set_rx_mode,
@@ -496,6 +534,7 @@ static const struct net_device_ops nsim_netdev_ops = { .ndo_bpf = nsim_bpf, .ndo_open = nsim_open, .ndo_stop = nsim_stop, + .net_shaper_ops = &nsim_shaper_ops, };static const struct net_device_ops nsim_vf_netdev_ops = {
tools/testing/selftests/drivers/net/Makefile +1 -0
@@ -9,6 +9,7 @@ TEST_PROGS := \ ping.py \ queues.py \ stats.py \ + shaper.py # end of TEST_PROGS include ../../lib.mk
tools/testing/selftests/drivers/net/shaper.py new file +458 -0
458 lines not on this page · raw message
tools/testing/selftests/net/lib/py/__init__.py +1 -0
@@ -6,3 +6,4 @@ from .netns import NetNS from .nsim import * from .utils import * from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily +from .ynl import NetshaperFamily
tools/testing/selftests/net/lib/py/ynl.py +5 -0
@@ -47,3 +47,8 @@ class NetdevFamily(YnlFamily): def __init__(self): super().__init__((SPEC_PATH / Path('netdev.yaml')).as_posix(), schema='') + +class NetshaperFamily(YnlFamily): + def __init__(self): + super().__init__((SPEC_PATH / Path('net_shaper.yaml')).as_posix(), + schema='')
Signature · 1 line
2.45.2
Paolo Abeni
maintainerPATCH v6 12/15 virtchnl: support queue rate limit and quanta size configuration
In reply to Paolo AbeniFrom: Wenjun Wu <[email protected]>
This patch adds new virtchnl opcodes and structures for rate limit and quanta size configuration, which include:
1. VIRTCHNL_OP_CONFIG_QUEUE_BW, to configure max bandwidth for each VF per queue.
2. VIRTCHNL_OP_CONFIG_QUANTA, to configure quanta size per queue.
3. VIRTCHNL_OP_GET_QOS_CAPS, VF queries current QoS configuration, such as enabled TCs, arbiter type, up2tc and bandwidth of VSI node. The configuration is previously set by DCB and PF, and now is the potential QoS capability of VF. VF can take it as reference to configure queue TC mapping.
--- include/linux/avf/virtchnl.h | 119 +++++++++++++++++++++++++++++++++++ 1 file changed, 119 insertions(+)
include/linux/avf/virtchnl.h +119 -0
@@ -89,6 +89,9 @@ enum virtchnl_rx_hsplit { VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8, }; +enum virtchnl_bw_limit_type { +VIRTCHNL_BW_SHAPER =0 , +};/* END GENERIC DEFINES */ /* Opcodes for VF-PF communication. These are placed in the v_opcode field
@@ -151,6 +154,11 @@ enum virtchnl_ops {VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 =55 ,VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 =56 ,VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 =57 , +/* opcode 57 - 65 are reserved */ +VIRTCHNL_OP_GET_QOS_CAPS =66 , +/* opcode 68 through 111 are reserved */ +VIRTCHNL_OP_CONFIG_QUEUE_BW =112 , +VIRTCHNL_OP_CONFIG_QUANTA =113 , VIRTCHNL_OP_MAX, };
@@ -261,6 +269,7 @@ VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);#define VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC BIT(26)#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28) +#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \ VIRTCHNL_VF_OFFLOAD_VLAN | \
@@ -1416,6 +1425,85 @@ struct virtchnl_fdir_del {VIRTCHNL_CHECK_STRUCT_LEN (12 , virtchnl_fdir_del); +struct virtchnl_shaper_bw { +/* Unit is Kbps */ +u32 committed; +u32 peak; +}; + +VIRTCHNL_CHECK_STRUCT_LEN (8 , virtchnl_shaper_bw); + +/* VIRTCHNL_OP_GET_QOS_CAPS +* VF sends this message to get its QoS Caps, such as +* TC number, Arbiter and Bandwidth. +*/ +struct virtchnl_qos_cap_elem { +u8 tc_num; +u8 tc_prio; +#define VIRTCHNL_ABITER_STRICT 0 +#define VIRTCHNL_ABITER_ETS 2 +u8 arbiter; +#define VIRTCHNL_STRICT_WEIGHT 1 +u8 weight; +enum virtchnl_bw_limit_type type; +union { +struct virtchnl_shaper_bw shaper; +u8 pad2[32 ]; + }; +}; + +VIRTCHNL_CHECK_STRUCT_LEN (40 , virtchnl_qos_cap_elem); + +struct virtchnl_qos_cap_list { +u16 vsi_id; +u16 num_elem; +struct virtchnl_qos_cap_elem cap[]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN (4 , virtchnl_qos_cap_list); +#define virtchnl_qos_cap_list_LEGACY_SIZEOF 44 + +/* VIRTCHNL_OP_CONFIG_QUEUE_BW */ +struct virtchnl_queue_bw { +u16 queue_id; +u8 tc; +u8 pad; +struct virtchnl_shaper_bw shaper; +}; + +VIRTCHNL_CHECK_STRUCT_LEN (12 , virtchnl_queue_bw); + +struct virtchnl_queues_bw_cfg { +u16 vsi_id; +u16 num_queues; +struct virtchnl_queue_bw cfg[]; +}; + +VIRTCHNL_CHECK_STRUCT_LEN (4 , virtchnl_queues_bw_cfg); +#define virtchnl_queues_bw_cfg_LEGACY_SIZEOF 16 + +enum virtchnl_queue_type { +VIRTCHNL_QUEUE_TYPE_TX =0 , +VIRTCHNL_QUEUE_TYPE_RX =1 , +}; + +/* structure to specify a chunk of contiguous queues */ +struct virtchnl_queue_chunk { +/* see enum virtchnl_queue_type */ +s32 type; +u16 start_queue_id; +u16 num_queues; +}; + +VIRTCHNL_CHECK_STRUCT_LEN (8 , virtchnl_queue_chunk); + +struct virtchnl_quanta_cfg { +u16 quanta_size; +struct virtchnl_queue_chunk queue_select; +}; + +VIRTCHNL_CHECK_STRUCT_LEN (12 , virtchnl_quanta_cfg); +#define __vss_byone (p, member, count, old) \ (struct_size(p, member, count) + (old - 1 - struct_size(p, member, 0)))
@@ -1438,6 +1526,8 @@ VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);__vss (virtchnl_vlan_filter_list_v2, __vss_byelem, p, m, c), \__vss (virtchnl_tc_info, __vss_byelem, p, m, c), \__vss (virtchnl_rdma_qvlist_info, __vss_byelem, p, m, c), \ +__vss (virtchnl_qos_cap_list, __vss_byelem, p, m, c), \ +__vss (virtchnl_queues_bw_cfg, __vss_byelem, p, m, c), \__vss (virtchnl_rss_key, __vss_byone, p, m, c), \ __vss(virtchnl_rss_lut, __vss_byone, p, m, c))
@@ -1637,6 +1727,35 @@ virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 : valid_len =sizeof (struct virtchnl_vlan_setting );break ; +case VIRTCHNL_OP_GET_QOS_CAPS : +break ; +case VIRTCHNL_OP_CONFIG_QUEUE_BW : + valid_len = virtchnl_queues_bw_cfg_LEGACY_SIZEOF; +if (msglen >= valid_len) { +struct virtchnl_queues_bw_cfg *q_bw = + (struct virtchnl_queues_bw_cfg *)msg; + + valid_len =virtchnl_struct_size (q_bw, cfg, + q_bw->num_queues); +if (q_bw->num_queues ==0 ) { + err_msg_format = true; +break ; + } + } +break ; +case VIRTCHNL_OP_CONFIG_QUANTA : + valid_len =sizeof (struct virtchnl_quanta_cfg ); +if (msglen >= valid_len) { +struct virtchnl_quanta_cfg *q_quanta = + (struct virtchnl_quanta_cfg *)msg; + +if (q_quanta->quanta_size ==0 || + q_quanta->queue_select.num_queues ==0 ) { + err_msg_format = true; +break ; + } + } +break ;/* These are always errors coming from the VF. */ case VIRTCHNL_OP_EVENT :case VIRTCHNL_OP_UNKNOWN :
Signature · 1 line
2.45.2
Paolo Abeni
maintainerPATCH v6 13/15 ice: Support VF queue rate limit and quanta size configuration
In reply to Paolo AbeniFrom: Wenjun Wu <[email protected]>
Add support to configure VF queue rate limit and quanta size.
For quanta size configuration, the quanta profiles are divided evenly by PF numbers. For each port, the first quanta profile is reserved for default. When VF is asked to set queue quanta size, PF will search for an available profile, change the fields and assigned this profile to the queue.
--- drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_base.c | 2 + drivers/net/ethernet/intel/ice/ice_common.c | 21 ++ .../net/ethernet/intel/ice/ice_hw_autogen.h | 8 + drivers/net/ethernet/intel/ice/ice_txrx.h | 1 + drivers/net/ethernet/intel/ice/ice_type.h | 1 + drivers/net/ethernet/intel/ice/ice_vf_lib.h | 8 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 335 ++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_virtchnl.h | 11 + .../intel/ice/ice_virtchnl_allowlist.c | 6 + 10 files changed, 395 insertions(+)
drivers/net/ethernet/intel/ice/ice.h +2 -0
@@ -659,6 +659,8 @@ struct ice_pf {struct ice_agg_node vf_agg_node[ICE_MAX_VF_AGG_NODES ];struct ice_dplls dplls;struct device *hwmon_dev; + +u8 num_quanta_prof_used; };extern struct workqueue_struct *ice_lag_wq;
drivers/net/ethernet/intel/ice/ice_base.c +2 -0
@@ -349,6 +349,8 @@ ice_setup_tx_ctx(struct ice_tx_ring *ring, struct ice_tlan_ctx *tlan_ctx, u16 pfbreak ; } + tlan_ctx->quanta_prof_idx = ring->quanta_prof_id; + tlan_ctx->tso_ena =ICE_TX_LEGACY ; tlan_ctx->tso_qnum = pf_q;
drivers/net/ethernet/intel/ice/ice_common.c +21 -0
@@ -2436,6 +2436,25 @@ ice_parse_func_caps(struct ice_hw *hw, struct ice_hw_func_caps *func_p,ice_recalc_port_limited_caps (hw, &func_p->common_cap); } +/** +* ice_func_id_to_logical_id - map from function id to logical pf id +* @active_function_bitmap: active function bitmap +* @pf_id: function number of device +* +* Return: logical PF ID. +*/ +static int ice_func_id_to_logical_id (u32 active_function_bitmap,u8 pf_id) +{ +u8 logical_id =0 ; +u8 i; + +for (i =0 ; i < pf_id; i++) +if (active_function_bitmap &BIT (i)) + logical_id++; + +return logical_id; +} +/** * ice_parse_valid_functions_cap - Parse ICE_AQC_CAPS_VALID_FUNCTIONS caps * @hw: pointer to the HW struct
@@ -2453,6 +2472,8 @@ ice_parse_valid_functions_cap(struct ice_hw *hw, struct ice_hw_dev_caps *dev_p, dev_p->num_funcs =hweight32 (number);ice_debug (hw,ICE_DBG_INIT ,"dev caps: num_funcs = %d\n" , dev_p->num_funcs); + + hw->logical_pf_id =ice_func_id_to_logical_id (number, hw->pf_id); }/**
drivers/net/ethernet/intel/ice/ice_hw_autogen.h +8 -0
@@ -6,6 +6,14 @@#ifndef _ICE_HW_AUTOGEN_H_#define _ICE_HW_AUTOGEN_H_ +#define GLCOMM_QUANTA_PROF (_i) (0x002D2D68 + ((_i) * 4)) +#define GLCOMM_QUANTA_PROF_MAX_INDEX 15 +#define GLCOMM_QUANTA_PROF_QUANTA_SIZE_S 0 +#define GLCOMM_QUANTA_PROF_QUANTA_SIZE_M ICE_M(0x3FFF, 0) +#define GLCOMM_QUANTA_PROF_MAX_CMD_S 16 +#define GLCOMM_QUANTA_PROF_MAX_CMD_M ICE_M(0xFF, 16) +#define GLCOMM_QUANTA_PROF_MAX_DESC_S 24 +#define GLCOMM_QUANTA_PROF_MAX_DESC_M ICE_M(0x3F, 24)#define QTX_COMM_DBELL (_DBQM) (0x002C0000 + ((_DBQM) * 4))#define QTX_COMM_HEAD (_DBQM) (0x000E0000 + ((_DBQM) * 4))#define QTX_COMM_HEAD_HEAD_S 0
drivers/net/ethernet/intel/ice/ice_txrx.h +1 -0
@@ -406,6 +406,7 @@ struct ice_tx_ring {#define ICE_TX_FLAGS_RING_VLAN_L2TAG2 BIT(2)u8 flags;u8 dcb_tc;/* Traffic class of ring */ +u16 quanta_prof_id; }____cacheline_internodealigned_in_smp ;static inline bool ice_ring_uses_build_skb (struct ice_rx_ring *ring)
drivers/net/ethernet/intel/ice/ice_type.h +1 -0
@@ -904,6 +904,7 @@ struct ice_hw {u8 revision_id;u8 pf_id;/* device profile info */ +u8 logical_pf_id;u16 max_burst_size;/* driver sets this value */
drivers/net/ethernet/intel/ice/ice_vf_lib.h +8 -0
@@ -59,6 +59,13 @@ struct ice_fdir_prof_info {u64 fdir_active_cnt; }; +struct ice_vf_qs_bw { +u32 committed; +u32 peak; +u16 queue_id; +u8 tc; +}; +/* VF operations */ struct ice_vf_ops {enum ice_disq_rst_src reset_type;
@@ -140,6 +147,7 @@ struct ice_vf {struct devlink_port devlink_port;u16 num_msix;/* num of MSI-X configured on this VF */ +struct ice_vf_qs_bw qs_bw[ICE_MAX_RSS_QS_PER_VF ]; };/* Flags for controlling behavior of ice_reset_vf */
drivers/net/ethernet/intel/ice/ice_virtchnl.c +335 -0
365 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_virtchnl.h +11 -0
@@ -13,6 +13,13 @@/* Restrict number of MAC Addr and VLAN that non-trusted VF can programmed */ #define ICE_MAX_VLAN_PER_VF 8 +#define ICE_DFLT_QUANTA 1024 +#define ICE_MAX_QUANTA_SIZE 4096 +#define ICE_MIN_QUANTA_SIZE 256 + +#define calc_quanta_desc (x) \ + max_t(u16, 12, min_t(u16, 63, (((x) + 66) / 132) * 2 + 4)) +/* MAC filters: 1 is reserved for the VF's default/perm_addr/LAA MAC, 1 for * broadcast, and 16 for additional unicast/multicast filters */
@@ -61,6 +68,10 @@ struct ice_virtchnl_ops {int (*dis_vlan_stripping_v2_msg)(struct ice_vf *vf,u8 *msg);int (*ena_vlan_insertion_v2_msg)(struct ice_vf *vf,u8 *msg);int (*dis_vlan_insertion_v2_msg)(struct ice_vf *vf,u8 *msg); +int (*get_qos_caps)(struct ice_vf *vf); +int (*cfg_q_tc_map)(struct ice_vf *vf,u8 *msg); +int (*cfg_q_bw)(struct ice_vf *vf,u8 *msg); +int (*cfg_q_quanta)(struct ice_vf *vf,u8 *msg); };#ifdef CONFIG_PCI_IOV
drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c +6 -0
@@ -84,6 +84,11 @@ static const u32 fdir_pf_allowlist_opcodes[] = { VIRTCHNL_OP_ADD_FDIR_FILTER, VIRTCHNL_OP_DEL_FDIR_FILTER, }; +static const u32 tc_allowlist_opcodes[] = { +VIRTCHNL_OP_GET_QOS_CAPS ,VIRTCHNL_OP_CONFIG_QUEUE_BW , +VIRTCHNL_OP_CONFIG_QUANTA , +}; +struct allowlist_opcode_info {const u32 *opcodes;size_t size;
@@ -104,6 +109,7 @@ static const struct allowlist_opcode_info allowlist_opcodes[] = {ALLOW_ITEM (VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF , adv_rss_pf_allowlist_opcodes),ALLOW_ITEM (VIRTCHNL_VF_OFFLOAD_FDIR_PF , fdir_pf_allowlist_opcodes),ALLOW_ITEM (VIRTCHNL_VF_OFFLOAD_VLAN_V2 , vlan_v2_allowlist_opcodes), + ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_QOS, tc_allowlist_opcodes), };/**
Signature · 1 line
2.45.2
Paolo Abeni
maintainerPATCH v6 14/15 iavf: Add net_shaper_ops support
In reply to Paolo AbeniFrom: Sudheer Mogilappagari <[email protected]>
Implement net_shaper_ops support for IAVF. This enables configuration of rate limiting on per queue basis. Customer intends to enforce bandwidth limit on Tx traffic steered to the queue by configuring rate limits on the queue.
To set rate limiting for a queue, update shaper object of given queues in driver and send VIRTCHNL_OP_CONFIG_QUEUE_BW to PF to update HW configuration.
Deleting shaper configured for queue is nothing but configuring shaper with bw_max 0. The PF restores the default rate limiting config when bw_max is zero.
---
v5 -> v6:
- adapted to new API
- dropped scope-related checks (moved into the core)
v4 -> v5:
- fix kdoc
--- drivers/net/ethernet/intel/Kconfig | 1 + drivers/net/ethernet/intel/iavf/iavf.h | 3 + drivers/net/ethernet/intel/iavf/iavf_main.c | 110 ++++++++++++++++++ drivers/net/ethernet/intel/iavf/iavf_txrx.h | 2 + .../net/ethernet/intel/iavf/iavf_virtchnl.c | 65 +++++++++++ 5 files changed, 181 insertions(+)
drivers/net/ethernet/intel/Kconfig +1 -0
@@ -258,6 +258,7 @@ config I40E_DCB config IAVF tristate select LIBIE + select NET_SHAPER config I40EVF tristate "Intel(R) Ethernet Adaptive Virtual Function support"
drivers/net/ethernet/intel/iavf/iavf.h +3 -0
@@ -34,6 +34,7 @@#include <net/tc_act/tc_gact.h> #include <net/tc_act/tc_mirred.h> #include <net/tc_act/tc_skbedit.h> +#include <net/net_shaper.h> #include "iavf_type.h" #include <linux/avf/virtchnl.h>
@@ -336,6 +337,7 @@ struct iavf_adapter {#define IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_INSERTION BIT_ULL(36)#define IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION BIT_ULL(37)#define IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION BIT_ULL(38) +#define IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW BIT_ULL(39)/* flags for processing extended capability messages during * __IAVF_INIT_EXTENDED_CAPS. Each capability exchange requires
@@ -581,6 +583,7 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,int iavf_config_rss (struct iavf_adapter *adapter);int iavf_lan_add_device (struct iavf_adapter *adapter);int iavf_lan_del_device (struct iavf_adapter *adapter); +void iavf_cfg_queues_bw (struct iavf_adapter *adapter);void iavf_enable_channels (struct iavf_adapter *adapter);void iavf_disable_channels (struct iavf_adapter *adapter);void iavf_add_cloud_filter (struct iavf_adapter *adapter);
drivers/net/ethernet/intel/iavf/iavf_main.c +110 -0
140 lines not on this page · raw message
drivers/net/ethernet/intel/iavf/iavf_txrx.h +2 -0
@@ -296,6 +296,8 @@ struct iavf_ring {*/ u32 rx_buf_len; +struct net_shaper q_shaper; +bool q_shaper_update; }____cacheline_internodealigned_in_smp ;#define IAVF_ITR_ADAPTIVE_MIN_INC 0x0002
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +65 -0
83 lines not on this page · raw message
Signature · 1 line
2.45.2
Jakub Kicinski
maintainerRe: [PATCH v6 net-next 14/15] iavf: Add net_shaper_ops support
Quoted 4 lines · On Wed, 4 Sep 2024 15:53:46 +0200 Paolo Abeni wrote:
drivers/net/ethernet/intel/iavf/iavf_main.c +4
-0+static int iavf_verify_handle (struct net_shaper_binding *binding, +const struct net_shaper_handle *handle, +struct netlink_ext_ack *extack) +{
drivers/net/ethernet/intel/iavf/iavf_main.c static netdev_features_t iavf_fix_features(struct net_device *netdev, · line 4940
Quoted 10 lines
drivers/net/ethernet/intel/iavf/iavf_main.c +10
-0+struct iavf_adapter *adapter =netdev_priv (binding->netdev); +int qid = handle->id; + +if (qid >= adapter->num_active_queues) { +NL_SET_ERR_MSG_FMT (extack,"Invalid shaper handle, queued id %d max %d" , + qid, adapter->num_active_queues); +return -EINVAL ; + } +return 0 ; +}
This can go once we have the real_num validation in the core? Otherwise LGTM!
Paolo Abeni
maintainerPATCH v6 15/15 iavf: add support to exchange qos capabilities
In reply to Paolo AbeniFrom: Sudheer Mogilappagari <[email protected]>
During driver initialization VF determines QOS capability is allowed by PF and receives QOS parameters. After which quanta size for queues is configured which is not configurable and is set to 1KB currently.
---
v5 -> v6:
- error out on bad rate
--- drivers/net/ethernet/intel/iavf/iavf.h | 10 ++ drivers/net/ethernet/intel/iavf/iavf_main.c | 51 +++++++++- .../net/ethernet/intel/iavf/iavf_virtchnl.c | 92 ++++++++++++++++++- 3 files changed, 150 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/iavf/iavf.h +10 -0
@@ -251,6 +251,9 @@ struct iavf_cloud_filter {#define IAVF_RESET_WAIT_DETECTED_COUNT 500#define IAVF_RESET_WAIT_COMPLETE_COUNT 2000 +#define IAVF_MAX_QOS_TC_NUM 8 +#define IAVF_DEFAULT_QUANTA_SIZE 1024 +/* board specific private data structure */ struct iavf_adapter {struct workqueue_struct *wq;
@@ -338,6 +341,8 @@ struct iavf_adapter {#define IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION BIT_ULL(37)#define IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION BIT_ULL(38)#define IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW BIT_ULL(39) +#define IAVF_FLAG_AQ_CFG_QUEUES_QUANTA_SIZE BIT_ULL(40) +#define IAVF_FLAG_AQ_GET_QOS_CAPS BIT_ULL(41)/* flags for processing extended capability messages during * __IAVF_INIT_EXTENDED_CAPS. Each capability exchange requires
@@ -410,6 +415,8 @@ struct iavf_adapter { VIRTCHNL_VF_OFFLOAD_FDIR_PF)#define ADV_RSS_SUPPORT (_a) ((_a)->vf_res->vf_cap_flags & \ VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF) +#define QOS_ALLOWED (_a) ((_a)->vf_res->vf_cap_flags & \ + VIRTCHNL_VF_OFFLOAD_QOS)struct virtchnl_vf_resource *vf_res;/* incl. all VSIs */ struct virtchnl_vsi_resource *vsi_res;/* our LAN VSI */ struct virtchnl_version_info pf_version;
@@ -418,6 +425,7 @@ struct iavf_adapter {struct virtchnl_vlan_caps vlan_v2_caps;u16 msg_enable;struct iavf_eth_stats current_stats; +struct virtchnl_qos_cap_list *qos_caps;struct iavf_vsi vsi;u32 aq_wait_count;/* RSS stuff */
@@ -584,6 +592,8 @@ int iavf_config_rss(struct iavf_adapter *adapter);int iavf_lan_add_device (struct iavf_adapter *adapter);int iavf_lan_del_device (struct iavf_adapter *adapter);void iavf_cfg_queues_bw (struct iavf_adapter *adapter); +void iavf_cfg_queues_quanta_size (struct iavf_adapter *adapter); +void iavf_get_qos_caps (struct iavf_adapter *adapter);void iavf_enable_channels (struct iavf_adapter *adapter);void iavf_disable_channels (struct iavf_adapter *adapter);void iavf_add_cloud_filter (struct iavf_adapter *adapter);
drivers/net/ethernet/intel/iavf/iavf_main.c +49 -2
93 lines not on this page · raw message
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +91 -1
116 lines not on this page · raw message
Signature · 1 line
2.45.2