Skip to contentSkip to conversation

net: introduce TX shaping H/W offload API

RFC PATCH Superseded Paolo Abeni 2 participants 19 messages Jul 24, 2024 – Aug 2, 2024

Subsystems (8)

v2 of 9 Superseded Latest is v9 →

lore
Share & tools
lore

Paolo Abeni

author maintainer
#0 lore raw

v2 cover net: introduce TX shaping H/W offload API

We have a plurality of shaping-related drivers API, but none flexible enough to meet existing demand from vendors[1].

This series introduces new device APIs to configure in a flexible way TX shaping H/W offload. The new functionality is exposed via a newly defined generic netlink interface and includes introspection capabilities. Some self-tests are included, on top of a dummy netdevsim implementation, and a basic implementation for the iavf driver.

Some usage examples:

* Configure shaping on a given queue:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do set --json '{"ifindex":'$IFINDEX',
			"shaper": {"handle":
				{"scope": "queue", "id":'$QUEUEID' },
			"bw-max": 2000000 }}'

* Container B/W sharing

The orchestration infrastructure wants to group the container-related queues under a RR scheduling and limit the aggregate bandwidth:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do group --json '{"ifindex":'$IFINDEX', 
			"inputs": [ 
			  {"handle": {"scope": "queue", "id":'$QID1' },
			   "weight": '$W1'}, 
			  {"handle": {"scope": "queue", "id":'$QID2' },
			   "weight": '$W2'}], 
			  {"handle": {"scope": "queue", "id":'$QID3' },
			   "weight": '$W3'}], 
			"output": { "handle": {"scope":"netdev"},
			"output": { "handle": {"scope":"netdev"},
			   "bw-max": 10000000}}'

{'handle': {'id': 0, 'scope': 'netdev'}}

* Delegation

A container wants to set a B/W limit on 2 of its own queues:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do group --json '{"ifindex":'$IFINDEX', 
			"inputs": [ 
			  {"handle": {"scope": "queue", "id":'$QID1' },
			   "weight": '$W1'}, 
			  {"handle": {"scope": "queue", "id":'$QID2' },
			   "weight": '$W2'}], 
			"output": { "handle": {"scope":"detached"},
			   "bw-max": 5000000}}'

{'handle': {'id': 0, 'scope': 'detached'}}

* Cleanup:

Deleting a single queue shaper:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do delete --json \
	'{"ifindex":'$IFINDEX', 
	  "handle": {"scope": "queue", "id":'$QID1' }}'

Deleting the last shaper under a group deletes the group, too:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do delete --json \
	'{"ifindex":'$IFINDEX', 
	  "handle": {"scope": "queue", "id":'$QID2' }}'
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
        --do get --json '{"ifindex":'$IF', 
			  "handle": { "scope": "detached", "id": 0}}'
Netlink error: Invalid argument
nl_len = 80 (64) nl_flags = 0x300 nl_type = 2
	error: -22
	extack: {'msg': "Can't find shaper for handle 10000000"}

Changes from RFC v1:
- set() and delete() ops operate on a single shaper
- added group() op to allow grouping and nesting
- split the NL implementation into multiple patches to help reviewing

RFC v1: https://lore.kernel.org/netdev/[email protected]/

[1] https://lore.kernel.org/netdev/[email protected]/

Paolo Abeni (7):
netlink: spec: add shaper YAML spec
net-shapers: implement NL get operation
net-shapers: implement NL set and delete operations
net-shapers: implement NL group operation
netlink: spec: add shaper introspection support
net: shaper: implement introspection support
testing: net-drv: add basic shaper test

Sudheer Mogilappagari (2):
iavf: Add net_shaper_ops support
iavf: add support to exchange qos capabilities

Wenjun Wu (2):
virtchnl: support queue rate limit and quanta size configuration
ice: Support VF queue rate limit and quanta size configuration

 Documentation/netlink/specs/shaper.yaml       | 337 ++++++
 Documentation/networking/kapi.rst             |   3 +
 MAINTAINERS                                   |   1 +
 drivers/net/Kconfig                           |   1 +
 drivers/net/ethernet/intel/Kconfig            |   1 +
 drivers/net/ethernet/intel/iavf/iavf.h        |  13 +
 drivers/net/ethernet/intel/iavf/iavf_main.c   | 215 +++-
 drivers/net/ethernet/intel/iavf/iavf_txrx.h   |   2 +
 .../net/ethernet/intel/iavf/iavf_virtchnl.c   | 157 ++-
 drivers/net/ethernet/intel/ice/ice.h          |   2 +
 drivers/net/ethernet/intel/ice/ice_base.c     |   2 +
 drivers/net/ethernet/intel/ice/ice_common.c   |  21 +
 .../net/ethernet/intel/ice/ice_hw_autogen.h   |   8 +
 drivers/net/ethernet/intel/ice/ice_txrx.h     |   1 +
 drivers/net/ethernet/intel/ice/ice_type.h     |   1 +
 drivers/net/ethernet/intel/ice/ice_vf_lib.h   |   8 +
 drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++
 drivers/net/ethernet/intel/ice/ice_virtchnl.h |  11 +
 .../intel/ice/ice_virtchnl_allowlist.c        |   6 +
 drivers/net/netdevsim/netdev.c                |  37 +
 include/linux/avf/virtchnl.h                  | 119 +++
 include/linux/netdevice.h                     |  17 +
 include/net/net_shaper.h                      | 169 +++
 include/uapi/linux/net_shaper.h               |  91 ++
 net/Kconfig                                   |   3 +
 net/Makefile                                  |   1 +
 net/core/dev.c                                |   2 +
 net/core/dev.h                                |   6 +
 net/shaper/Makefile                           |   9 +
 net/shaper/shaper.c                           | 962 ++++++++++++++++++
 net/shaper/shaper_nl_gen.c                    | 142 +++
 net/shaper/shaper_nl_gen.h                    |  30 +
 tools/testing/selftests/drivers/net/Makefile  |   1 +
 tools/testing/selftests/drivers/net/shaper.py | 267 +++++
 .../testing/selftests/net/lib/py/__init__.py  |   1 +
 tools/testing/selftests/net/lib/py/ynl.py     |   5 +
 36 files changed, 2983 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/netlink/specs/shaper.yaml
 create mode 100644 include/net/net_shaper.h
 create mode 100644 include/uapi/linux/net_shaper.h
 create mode 100644 net/shaper/Makefile
 create mode 100644 net/shaper/shaper.c
 create mode 100644 net/shaper/shaper_nl_gen.c
 create mode 100644 net/shaper/shaper_nl_gen.h
 create mode 100755 tools/testing/selftests/drivers/net/shaper.py
Signature · 1 line
2.45.2
author maintainer
#1 lore raw

RFC PATCH v2 1/11 netlink: spec: add shaper YAML spec

Define the user-space visible interface to query, configure and delete network shapers via yaml definition.

Add dummy implementations for the relevant NL callbacks.

set() and delete() operations touch a single shaper creating/updating or deleting it.
The group() operation creates a shaper's group, nesting multiple input shapers under the specified output shaper.

---
RFC v1 -> RFC v2:
- u64 -> uint
- net_shapers -> net-shapers
- documented all the attributes
- dropped [ admin-perm ] for get() op
- group op
- set/delete touch a single shaper

---
 Documentation/netlink/specs/shaper.yaml | 262 ++++++++++++++++++++++++
 MAINTAINERS                             |   1 +
 include/uapi/linux/net_shaper.h         |  74 +++++++
 net/Kconfig                             |   3 +
 net/Makefile                            |   1 +
 net/shaper/Makefile                     |   9 +
 net/shaper/shaper.c                     |  34 +++
 net/shaper/shaper_nl_gen.c              | 117 +++++++++++
 net/shaper/shaper_nl_gen.h              |  27 +++
 9 files changed, 528 insertions(+)
 create mode 100644 Documentation/netlink/specs/shaper.yaml
 create mode 100644 include/uapi/linux/net_shaper.h
 create mode 100644 net/shaper/Makefile
 create mode 100644 net/shaper/shaper.c
 create mode 100644 net/shaper/shaper_nl_gen.c
 create mode 100644 net/shaper/shaper_nl_gen.h
Documentation/netlink/specs/shaper.yaml new file +262 -0
new file mode 100644
index 000000000000..7327f5596fdb
@@ -0,0 +1,262 @@
+# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
+
+name: net-shaper
+
+doc: Network device HW Rate Limiting offload
+
+definitions:
+  -
+    type: enum
+    name: scope
+    doc: the different scopes where a shaper can be attached
+    render-max: true
+    entries:
+      - name: unspec
+        doc: The scope is not specified
+      -
+        name: port
+        doc: The root for the whole H/W
+      -
+        name: netdev
+        doc: The main shaper for the given network device.
+      -
+        name: queue
+        doc: The shaper is attached to the given device queue.
+      -
+        name: detached
+        doc: |
+             The shaper is not attached to any user-visible network
+             device component and allows nesting and grouping of
+             queues or others detached shapers.
+  -
+    type: enum
+    name: metric
+    doc: different metric each shaper can support
+    entries:
+      -
+        name: bps
+        doc: Shaper operates on a bits per second basis
+      -
+        name: pps
+        doc: Shaper operates on a packets per second basis
+
+attribute-sets:
+  -
+    name: net-shaper
+    attributes:
+      -
+        name: ifindex
+        type: u32
+        doc: Interface index owing the specified shaper[s]
+      -
+        name: handle
+        type: nest
+        nested-attributes: handle
+        doc: Unique identifier for the given shaper
+      -
+        name: metric
+        type: u32
+        enum: metric
+        doc: Metric used by the given shaper for bw-min, bw-max and burst
+      -
+        name: bw-min
+        type: uint
+        doc: Minimum guaranteed B/W for the given shaper
+      -
+        name: bw-max
+        type: uint
+        doc: Shaping B/W for the given shaper or 0 when unlimited
+      -
+        name: burst
+        type: uint
+        doc: Maximum burst-size for bw-min and bw-max
+      -
+        name: priority
+        type: u32
+        doc: Scheduling priority for the given shaper
+      -
+        name: weight
+        type: u32
+        doc: |
+          Weighted round robin weight for given shaper.
+          The scheduling is applied to all the sibling
+          shapers with the same priority
+      -
+        name: scope
+        type: u32
+        enum: scope
+        doc: The given handle scope
+      -
+        name: id
+        type: u32
+        doc: |
+          The given handle id. The id semantic depends on the actual
+          scope, e.g. for 'queue' scope it's the queue id, for
+          'detached' scope it's the shaper group identifier.
+      -
+        name: parent
+        type: nest
+        nested-attributes: handle
+        doc: |
+          Identifier for the parent of the affected shaper,
+          The parent handle value is implied by the shaper handle itself,
+          except for the output shaper in the 'group' operation.
+      -
+        name: inputs
+        type: nest
+        multi-attr: true
+        nested-attributes: ns-info
+        doc: |
+           Describes a set of inputs shapers for a @group operation
+      -
+        name: output
+        type: nest
+        nested-attributes: ns-output-info
+        doc: |
+           Describes the output shaper for a @group operation
+           Differently from @inputs and @shaper allow specifying
+           the shaper parent handle, too.
+
+      -
+        name: shaper
+        type: nest
+        nested-attributes: ns-info
+        doc: |
+           Describes a single shaper for a @set operation
+  -
+    name: handle
+    subset-of: net-shaper
+    attributes:
+      -
+        name: scope
+      -
+        name: id
+  -
+    name: ns-info
+    subset-of: net-shaper
+    attributes:
+      -
+        name: handle
+      -
+        name: metric
+      -
+        name: bw-min
+      -
+        name: bw-max
+      -
+        name: burst
+      -
+        name: priority
+      -
+        name: weight
+  -
+    name: ns-output-info
+    subset-of: net-shaper
+    attributes:
+      -
+        name: parent
+      -
+        name: handle
+      -
+        name: metric
+      -
+        name: bw-min
+      -
+        name: bw-max
+      -
+        name: burst
+      -
+        name: priority
+      -
+        name: weight
+
+operations:
+  list:
+    -
+      name: get
+      doc: |
+        Get / Dump information about a/all the shaper for a given device
+      attribute-set: net-shaper
+
+      do:
+        request:
+          attributes:
+            - ifindex
+            - handle
+        reply:
+          attributes: &ns-attrs
+            - parent
+            - handle
+            - metric
+            - bw-min
+            - bw-max
+            - burst
+            - priority
+            - weight
+
+      dump:
+        request:
+          attributes:
+            - ifindex
+        reply:
+          attributes: *ns-attrs
+    -
+      name: set
+      doc: |
+        Create or configures the specified shaper.
+        On failures the extack is set accordingly.
+        Can't create @detached scope shaper, use
+        the @group operation instead.
+      attribute-set: net-shaper
+      flags: [ admin-perm ]
+
+      do:
+        request:
+          attributes:
+            - ifindex
+            - shaper
+
+    -
+      name: delete
+      doc: |
+        Clear (remove) the specified shaper. If after the removal
+        the parent shaper has no more children and the parent
+        shaper scope is @detached, even the parent is deleted,
+        recursively.
+        On failures the extack is set accordingly.
+      attribute-set: net-shaper
+      flags: [ admin-perm ]
+
+      do:
+        request:
+          attributes:
+            - ifindex
+            - handle
+
+    -
+      name: group
+      doc: |
+        Group the specified input shapers under the specified
+        output shaper, eventually creating the latter, if needed.
+        Input shapers scope must be either @queue or @detached.
+        Output shaper scope must be either @detached or @netdev.
+        When using an output @detached scope shaper, if the
+        @handle @id is not specified, a new shaper of such scope
+        is created and, otherwise the specified output shaper
+        must be already existing.
+        The operation is atomic, on failures the extack is set
+        accordingly and no change is applied to the device
+        shaping configuration, otherwise the output shaper
+        handle is provided as reply.
+      attribute-set: net-shaper
+      flags: [ admin-perm ]
+
+      do:
+        request:
+          attributes:
+            - ifindex
+            - inputs
+            - output
+        reply:
+          attributes:
+            - handle
MAINTAINERS +1 -0
index 4a096207f8b4..1aaa27bb67eb 100644
@@ -15816,6 +15816,7 @@ F:	include/linux/netdevice.h
 F:	include/uapi/linux/cn_proc.h
 F:	include/uapi/linux/if_*
 F:	include/uapi/linux/netdevice.h
+F:	include/uapu/linux/net_shaper.h
 X:	drivers/net/wireless/
 
 NETWORKING DRIVERS (WIRELESS)
include/uapi/linux/net_shaper.h new file +74 -0

74 lines not on this page · raw message

net/Kconfig +3 -0
index d27d0deac0bf..31fccfed04f7 100644
@@ -66,6 +66,9 @@ config SKB_DECRYPTED
 config SKB_EXTENSIONS
 	bool
 
+config NET_SHAPER
+	bool
+
 menu "Networking options"
 
 source "net/packet/Kconfig"
net/Makefile +1 -0
index 65bb8c72a35e..60ed5190eda8 100644
@@ -79,3 +79,4 @@ obj-$(CONFIG_XDP_SOCKETS)	+= xdp/
 obj-$(CONFIG_MPTCP)		+= mptcp/
 obj-$(CONFIG_MCTP)		+= mctp/
 obj-$(CONFIG_NET_HANDSHAKE)	+= handshake/
+obj-$(CONFIG_NET_SHAPER)	+= shaper/
net/shaper/Makefile new file +9 -0
new file mode 100644
index 000000000000..13375884d60e
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Makefile for the Generic HANDSHAKE service
+#
+# Copyright (c) 2024, Red Hat, Inc.
+#
+
+obj-y += shaper.o shaper_nl_gen.o
+
net/shaper/shaper.c new file +34 -0
new file mode 100644
index 000000000000..49de88c68e2f
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+#include <linux/kernel.h>
+#include <linux/skbuff.h>
+
+#include "shaper_nl_gen.h"
+
+int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
+{
+	return -EOPNOTSUPP;
+}
+
+int net_shaper_nl_get_dumpit(struct sk_buff *skb,
+			     struct netlink_callback *cb)
+{
+	return -EOPNOTSUPP;
+}
+
+int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
+{
+	return -EOPNOTSUPP;
+}
+
+int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
+{
+	return -EOPNOTSUPP;
+}
+
+static int __init shaper_init(void)
+{
+	return genl_register_family(&net_shaper_nl_family);
+}
+
+subsys_initcall(shaper_init);
net/shaper/shaper_nl_gen.c new file +117 -0

117 lines not on this page · raw message

net/shaper/shaper_nl_gen.h new file +27 -0
new file mode 100644
index 000000000000..00cee4efd21e
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */
+/* Do not edit directly, auto-generated from: */
+/*	Documentation/netlink/specs/shaper.yaml */
+/* YNL-GEN kernel header */
+
+#ifndef _LINUX_NET_SHAPER_GEN_H
+#define _LINUX_NET_SHAPER_GEN_H
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include <uapi/linux/net_shaper.h>
+
+/* Common nested types */
+extern const struct nla_policy net_shaper_handle_nl_policy[NET_SHAPER_A_ID + 1];
+extern const struct nla_policy net_shaper_ns_info_nl_policy[NET_SHAPER_A_WEIGHT + 1];
+extern const struct nla_policy net_shaper_ns_output_info_nl_policy[NET_SHAPER_A_PARENT + 1];
+
+int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
+int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info);
+
+extern struct genl_family net_shaper_nl_family;
+
+#endif /* _LINUX_NET_SHAPER_GEN_H */
Signature · 1 line
2.45.2
author maintainer
#3 lore raw

RFC PATCH v2 2/11 net-shapers: implement NL get operation

In reply to Paolo Abeni

Introduce the basic infrastructure to implement the net-shaper core functionality. Each network devices carries a net-shaper cache, the NL get() operation fetches the data from such cache.

The cache is initially empty, will be fill by the set()/group() operation implemented later and is destroyed at device cleanup time.

---
 Documentation/networking/kapi.rst |   3 +
 include/linux/netdevice.h         |  17 ++
 include/net/net_shaper.h          | 158 +++++++++++++++++++
 net/core/dev.c                    |   2 +
 net/core/dev.h                    |   6 +
 net/shaper/shaper.c               | 251 +++++++++++++++++++++++++++++-
 6 files changed, 435 insertions(+), 2 deletions(-)
 create mode 100644 include/net/net_shaper.h
Documentation/networking/kapi.rst +3 -0
index ea55f462cefa..98682b9a13ee 100644
@@ -104,6 +104,9 @@ Driver Support
 .. kernel-doc:: include/linux/netdevice.h
    :internal:
 
+.. kernel-doc:: include/net/net_shaper.h
+   :internal:
+
 PHY Support
 -----------
 
include/linux/netdevice.h +17 -0
index 607009150b5f..d3d952be711c 100644
@@ -81,6 +81,8 @@ struct xdp_frame;
 struct xdp_metadata_ops;
 struct xdp_md;
 struct ethtool_netdev_state;
+struct net_shaper_ops;
+struct net_shaper_data;
 
 typedef u32 xdp_features_t;
 
@@ -1598,6 +1600,14 @@ struct net_device_ops {
 	int			(*ndo_hwtstamp_set)(struct net_device *dev,
 						    struct kernel_hwtstamp_config *kernel_config,
 						    struct netlink_ext_ack *extack);
+
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+	/**
+	 * @net_shaper_ops: Device shaping offload operations
+	 * see include/net/net_shapers.h
+	 */
+	const struct net_shaper_ops *net_shaper_ops;
+#endif
 };
 
 /**
@@ -2408,6 +2418,13 @@ struct net_device {
 	/** @irq_moder: dim parameters used if IS_ENABLED(CONFIG_DIMLIB). */
 	struct dim_irq_moder	*irq_moder;
 
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+	/**
+	 * @net_shaper_data: data tracking the current shaper status
+	 *  see include/net/net_shapers.h
+	 */
+	struct net_shaper_data *net_shaper_data;
+#endif
 	u8			priv[] ____cacheline_aligned
 				       __counted_by(priv_len);
 } ____cacheline_aligned;
include/net/net_shaper.h new file +158 -0

158 lines not on this page · raw message

net/core/dev.c +2 -0
index 6ea1d20676fb..3dc1dd428eda 100644
@@ -11169,6 +11169,8 @@ void free_netdev(struct net_device *dev)
 	/* Flush device addresses */
 	dev_addr_flush(dev);
 
+	dev_shaper_flush(dev);
+
 	list_for_each_entry_safe(p, n, &dev->napi_list, dev_list)
 		netif_napi_del(p);
 
net/core/dev.h +6 -0
index 5654325c5b71..e376fc1c867b 100644
@@ -35,6 +35,12 @@ void dev_addr_flush(struct net_device *dev);
 int dev_addr_init(struct net_device *dev);
 void dev_addr_check(struct net_device *dev);
 
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+void dev_shaper_flush(struct net_device *dev);
+#else
+static inline void dev_shaper_flush(struct net_device *dev) {}
+#endif
+
 /* sysctls not referred to from outside net/core/ */
 extern int		netdev_unregister_timeout_secs;
 extern int		weight_p;
net/shaper/shaper.c +249 -2

274 lines not on this page · raw message

Signature · 1 line
2.45.2
author maintainer
#4 lore raw

RFC PATCH v2 3/11 net-shapers: implement NL set and delete operations

In reply to Paolo Abeni

Both NL operations directly map on the homonymous device shaper callbacks and update accordingly the shapers cache. Implement the cache modification helpers to additionally deal with DETACHED scope shaper. That will be needed by the group() operation implemented in the next patch.

---
 net/shaper/shaper.c | 323 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 321 insertions(+), 2 deletions(-)
net/shaper/shaper.c +321 -2
index 93dd491ac7c2..7802c9ba6d9c 100644
@@ -19,6 +19,35 @@ struct net_shaper_nl_ctx {
 	u32 start_handle;
 };
 
+static u32 default_parent(u32 handle)
+{
+	enum net_shaper_scope parent, scope = net_shaper_handle_scope(handle);
+
+	switch (scope) {
+	case NET_SHAPER_SCOPE_PORT:
+	case NET_SHAPER_SCOPE_UNSPEC:
+		parent = NET_SHAPER_SCOPE_UNSPEC;
+		break;
+
+	case NET_SHAPER_SCOPE_QUEUE:
+	case NET_SHAPER_SCOPE_DETACHED:
+		parent = NET_SHAPER_SCOPE_NETDEV;
+		break;
+
+	case NET_SHAPER_SCOPE_NETDEV:
+	case NET_SHAPER_SCOPE_VF:
+		parent = NET_SHAPER_SCOPE_PORT;
+		break;
+	}
+
+	return net_shaper_make_handle(parent, 0);
+}
+
+static bool is_detached(u32 handle)
+{
+	return net_shaper_handle_scope(handle) == NET_SHAPER_SCOPE_DETACHED;
+}
+
 static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
 		       const struct genl_info *info)
 {
@@ -117,6 +146,115 @@ static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle)
 	return xa ? xa_load(xa, handle) : NULL;
 }
 
+/* allocate on demand the per device shaper's cache */
+static struct xarray *__sc_init(struct net_device *dev,
+				struct netlink_ext_ack *extack)
+{
+	if (!dev->net_shaper_data) {
+		dev->net_shaper_data = kmalloc(sizeof(*dev->net_shaper_data),
+					       GFP_KERNEL);
+		if (!dev->net_shaper_data) {
+			NL_SET_ERR_MSG(extack, "Can't allocate memory for shaper data");
+			return NULL;
+		}
+
+		xa_init(&dev->net_shaper_data->shapers);
+		idr_init(&dev->net_shaper_data->detached_ids);
+	}
+	return &dev->net_shaper_data->shapers;
+}
+
+/* prepare the cache to actually insert the given shaper, doing
+ * in advance the needed allocations
+ */
+static int sc_prepare_insert(struct net_device *dev, u32 *handle,
+			     struct netlink_ext_ack *extack)
+{
+	enum net_shaper_scope scope = net_shaper_handle_scope(*handle);
+	struct xarray *xa = __sc_init(dev, extack);
+	struct net_shaper_info *prev, *cur;
+	bool id_allocated = false;
+	int ret, id;
+
+	if (!xa)
+		return -ENOMEM;
+
+	cur = xa_load(xa, *handle);
+	if (cur)
+		return 0;
+
+	/* allocated a new id, if needed */
+	if (scope == NET_SHAPER_SCOPE_DETACHED &&
+	    net_shaper_handle_id(*handle) == NET_SHAPER_ID_UNSPEC) {
+		xa_lock(xa);
+		id = idr_alloc(&dev->net_shaper_data->detached_ids, NULL,
+			       0, NET_SHAPER_ID_UNSPEC, GFP_ATOMIC);
+		xa_unlock(xa);
+
+		if (id < 0) {
+			NL_SET_ERR_MSG(extack, "Can't allocate new id for detached shaper");
+			return id;
+		}
+
+		*handle = net_shaper_make_handle(scope, id);
+		id_allocated = true;
+	}
+
+	cur = kmalloc(sizeof(*cur), GFP_KERNEL | __GFP_ZERO);
+	if (!cur) {
+		NL_SET_ERR_MSG(extack, "Can't allocate memory for cached shaper");
+		ret = -ENOMEM;
+		goto free_id;
+	}
+
+	/* mark 'tentative' shaper inside the cache */
+	xa_lock(xa);
+	prev = __xa_store(xa, *handle, cur, GFP_KERNEL);
+	__xa_set_mark(xa, *handle, XA_MARK_0);
+	xa_unlock(xa);
+	if (xa_err(prev)) {
+		NL_SET_ERR_MSG(extack, "Can't insert shaper into cache");
+		kfree(cur);
+		ret = xa_err(prev);
+		goto free_id;
+	}
+	return 0;
+
+free_id:
+	if (id_allocated) {
+		xa_lock(xa);
+		idr_remove(&dev->net_shaper_data->detached_ids,
+			   net_shaper_handle_id(*handle));
+		xa_unlock(xa);
+	}
+	return ret;
+}
+
+/* commit the tentative insert with the actual values.
+ * Must be called only after a successful sc_prepare_insert()
+ */
+static void sc_commit(struct net_device *dev, int nr_shapers,
+		      const struct net_shaper_info *shapers)
+{
+	struct xarray *xa = __sc_container(dev);
+	struct net_shaper_info *cur;
+	int i;
+
+	xa_lock(xa);
+	for (i = 0; i < nr_shapers; ++i) {
+		cur = xa_load(xa, shapers[i].handle);
+		if (WARN_ON_ONCE(!cur))
+			continue;
+
+		/* successful update: drop the tentative mark
+		 * and update the cache
+		 */
+		__xa_clear_mark(xa, shapers[i].handle, XA_MARK_0);
+		*cur = shapers[i];
+	}
+	xa_unlock(xa);
+}
+
 static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
 			u32 *handle)
 {
@@ -154,6 +292,68 @@ static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
 	return 0;
 }
 
+static int __parse_shaper(struct net_device *dev, struct nlattr **tb,
+			  const struct genl_info *info,
+			  struct net_shaper_info *shaper)
+{
+	struct net_shaper_info *old;
+	int ret;
+
+	/* the shaper handle is the only mandatory attribute */
+	if (NL_REQ_ATTR_CHECK(info->extack, NULL, tb, NET_SHAPER_A_HANDLE))
+		return -EINVAL;
+
+	ret = parse_handle(tb[NET_SHAPER_A_HANDLE], info, &shaper->handle);
+	if (ret)
+		return ret;
+
+	/* fetch existing data, if any, so that user provide info will
+	 * incrementally update the existing shaper configuration
+	 */
+	old = sc_lookup(dev, shaper->handle);
+	if (old)
+		*shaper = *old;
+	else
+		shaper->parent = default_parent(shaper->handle);
+
+	if (tb[NET_SHAPER_A_METRIC])
+		shaper->metric = nla_get_u32(tb[NET_SHAPER_A_METRIC]);
+
+	if (tb[NET_SHAPER_A_BW_MIN])
+		shaper->bw_min = nla_get_uint(tb[NET_SHAPER_A_BW_MIN]);
+
+	if (tb[NET_SHAPER_A_BW_MAX])
+		shaper->bw_max = nla_get_uint(tb[NET_SHAPER_A_BW_MAX]);
+
+	if (tb[NET_SHAPER_A_BURST])
+		shaper->burst = nla_get_uint(tb[NET_SHAPER_A_BURST]);
+
+	if (tb[NET_SHAPER_A_PRIORITY])
+		shaper->priority = nla_get_u32(tb[NET_SHAPER_A_PRIORITY]);
+
+	if (tb[NET_SHAPER_A_WEIGHT])
+		shaper->weight = nla_get_u32(tb[NET_SHAPER_A_WEIGHT]);
+	return 0;
+}
+
+/* fetch the cached shaper info and update them with the user-provided
+ * attributes
+ */
+static int parse_shaper(struct net_device *dev, const struct nlattr *attr,
+			const struct genl_info *info,
+			struct net_shaper_info *shaper)
+{
+	struct nlattr *tb[NET_SHAPER_A_WEIGHT + 1];
+	int ret;
+
+	ret = nla_parse_nested(tb, NET_SHAPER_A_WEIGHT, attr,
+			       net_shaper_ns_info_nl_policy, info->extack);
+	if (ret < 0)
+		return ret;
+
+	return __parse_shaper(dev, tb, info, shaper);
+}
+
 int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
 {
 	struct net_shaper_info *shaper;
@@ -239,14 +439,133 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb,
 	return ret;
 }
 
+/* Update the H/W and on success update the local cache, too */
+static int net_shaper_set(struct net_device *dev,
+			  const struct net_shaper_info *shaper,
+			  struct netlink_ext_ack *extack)
+{
+	enum net_shaper_scope scope;
+	u32 handle = shaper->handle;
+	int ret;
+
+	scope = net_shaper_handle_scope(handle);
+	if (scope == NET_SHAPER_SCOPE_PORT ||
+	    scope == NET_SHAPER_SCOPE_UNSPEC) {
+		NL_SET_ERR_MSG_FMT(extack, "Can't set shaper %x with scope %d",
+				   handle, scope);
+		return -EINVAL;
+	}
+	if (scope == NET_SHAPER_SCOPE_DETACHED && !sc_lookup(dev, handle)) {
+		NL_SET_ERR_MSG_FMT(extack, "Shaper %x with detached scope does not exist",
+				   handle);
+		return -EINVAL;
+	}
+
+	ret = sc_prepare_insert(dev, &handle, extack);
+	if (ret)
+		return ret;
+
+	ret = dev->netdev_ops->net_shaper_ops->set(dev, shaper, extack);
+	sc_commit(dev, 1, shaper);
+	return ret;
+}
+
 int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
 {
-	return -EOPNOTSUPP;
+	struct net_shaper_info shaper;
+	struct net_device *dev;
+	struct nlattr *attr;
+	int ret;
+
+	if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_SHAPER))
+		return -EINVAL;
+
+	ret = fetch_dev(info, &dev);
+	if (ret)
+		return ret;
+
+	attr = info->attrs[NET_SHAPER_A_SHAPER];
+	ret = parse_shaper(dev, attr, info, &shaper);
+	if (ret)
+		goto put;
+
+	ret = net_shaper_set(dev, &shaper, info->extack);
+
+put:
+	dev_put(dev);
+	return ret;
+}
+
+static int net_shaper_delete(struct net_device *dev, u32 handle,
+			     struct netlink_ext_ack *extack)
+{
+	struct net_shaper_info *parent, *shaper = sc_lookup(dev, handle);
+	struct xarray *xa = __sc_container(dev);
+	enum net_shaper_scope pscope;
+	u32 parent_handle;
+	int ret;
+
+	if (!xa || !shaper) {
+		NL_SET_ERR_MSG_FMT(extack, "Shaper %x not found", handle);
+		return -EINVAL;
+	}
+
+	if (is_detached(handle) && shaper->children > 0) {
+		NL_SET_ERR_MSG_FMT(extack, "Can't delete detached shaper %d with %d child nodes",
+				   handle, shaper->children);
+		return -EINVAL;
+	}
+
+	while (shaper) {
+		parent_handle = shaper->parent;
+		pscope = net_shaper_handle_scope(parent_handle);
+
+		ret = dev->netdev_ops->net_shaper_ops->delete(dev, handle, extack);
+		if (ret < 0)
+			return ret;
+
+		xa_lock(xa);
+		__xa_erase(xa, handle);
+		if (is_detached(handle))
+			idr_remove(&dev->net_shaper_data->detached_ids,
+				   net_shaper_handle_id(handle));
+		xa_unlock(xa);
+		kfree(shaper);
+		shaper = NULL;
+
+		if (pscope == NET_SHAPER_SCOPE_DETACHED) {
+			parent = sc_lookup(dev, parent_handle);
+			if (parent && !--parent->children) {
+				shaper = parent;
+				handle = parent_handle;
+			}
+		}
+	}
+	return 0;
 }
 
 int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
 {
-	return -EOPNOTSUPP;
+	struct net_device *dev;
+	u32 handle;
+	int ret;
+
+	if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
+		return -EINVAL;
+
+	ret = fetch_dev(info, &dev);
+	if (ret)
+		return ret;
+
+	ret = parse_handle(info->attrs[NET_SHAPER_A_HANDLE], info, &handle);
+	if (ret)
+		goto put;
+
+	ret = net_shaper_delete(dev, handle, info->extack);
+
+put:
+	dev_put(dev);
+	return ret;
 }
 
 int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
Signature · 1 line
2.45.2
author maintainer
#6 lore raw

RFC PATCH v2 4/11 net-shapers: implement NL group operation

In reply to Paolo Abeni

Alllow grouping multiple inputs shaper under the given output's one.

Try hard to pre-allocated the needed resources, to avoid non trivial H/W configuration rollbacks in case of any failure.

---
 net/shaper/shaper.c | 265 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 264 insertions(+), 1 deletion(-)
net/shaper/shaper.c +264 -1
index 7802c9ba6d9c..d99d000d7e7a 100644
@@ -48,6 +48,18 @@ static bool is_detached(u32 handle)
 	return net_shaper_handle_scope(handle) == NET_SHAPER_SCOPE_DETACHED;
 }
 
+/* count the number of [multi] attributes of the given type */
+static int attr_list_len(struct genl_info *info, int type)
+{
+	struct nlattr *attr;
+	int rem, cnt = 0;
+
+	nla_for_each_attr_type(attr, type, genlmsg_data(info->genlhdr),
+			       genlmsg_len(info->genlhdr), rem)
+		cnt++;
+	return cnt;
+}
+
 static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
 		       const struct genl_info *info)
 {
@@ -255,6 +267,27 @@ static void sc_commit(struct net_device *dev, int nr_shapers,
 	xa_unlock(xa);
 }
 
+/* rollback all the tentative inserts from the shaper cache */
+static void sc_rollback(struct net_device *dev)
+{
+	struct xarray *xa = __sc_container(dev);
+	struct net_shaper_info *cur;
+	unsigned long index;
+
+	if (!xa)
+		return;
+
+	xa_lock(xa);
+	xa_for_each_marked(xa, index, cur, XA_MARK_0) {
+		if (is_detached(index))
+			idr_remove(&dev->net_shaper_data->detached_ids,
+				   net_shaper_handle_id(index));
+		__xa_erase(xa, index);
+		kfree(cur);
+	}
+	xa_unlock(xa);
+}
+
 static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
 			u32 *handle)
 {
@@ -354,6 +387,36 @@ static int parse_shaper(struct net_device *dev, const struct nlattr *attr,
 	return __parse_shaper(dev, tb, info, shaper);
 }
 
+/* alike parse_shaper(), but additionally allow the user specifying
+ * the shaper's parent handle
+ */
+static int parse_output_shaper(struct net_device *dev,
+			       const struct nlattr *attr,
+			       const struct genl_info *info,
+			       struct net_shaper_info *shaper)
+{
+	struct nlattr *tb[NET_SHAPER_A_PARENT + 1];
+	int ret;
+
+	ret = nla_parse_nested(tb, NET_SHAPER_A_PARENT, attr,
+			       net_shaper_ns_output_info_nl_policy,
+			       info->extack);
+	if (ret < 0)
+		return ret;
+
+	ret = __parse_shaper(dev, tb, info, shaper);
+	if (ret)
+		return ret;
+
+	if (tb[NET_SHAPER_A_PARENT]) {
+		ret = parse_handle(tb[NET_SHAPER_A_PARENT], info,
+				   &shaper->parent);
+		if (ret)
+			return ret;
+	}
+	return 0;
+}
+
 int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
 {
 	struct net_shaper_info *shaper;
@@ -568,9 +631,209 @@ int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
 	return ret;
 }
 
+/* Update the H/W and on success update the local cache, too */
+static int net_shaper_group(struct net_device *dev, int nr_inputs,
+			    struct net_shaper_info *inputs,
+			    struct net_shaper_info *output,
+			    struct netlink_ext_ack *extack)
+{
+	enum net_shaper_scope scope, output_scope, output_pscope;
+	struct net_shaper_info *parent = NULL;
+	int i, ret;
+
+	output_scope = net_shaper_handle_scope(output->handle);
+	if (output_scope != NET_SHAPER_SCOPE_DETACHED &&
+	    output_scope != NET_SHAPER_SCOPE_NETDEV) {
+		NL_SET_ERR_MSG_FMT(extack, "Invalid scope %d for output shaper %x",
+				   output_scope, output->handle);
+		return -EINVAL;
+	}
+
+	output_pscope = net_shaper_handle_scope(output->parent);
+	if (output_scope == NET_SHAPER_SCOPE_DETACHED) {
+		if (net_shaper_handle_id(output->handle) !=
+		    NET_SHAPER_ID_UNSPEC && !sc_lookup(dev, output->handle)) {
+			NL_SET_ERR_MSG_FMT(extack, "Output shaper %x does not exists",
+					   output->handle);
+			return -EINVAL;
+		}
+		if (output_pscope != NET_SHAPER_SCOPE_DETACHED &&
+		    output_pscope != NET_SHAPER_SCOPE_NETDEV) {
+			NL_SET_ERR_MSG_FMT(extack, "Invalid scope %d for output parent shaper %x",
+					   output_pscope, output->parent);
+			return -EINVAL;
+		}
+	}
+
+	if (output_pscope == NET_SHAPER_SCOPE_DETACHED) {
+		parent = sc_lookup(dev, output->parent);
+		if (!parent) {
+			NL_SET_ERR_MSG_FMT(extack, "Output parent shaper %x does not exists",
+					   output->parent);
+			return -EINVAL;
+		}
+	}
+
+	/* for newly created detached scope shaper, the following will update the
+	 * handle, due to id allocation
+	 */
+	ret = sc_prepare_insert(dev, &output->handle, extack);
+	if (ret)
+		goto rollback;
+
+	for (i = 0; i < nr_inputs; ++i) {
+		scope = net_shaper_handle_scope(inputs[i].handle);
+		if (scope != NET_SHAPER_SCOPE_QUEUE &&
+		    scope != NET_SHAPER_SCOPE_DETACHED) {
+			ret = -EINVAL;
+			NL_SET_ERR_MSG_FMT(extack, "Invalid scope %d for input shaper %d handle %x",
+					   scope, i, inputs[i].handle);
+			goto rollback;
+		}
+		if (scope == NET_SHAPER_SCOPE_DETACHED &&
+		    !sc_lookup(dev, inputs[i].handle)) {
+			ret = -EINVAL;
+			NL_SET_ERR_MSG_FMT(extack,
+					   "Can't create detached shaper as input %d handle %x",
+					   i, inputs[i].handle);
+			goto rollback;
+		}
+
+		ret = sc_prepare_insert(dev, &inputs[i].handle, extack);
+		if (ret)
+			goto rollback;
+
+		/* the inputs shapers will be nested to the output */
+		if (inputs[i].parent != output->handle) {
+			inputs[i].parent = output->handle;
+			output->children++;
+		}
+	}
+
+	ret = dev->netdev_ops->net_shaper_ops->group(dev, nr_inputs,
+						     inputs, output,
+						     extack);
+	if (ret < 0)
+		goto rollback;
+
+	if (parent)
+		parent->children++;
+	sc_commit(dev, 1, output);
+	sc_commit(dev, nr_inputs, inputs);
+	return ret;
+
+rollback:
+	sc_rollback(dev);
+	return ret;
+}
+
+static int nla_handle_total_size(void)
+{
+	return nla_total_size(nla_total_size(sizeof(u32)) +
+			      nla_total_size(sizeof(u32)));
+}
+
+static int group_send_reply(struct genl_info *info, u32 handle, int id)
+{
+	struct nlattr *handle_attr;
+	struct sk_buff *msg;
+	int ret = -EMSGSIZE;
+	void *hdr;
+
+/* prepare the msg reply in advance, to avoid device operation rollback */
+	msg = genlmsg_new(nla_handle_total_size(), GFP_KERNEL);
+	if (!msg)
+		return ret;
+
+	hdr = genlmsg_iput(msg, info);
+	if (!hdr)
+		goto free_msg;
+
+	handle_attr = nla_nest_start(msg, NET_SHAPER_A_HANDLE);
+	if (!handle_attr)
+		goto free_msg;
+
+	if (nla_put_u32(msg, NET_SHAPER_A_SCOPE,
+			net_shaper_handle_scope(handle)))
+		goto free_msg;
+
+	if (nla_put_u32(msg, NET_SHAPER_A_ID, id))
+		goto free_msg;
+
+	nla_nest_end(msg, handle_attr);
+	genlmsg_end(msg, hdr);
+
+	ret =  genlmsg_reply(msg, info);
+	if (ret)
+		goto free_msg;
+
+	return ret;
+
+free_msg:
+	nlmsg_free(msg);
+	return ret;
+}
+
 int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
 {
-	return -EOPNOTSUPP;
+	struct net_shaper_info *inputs, output;
+	int i, ret, rem, nr_inputs;
+	struct net_device *dev;
+	struct nlattr *attr;
+
+	if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_INPUTS) ||
+	    GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_OUTPUT))
+		return -EINVAL;
+
+	ret = fetch_dev(info, &dev);
+	if (ret)
+		return ret;
+
+	nr_inputs = attr_list_len(info, NET_SHAPER_A_INPUTS);
+	inputs = kcalloc(nr_inputs, sizeof(struct net_shaper_info), GFP_KERNEL);
+	if (!inputs) {
+		GENL_SET_ERR_MSG_FMT(info, "Can't allocate memory for %d input shapers",
+				     nr_inputs);
+		ret = -ENOMEM;
+		goto put;
+	}
+
+	ret = parse_output_shaper(dev, info->attrs[NET_SHAPER_A_OUTPUT], info,
+				  &output);
+	if (ret)
+		goto free_shapers;
+
+	i = 0;
+	nla_for_each_attr_type(attr, NET_SHAPER_A_INPUTS,
+			       genlmsg_data(info->genlhdr),
+			       genlmsg_len(info->genlhdr), rem) {
+		if (WARN_ON_ONCE(i >= nr_inputs))
+			goto free_shapers;
+
+		ret = parse_shaper(dev, attr, info, &inputs[i++]);
+		if (ret)
+			goto free_shapers;
+	}
+
+	ret = net_shaper_group(dev, nr_inputs, inputs, &output, info->extack);
+	if (ret < 0)
+		goto free_shapers;
+
+	ret = group_send_reply(info, output.handle, ret);
+	if (ret) {
+		/* Error on reply is not fatal to avoid rollback a successful
+		 * configuration
+		 */
+		GENL_SET_ERR_MSG_FMT(info, "Can't send reply %d", ret);
+		ret = 0;
+	}
+
+free_shapers:
+	kfree(inputs);
+
+put:
+	dev_put(dev);
+	return ret;
 }
 
 void dev_shaper_flush(struct net_device *dev)
Signature · 1 line
2.45.2
author maintainer
#7 lore raw

RFC PATCH v2 5/11 netlink: spec: add shaper introspection support

In reply to Paolo Abeni

Allow the user-space to fine-grain query the shaping features supported by the NIC on each domain.

---
 Documentation/netlink/specs/shaper.yaml | 75 +++++++++++++++++++++++++
 include/uapi/linux/net_shaper.h         | 17 ++++++
 net/shaper/shaper.c                     | 11 ++++
 net/shaper/shaper_nl_gen.c              | 25 +++++++++
 net/shaper/shaper_nl_gen.h              |  3 +
 5 files changed, 131 insertions(+)
Documentation/netlink/specs/shaper.yaml +75 -0
index 7327f5596fdb..55bf4e49fb2f 100644
@@ -169,6 +169,52 @@ attribute-sets:
         name: priority
       -
         name: weight
+  -
+    name: capabilities
+    attributes:
+      -
+        name: ifindex
+        type: u32
+      -
+        name: scope
+        type: u32
+        enum: scope
+        doc: The scope to which the queried capabilities apply
+      -
+        name: support-metric-bps
+        type: flag
+        doc: the device accepts 'bps' metric for bw-min, bw-max and burst
+      -
+        name: support-metric-pps
+        type: flag
+        doc: the device accepts 'pps' metric for bw-min, bw-max and burst
+      -
+        name: support-nesting
+        type: flag
+        doc: |
+          the device supports nesting shaper belonging to this scope
+          below 'detached' scoped shapers. Only 'queue' and 'detached'
+          scope and flag 'support-nesting'.
+      -
+        name: support-bw-min
+        type: flag
+        doc: the device supports a minimum guaranteed bw
+      -
+        name: support-bw-max
+        type: flag
+        doc: the device supports maximum bw shaping
+      -
+        name: support-burst
+        type: flag
+        doc: the device supports a maximum burst size
+      -
+        name: support-priority
+        type: flag
+        doc: the device supports priority scheduling
+      -
+        name: support-weight
+        type: flag
+        doc: the device supports weighted round robin scheduling
 
 operations:
   list:
@@ -260,3 +306,32 @@ operations:
         reply:
           attributes:
             - handle
+
+    -
+      name: cap-get
+      doc: |
+        Get / Dump the shaper capabilities supported by the given device
+      attribute-set: capabilities
+
+      do:
+        request:
+          attributes:
+            - ifindex
+            - scope
+        reply:
+          attributes: &cap-attrs
+            - support-metric-bps
+            - support-metric-pps
+            - support-nesting
+            - support-bw-min
+            - support-bw-max
+            - support-burst
+            - support-priority
+            - support-weight
+
+      dump:
+        request:
+          attributes:
+            - ifindex
+        reply:
+          attributes: *cap-attrs
include/uapi/linux/net_shaper.h +17 -0
index ab3d4eb0e1ab..12d16d30472e 100644
@@ -61,11 +61,28 @@ enum {
 	NET_SHAPER_A_MAX = (__NET_SHAPER_A_MAX - 1)
 };
 
+enum {
+	NET_SHAPER_A_CAPABILITIES_IFINDEX = 1,
+	NET_SHAPER_A_CAPABILITIES_SCOPE,
+	NET_SHAPER_A_CAPABILITIES_SUPPORT_METRIC_BPS,
+	NET_SHAPER_A_CAPABILITIES_SUPPORT_METRIC_PPS,
+	NET_SHAPER_A_CAPABILITIES_SUPPORT_NESTING,
+	NET_SHAPER_A_CAPABILITIES_SUPPORT_BW_MIN,
+	NET_SHAPER_A_CAPABILITIES_SUPPORT_BW_MAX,
+	NET_SHAPER_A_CAPABILITIES_SUPPORT_BURST,
+	NET_SHAPER_A_CAPABILITIES_SUPPORT_PRIORITY,
+	NET_SHAPER_A_CAPABILITIES_SUPPORT_WEIGHT,
+
+	__NET_SHAPER_A_CAPABILITIES_MAX,
+	NET_SHAPER_A_CAPABILITIES_MAX = (__NET_SHAPER_A_CAPABILITIES_MAX - 1)
+};
+
 enum {
 	NET_SHAPER_CMD_GET = 1,
 	NET_SHAPER_CMD_SET,
 	NET_SHAPER_CMD_DELETE,
 	NET_SHAPER_CMD_GROUP,
+	NET_SHAPER_CMD_CAP_GET,
 
 	__NET_SHAPER_CMD_MAX,
 	NET_SHAPER_CMD_MAX = (__NET_SHAPER_CMD_MAX - 1)
net/shaper/shaper.c +11 -0
index d99d000d7e7a..7b7e9a777929 100644
@@ -836,6 +836,17 @@ int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
 	return ret;
 }
 
+int net_shaper_nl_cap_get_doit(struct sk_buff *skb, struct genl_info *info)
+{
+	return 0;
+}
+
+int net_shaper_nl_cap_get_dumpit(struct sk_buff *skb,
+				 struct netlink_callback *cb)
+{
+	return 0;
+}
+
 void dev_shaper_flush(struct net_device *dev)
 {
 	struct xarray *xa = __sc_container(dev);
net/shaper/shaper_nl_gen.c +25 -0
index b444d1ff55a1..cc98d58c0862 100644
@@ -67,6 +67,17 @@ static const struct nla_policy net_shaper_group_nl_policy[NET_SHAPER_A_OUTPUT +
 	[NET_SHAPER_A_OUTPUT] = NLA_POLICY_NESTED(net_shaper_ns_output_info_nl_policy),
 };
 
+/* NET_SHAPER_CMD_CAP_GET - do */
+static const struct nla_policy net_shaper_cap_get_do_nl_policy[NET_SHAPER_A_CAPABILITIES_SCOPE + 1] = {
+	[NET_SHAPER_A_CAPABILITIES_IFINDEX] = { .type = NLA_U32, },
+	[NET_SHAPER_A_CAPABILITIES_SCOPE] = NLA_POLICY_MAX(NLA_U32, 4),
+};
+
+/* NET_SHAPER_CMD_CAP_GET - dump */
+static const struct nla_policy net_shaper_cap_get_dump_nl_policy[NET_SHAPER_A_CAPABILITIES_IFINDEX + 1] = {
+	[NET_SHAPER_A_CAPABILITIES_IFINDEX] = { .type = NLA_U32, },
+};
+
 /* Ops table for net_shaper */
 static const struct genl_split_ops net_shaper_nl_ops[] = {
 	{
@@ -104,6 +115,20 @@ static const struct genl_split_ops net_shaper_nl_ops[] = {
 		.maxattr	= NET_SHAPER_A_OUTPUT,
 		.flags		= GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
 	},
+	{
+		.cmd		= NET_SHAPER_CMD_CAP_GET,
+		.doit		= net_shaper_nl_cap_get_doit,
+		.policy		= net_shaper_cap_get_do_nl_policy,
+		.maxattr	= NET_SHAPER_A_CAPABILITIES_SCOPE,
+		.flags		= GENL_CMD_CAP_DO,
+	},
+	{
+		.cmd		= NET_SHAPER_CMD_CAP_GET,
+		.dumpit		= net_shaper_nl_cap_get_dumpit,
+		.policy		= net_shaper_cap_get_dump_nl_policy,
+		.maxattr	= NET_SHAPER_A_CAPABILITIES_IFINDEX,
+		.flags		= GENL_CMD_CAP_DUMP,
+	},
 };
 
 struct genl_family net_shaper_nl_family __ro_after_init = {
net/shaper/shaper_nl_gen.h +3 -0
index 00cee4efd21e..97fbd4502364 100644
@@ -21,6 +21,9 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
 int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info);
 int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info);
 int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_cap_get_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_cap_get_dumpit(struct sk_buff *skb,
+				 struct netlink_callback *cb);
 
 extern struct genl_family net_shaper_nl_family;
 
Signature · 1 line
2.45.2
author maintainer
#8 lore raw

RFC PATCH v2 6/11 net: shaper: implement introspection support

In reply to Paolo Abeni

The netlink op is a simple wrapper around the device callback.

Extend the existing fetch_dev() helper adding an attribute argument for the requested device. Reuse such helper in the newly implemented operation.

---
 include/net/net_shaper.h |  11 ++++
 net/shaper/shaper.c      | 108 +++++++++++++++++++++++++++++++++++----
 2 files changed, 109 insertions(+), 10 deletions(-)
include/net/net_shaper.h +11 -0
index 8cd65d727e52..402f9d8476ee 100644
@@ -103,6 +103,17 @@ struct net_shaper_ops {
 	 */
 	int (*delete)(struct net_device *dev, u32 handle,
 		      struct netlink_ext_ack *extack);
+
+	/**
+	 * @capabilities: get the shaper features supported by the NIC
+	 *
+	 * Fills the bitmask @cap with the supported cababilites for the
+	 * specified @scope and device @dev.
+	 *
+	 * Returns 0 on success or a negative error value otherwise.
+	 */
+	int (*capabilities)(struct net_device *dev,
+			    enum net_shaper_scope scope, unsigned long *cap);
 };
 
 #define NET_SHAPER_SCOPE_SHIFT	26
net/shaper/shaper.c +98 -10

164 lines not on this page · raw message

Signature · 1 line
2.45.2
author maintainer
#9 lore raw

RFC PATCH v2 7/11 testing: net-drv: add basic shaper test

In reply to Paolo Abeni

Leverage a basic/dummy netdevsim implementation to do functional coverage for NL interface.

---
rfc v1 -> v2:
- added more test-cases WRT nesting and grouping

---
 drivers/net/Kconfig                           |   1 +
 drivers/net/netdevsim/netdev.c                |  37 +++
 tools/testing/selftests/drivers/net/Makefile  |   1 +
 tools/testing/selftests/drivers/net/shaper.py | 267 ++++++++++++++++++
 .../testing/selftests/net/lib/py/__init__.py  |   1 +
 tools/testing/selftests/net/lib/py/ynl.py     |   5 +
 6 files changed, 312 insertions(+)
 create mode 100755 tools/testing/selftests/drivers/net/shaper.py
drivers/net/Kconfig +1 -0
index 9920b3a68ed1..1fd5acdc73c6 100644
@@ -641,6 +641,7 @@ config NETDEVSIM
 	depends on PTP_1588_CLOCK_MOCK || PTP_1588_CLOCK_MOCK=n
 	select NET_DEVLINK
 	select PAGE_POOL
+	select NET_SHAPER
 	help
 	  This driver is a developer testing tool and software model that can
 	  be used to test various control path networking APIs, especially
drivers/net/netdevsim/netdev.c +37 -0
index 017a6102be0a..ab9c3d5055e7 100644
@@ -22,6 +22,7 @@
 #include <net/netdev_queues.h>
 #include <net/page_pool/helpers.h>
 #include <net/netlink.h>
+#include <net/net_shaper.h>
 #include <net/pkt_cls.h>
 #include <net/rtnetlink.h>
 #include <net/udp_tunnel.h>
@@ -475,6 +476,41 @@ static int nsim_stop(struct net_device *dev)
 	return 0;
 }
 
+static int nsim_shaper_set(struct net_device *dev,
+			   const struct net_shaper_info *shaper,
+			   struct netlink_ext_ack *extack)
+{
+	return 0;
+}
+
+static int nsim_shaper_del(struct net_device *dev, u32 handles,
+			   struct netlink_ext_ack *extack)
+{
+	return 0;
+}
+
+static int nsim_shaper_group(struct net_device *dev, int nr_inputs,
+			     const struct net_shaper_info *inputs,
+			     const struct net_shaper_info *output,
+			     struct netlink_ext_ack *extack)
+{
+	return 0;
+}
+
+static int nsim_shaper_cap(struct net_device *dev, enum net_shaper_scope scope,
+			   unsigned long *flags)
+{
+	*flags = ULONG_MAX;
+	return 0;
+}
+
+static const struct net_shaper_ops nsim_shaper_ops = {
+	.set			= nsim_shaper_set,
+	.delete			= nsim_shaper_del,
+	.group			= nsim_shaper_group,
+	.capabilities		= nsim_shaper_cap,
+};
+
 static const struct net_device_ops nsim_netdev_ops = {
 	.ndo_start_xmit		= nsim_start_xmit,
 	.ndo_set_rx_mode	= nsim_set_rx_mode,
@@ -496,6 +532,7 @@ static const struct net_device_ops nsim_netdev_ops = {
 	.ndo_bpf		= nsim_bpf,
 	.ndo_open		= nsim_open,
 	.ndo_stop		= nsim_stop,
+	.net_shaper_ops		= &nsim_shaper_ops,
 };
 
 static const struct net_device_ops nsim_vf_netdev_ops = {
tools/testing/selftests/drivers/net/Makefile +1 -0
index e54f382bcb02..432306f11664 100644
@@ -6,6 +6,7 @@ TEST_PROGS := \
 	ping.py \
 	queues.py \
 	stats.py \
+	shaper.py
 # end of TEST_PROGS
 
 include ../../lib.mk
tools/testing/selftests/drivers/net/shaper.py new file +267 -0

267 lines not on this page · raw message

tools/testing/selftests/net/lib/py/__init__.py +1 -0
index b6d498d125fe..ef1aa52f4761 100644
@@ -6,3 +6,4 @@ from .netns import NetNS
 from .nsim import *
 from .utils import *
 from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily
+from .ynl import ShaperFamily
tools/testing/selftests/net/lib/py/ynl.py +5 -0
index 1ace58370c06..42e825905ade 100644
@@ -47,3 +47,8 @@ class NetdevFamily(YnlFamily):
     def __init__(self):
         super().__init__((SPEC_PATH / Path('netdev.yaml')).as_posix(),
                          schema='')
+
+class ShaperFamily(YnlFamily):
+    def __init__(self):
+        super().__init__((SPEC_PATH / Path('shaper.yaml')).as_posix(),
+                         schema='')
Signature · 1 line
2.45.2
author maintainer
#10 lore raw

RFC PATCH v2 8/11 virtchnl: support queue rate limit and quanta size configuration

In reply to Paolo Abeni

From: Wenjun Wu <[email protected]>

This patch adds new virtchnl opcodes and structures for rate limit and quanta size configuration, which include:
1. VIRTCHNL_OP_CONFIG_QUEUE_BW, to configure max bandwidth for each VF per queue.
2. VIRTCHNL_OP_CONFIG_QUANTA, to configure quanta size per queue.
3. VIRTCHNL_OP_GET_QOS_CAPS, VF queries current QoS configuration, such as enabled TCs, arbiter type, up2tc and bandwidth of VSI node. The configuration is previously set by DCB and PF, and now is the potential QoS capability of VF. VF can take it as reference to configure queue TC mapping.

---
 include/linux/avf/virtchnl.h | 119 +++++++++++++++++++++++++++++++++++
 1 file changed, 119 insertions(+)
include/linux/avf/virtchnl.h +119 -0
index 8e177b67e82f..8ff31845f685 100644
@@ -89,6 +89,9 @@ enum virtchnl_rx_hsplit {
 	VIRTCHNL_RX_HSPLIT_SPLIT_SCTP    = 8,
 };
 
+enum virtchnl_bw_limit_type {
+	VIRTCHNL_BW_SHAPER = 0,
+};
 /* END GENERIC DEFINES */
 
 /* Opcodes for VF-PF communication. These are placed in the v_opcode field
@@ -151,6 +154,11 @@ enum virtchnl_ops {
 	VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
 	VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
 	VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+	/* opcode 57 - 65 are reserved */
+	VIRTCHNL_OP_GET_QOS_CAPS = 66,
+	/* opcode 68 through 111 are reserved */
+	VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+	VIRTCHNL_OP_CONFIG_QUANTA = 113,
 	VIRTCHNL_OP_MAX,
 };
 
@@ -260,6 +268,7 @@ VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
 #define VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC	BIT(26)
 #define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF		BIT(27)
 #define VIRTCHNL_VF_OFFLOAD_FDIR_PF		BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS			BIT(29)
 
 #define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
 			       VIRTCHNL_VF_OFFLOAD_VLAN | \
@@ -1405,6 +1414,85 @@ struct virtchnl_fdir_del {
 
 VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
 
+struct virtchnl_shaper_bw {
+	/* Unit is Kbps */
+	u32 committed;
+	u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+	u8 tc_num;
+	u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT      0
+#define VIRTCHNL_ABITER_ETS         2
+	u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT      1
+	u8 weight;
+	enum virtchnl_bw_limit_type type;
+	union {
+		struct virtchnl_shaper_bw shaper;
+		u8 pad2[32];
+	};
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+	u16 vsi_id;
+	u16 num_elem;
+	struct virtchnl_qos_cap_elem cap[];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_qos_cap_list);
+#define virtchnl_qos_cap_list_LEGACY_SIZEOF	44
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+	u16 queue_id;
+	u8 tc;
+	u8 pad;
+	struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+	u16 vsi_id;
+	u16 num_queues;
+	struct virtchnl_queue_bw cfg[];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_queues_bw_cfg);
+#define virtchnl_queues_bw_cfg_LEGACY_SIZEOF	16
+
+enum virtchnl_queue_type {
+	VIRTCHNL_QUEUE_TYPE_TX			= 0,
+	VIRTCHNL_QUEUE_TYPE_RX			= 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+	/* see enum virtchnl_queue_type */
+	s32 type;
+	u16 start_queue_id;
+	u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+struct virtchnl_quanta_cfg {
+	u16 quanta_size;
+	struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
 #define __vss_byone(p, member, count, old)				      \
 	(struct_size(p, member, count) + (old - 1 - struct_size(p, member, 0)))
 
@@ -1427,6 +1515,8 @@ VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
 		 __vss(virtchnl_vlan_filter_list_v2, __vss_byelem, p, m, c),  \
 		 __vss(virtchnl_tc_info, __vss_byelem, p, m, c),	      \
 		 __vss(virtchnl_rdma_qvlist_info, __vss_byelem, p, m, c),     \
+		 __vss(virtchnl_qos_cap_list, __vss_byelem, p, m, c),	      \
+		 __vss(virtchnl_queues_bw_cfg, __vss_byelem, p, m, c),	      \
 		 __vss(virtchnl_rss_key, __vss_byone, p, m, c),		      \
 		 __vss(virtchnl_rss_lut, __vss_byone, p, m, c))
 
@@ -1626,6 +1716,35 @@ virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
 	case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
 		valid_len = sizeof(struct virtchnl_vlan_setting);
 		break;
+	case VIRTCHNL_OP_GET_QOS_CAPS:
+		break;
+	case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+		valid_len = virtchnl_queues_bw_cfg_LEGACY_SIZEOF;
+		if (msglen >= valid_len) {
+			struct virtchnl_queues_bw_cfg *q_bw =
+				(struct virtchnl_queues_bw_cfg *)msg;
+
+			valid_len = virtchnl_struct_size(q_bw, cfg,
+							 q_bw->num_queues);
+			if (q_bw->num_queues == 0) {
+				err_msg_format = true;
+				break;
+			}
+		}
+		break;
+	case VIRTCHNL_OP_CONFIG_QUANTA:
+		valid_len = sizeof(struct virtchnl_quanta_cfg);
+		if (msglen >= valid_len) {
+			struct virtchnl_quanta_cfg *q_quanta =
+				(struct virtchnl_quanta_cfg *)msg;
+
+			if (q_quanta->quanta_size == 0 ||
+			    q_quanta->queue_select.num_queues == 0) {
+				err_msg_format = true;
+				break;
+			}
+		}
+		break;
 	/* These are always errors coming from the VF. */
 	case VIRTCHNL_OP_EVENT:
 	case VIRTCHNL_OP_UNKNOWN:
Signature · 1 line
2.45.2
author maintainer
#11 lore raw

RFC PATCH v2 9/11 ice: Support VF queue rate limit and quanta size configuration

In reply to Paolo Abeni

From: Wenjun Wu <[email protected]>

Add support to configure VF queue rate limit and quanta size.

For quanta size configuration, the quanta profiles are divided evenly by PF numbers. For each port, the first quanta profile is reserved for default. When VF is asked to set queue quanta size, PF will search for an available profile, change the fields and assigned this profile to the queue.

---
 drivers/net/ethernet/intel/ice/ice.h          |   2 +
 drivers/net/ethernet/intel/ice/ice_base.c     |   2 +
 drivers/net/ethernet/intel/ice/ice_common.c   |  21 ++
 .../net/ethernet/intel/ice/ice_hw_autogen.h   |   8 +
 drivers/net/ethernet/intel/ice/ice_txrx.h     |   1 +
 drivers/net/ethernet/intel/ice/ice_type.h     |   1 +
 drivers/net/ethernet/intel/ice/ice_vf_lib.h   |   8 +
 drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++++++++++++++
 drivers/net/ethernet/intel/ice/ice_virtchnl.h |  11 +
 .../intel/ice/ice_virtchnl_allowlist.c        |   6 +
 10 files changed, 393 insertions(+)
drivers/net/ethernet/intel/ice/ice.h +2 -0
index 99a75a59078e..ddb31ad284a4 100644
@@ -659,6 +659,8 @@ struct ice_pf {
 	struct ice_agg_node vf_agg_node[ICE_MAX_VF_AGG_NODES];
 	struct ice_dplls dplls;
 	struct device *hwmon_dev;
+
+	u8 num_quanta_prof_used;
 };
 
 extern struct workqueue_struct *ice_lag_wq;
drivers/net/ethernet/intel/ice/ice_base.c +2 -0
index 5d396c1a7731..54cd9fe27d77 100644
@@ -349,6 +349,8 @@ ice_setup_tx_ctx(struct ice_tx_ring *ring, struct ice_tlan_ctx *tlan_ctx, u16 pf
 		break;
 	}
 
+	tlan_ctx->quanta_prof_idx = ring->quanta_prof_id;
+
 	tlan_ctx->tso_ena = ICE_TX_LEGACY;
 	tlan_ctx->tso_qnum = pf_q;
 
drivers/net/ethernet/intel/ice/ice_common.c +21 -0
index 009716a12a26..b22e71dc59d4 100644
@@ -2436,6 +2436,25 @@ ice_parse_func_caps(struct ice_hw *hw, struct ice_hw_func_caps *func_p,
 	ice_recalc_port_limited_caps(hw, &func_p->common_cap);
 }
 
+/**
+ * ice_func_id_to_logical_id - map from function id to logical pf id
+ * @active_function_bitmap: active function bitmap
+ * @pf_id: function number of device
+ *
+ * Return: logical PF ID.
+ */
+static int ice_func_id_to_logical_id(u32 active_function_bitmap, u8 pf_id)
+{
+	u8 logical_id = 0;
+	u8 i;
+
+	for (i = 0; i < pf_id; i++)
+		if (active_function_bitmap & BIT(i))
+			logical_id++;
+
+	return logical_id;
+}
+
 /**
  * ice_parse_valid_functions_cap - Parse ICE_AQC_CAPS_VALID_FUNCTIONS caps
  * @hw: pointer to the HW struct
@@ -2453,6 +2472,8 @@ ice_parse_valid_functions_cap(struct ice_hw *hw, struct ice_hw_dev_caps *dev_p,
 	dev_p->num_funcs = hweight32(number);
 	ice_debug(hw, ICE_DBG_INIT, "dev caps: num_funcs = %d\n",
 		  dev_p->num_funcs);
+
+	hw->logical_pf_id = ice_func_id_to_logical_id(number, hw->pf_id);
 }
 
 /**
drivers/net/ethernet/intel/ice/ice_hw_autogen.h +8 -0
index 91cbae1eec89..af9302f0e376 100644
@@ -6,6 +6,14 @@
 #ifndef _ICE_HW_AUTOGEN_H_
 #define _ICE_HW_AUTOGEN_H_
 
+#define GLCOMM_QUANTA_PROF(_i)			(0x002D2D68 + ((_i) * 4))
+#define GLCOMM_QUANTA_PROF_MAX_INDEX		15
+#define GLCOMM_QUANTA_PROF_QUANTA_SIZE_S	0
+#define GLCOMM_QUANTA_PROF_QUANTA_SIZE_M	ICE_M(0x3FFF, 0)
+#define GLCOMM_QUANTA_PROF_MAX_CMD_S		16
+#define GLCOMM_QUANTA_PROF_MAX_CMD_M		ICE_M(0xFF, 16)
+#define GLCOMM_QUANTA_PROF_MAX_DESC_S		24
+#define GLCOMM_QUANTA_PROF_MAX_DESC_M		ICE_M(0x3F, 24)
 #define QTX_COMM_DBELL(_DBQM)			(0x002C0000 + ((_DBQM) * 4))
 #define QTX_COMM_HEAD(_DBQM)			(0x000E0000 + ((_DBQM) * 4))
 #define QTX_COMM_HEAD_HEAD_S			0
drivers/net/ethernet/intel/ice/ice_txrx.h +1 -0
index feba314a3fe4..ea2fae9035b5 100644
@@ -406,6 +406,7 @@ struct ice_tx_ring {
 #define ICE_TX_FLAGS_RING_VLAN_L2TAG2	BIT(2)
 	u8 flags;
 	u8 dcb_tc;			/* Traffic class of ring */
+	u16 quanta_prof_id;
 } ____cacheline_internodealigned_in_smp;
 
 static inline bool ice_ring_uses_build_skb(struct ice_rx_ring *ring)
drivers/net/ethernet/intel/ice/ice_type.h +1 -0
index 96037bef3e78..cad4d68b19dc 100644
@@ -903,6 +903,7 @@ struct ice_hw {
 	u8 revision_id;
 
 	u8 pf_id;		/* device profile info */
+	u8 logical_pf_id;
 
 	u16 max_burst_size;	/* driver sets this value */
 
drivers/net/ethernet/intel/ice/ice_vf_lib.h +8 -0
index fec16919ec19..becf5a009b4e 100644
@@ -52,6 +52,13 @@ struct ice_mdd_vf_events {
 	u16 last_printed;
 };
 
+struct ice_vf_qs_bw {
+	u32 committed;
+	u32 peak;
+	u16 queue_id;
+	u8 tc;
+};
+
 /* VF operations */
 struct ice_vf_ops {
 	enum ice_disq_rst_src reset_type;
@@ -132,6 +139,7 @@ struct ice_vf {
 	struct devlink_port devlink_port;
 
 	u16 num_msix;			/* num of MSI-X configured on this VF */
+	struct ice_vf_qs_bw qs_bw[ICE_MAX_RSS_QS_PER_VF];
 };
 
 /* Flags for controlling behavior of ice_reset_vf */
drivers/net/ethernet/intel/ice/ice_virtchnl.c +333 -0

363 lines not on this page · raw message

drivers/net/ethernet/intel/ice/ice_virtchnl.h +11 -0
index 3a4115869153..0c629aef9baf 100644
@@ -13,6 +13,13 @@
 /* Restrict number of MAC Addr and VLAN that non-trusted VF can programmed */
 #define ICE_MAX_VLAN_PER_VF		8
 
+#define ICE_DFLT_QUANTA 1024
+#define ICE_MAX_QUANTA_SIZE 4096
+#define ICE_MIN_QUANTA_SIZE 256
+
+#define calc_quanta_desc(x)	\
+	max_t(u16, 12, min_t(u16, 63, (((x) + 66) / 132) * 2 + 4))
+
 /* MAC filters: 1 is reserved for the VF's default/perm_addr/LAA MAC, 1 for
  * broadcast, and 16 for additional unicast/multicast filters
  */
@@ -61,6 +68,10 @@ struct ice_virtchnl_ops {
 	int (*dis_vlan_stripping_v2_msg)(struct ice_vf *vf, u8 *msg);
 	int (*ena_vlan_insertion_v2_msg)(struct ice_vf *vf, u8 *msg);
 	int (*dis_vlan_insertion_v2_msg)(struct ice_vf *vf, u8 *msg);
+	int (*get_qos_caps)(struct ice_vf *vf);
+	int (*cfg_q_tc_map)(struct ice_vf *vf, u8 *msg);
+	int (*cfg_q_bw)(struct ice_vf *vf, u8 *msg);
+	int (*cfg_q_quanta)(struct ice_vf *vf, u8 *msg);
 };
 
 #ifdef CONFIG_PCI_IOV
drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c +6 -0
index d796dbd2a440..c105a82ee136 100644
@@ -84,6 +84,11 @@ static const u32 fdir_pf_allowlist_opcodes[] = {
 	VIRTCHNL_OP_ADD_FDIR_FILTER, VIRTCHNL_OP_DEL_FDIR_FILTER,
 };
 
+static const u32 tc_allowlist_opcodes[] = {
+	VIRTCHNL_OP_GET_QOS_CAPS, VIRTCHNL_OP_CONFIG_QUEUE_BW,
+	VIRTCHNL_OP_CONFIG_QUANTA,
+};
+
 struct allowlist_opcode_info {
 	const u32 *opcodes;
 	size_t size;
@@ -104,6 +109,7 @@ static const struct allowlist_opcode_info allowlist_opcodes[] = {
 	ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF, adv_rss_pf_allowlist_opcodes),
 	ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_FDIR_PF, fdir_pf_allowlist_opcodes),
 	ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_VLAN_V2, vlan_v2_allowlist_opcodes),
+	ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_QOS, tc_allowlist_opcodes),
 };
 
 /**
Signature · 1 line
2.45.2
author maintainer
#12 lore raw

RFC PATCH v2 10/11 iavf: Add net_shaper_ops support

In reply to Paolo Abeni

From: Sudheer Mogilappagari <[email protected]>

Implement net_shaper_ops support for IAVF. This enables configuration of rate limiting on per queue basis. Customer intends to enforce bandwidth limit on Tx traffic steered to the queue by configuring rate limits on the queue.

To set rate limiting for a queue, update shaper object of given queues in driver and send VIRTCHNL_OP_CONFIG_QUEUE_BW to PF to update HW configuration.

Deleting shaper configured for queue is nothing but configuring shaper with bw_max 0. The PF restores the default rate limiting config when bw_max is zero.

---
 drivers/net/ethernet/intel/Kconfig            |   1 +
 drivers/net/ethernet/intel/iavf/iavf.h        |   3 +
 drivers/net/ethernet/intel/iavf/iavf_main.c   | 171 ++++++++++++++++++
 drivers/net/ethernet/intel/iavf/iavf_txrx.h   |   2 +
 .../net/ethernet/intel/iavf/iavf_virtchnl.c   |  65 +++++++
 5 files changed, 242 insertions(+)
drivers/net/ethernet/intel/Kconfig +1 -0
index 0375c7448a57..20bc40eec487 100644
@@ -258,6 +258,7 @@ config I40E_DCB
 config IAVF
 	tristate
 	select LIBIE
+	select NET_SHAPER
 
 config I40EVF
 	tristate "Intel(R) Ethernet Adaptive Virtual Function support"
drivers/net/ethernet/intel/iavf/iavf.h +3 -0
index 23a6557fc3db..f5d1142ea427 100644
@@ -33,6 +33,7 @@
 #include <net/udp.h>
 #include <net/tc_act/tc_gact.h>
 #include <net/tc_act/tc_mirred.h>
+#include <net/net_shaper.h>
 
 #include "iavf_type.h"
 #include <linux/avf/virtchnl.h>
@@ -335,6 +336,7 @@ struct iavf_adapter {
 #define IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_INSERTION	BIT_ULL(36)
 #define IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION		BIT_ULL(37)
 #define IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION	BIT_ULL(38)
+#define IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW		BIT_ULL(39)
 
 	/* flags for processing extended capability messages during
 	 * __IAVF_INIT_EXTENDED_CAPS. Each capability exchange requires
@@ -551,6 +553,7 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
 int iavf_config_rss(struct iavf_adapter *adapter);
 int iavf_lan_add_device(struct iavf_adapter *adapter);
 int iavf_lan_del_device(struct iavf_adapter *adapter);
+void iavf_cfg_queues_bw(struct iavf_adapter *adapter);
 void iavf_enable_channels(struct iavf_adapter *adapter);
 void iavf_disable_channels(struct iavf_adapter *adapter);
 void iavf_add_cloud_filter(struct iavf_adapter *adapter);
drivers/net/ethernet/intel/iavf/iavf_main.c +171 -0

201 lines not on this page · raw message

drivers/net/ethernet/intel/iavf/iavf_txrx.h +2 -0
index d7b5587aeb8e..dd503ee50b7f 100644
@@ -296,6 +296,8 @@ struct iavf_ring {
 					 */
 
 	u32 rx_buf_len;
+	struct net_shaper_info q_shaper;
+	bool q_shaper_update;
 } ____cacheline_internodealigned_in_smp;
 
 #define IAVF_ITR_ADAPTIVE_MIN_INC	0x0002
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +65 -0

83 lines not on this page · raw message

Signature · 1 line
2.45.2
author maintainer
#14 lore raw

RFC PATCH v2 11/11 iavf: add support to exchange qos capabilities

In reply to Paolo Abeni

From: Sudheer Mogilappagari <[email protected]>

During driver initialization VF determines QOS capability is allowed by PF and receives QOS parameters. After which quanta size for queues is configured which is not configurable and is set to 1KB currently.

---
 drivers/net/ethernet/intel/iavf/iavf.h        | 10 ++
 drivers/net/ethernet/intel/iavf/iavf_main.c   | 46 +++++++++-
 .../net/ethernet/intel/iavf/iavf_virtchnl.c   | 92 ++++++++++++++++++-
 3 files changed, 145 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/iavf/iavf.h +10 -0
index f5d1142ea427..dd9cca067360 100644
@@ -250,6 +250,9 @@ struct iavf_cloud_filter {
 #define IAVF_RESET_WAIT_DETECTED_COUNT 500
 #define IAVF_RESET_WAIT_COMPLETE_COUNT 2000
 
+#define IAVF_MAX_QOS_TC_NUM		8
+#define IAVF_DEFAULT_QUANTA_SIZE	1024
+
 /* board specific private data structure */
 struct iavf_adapter {
 	struct workqueue_struct *wq;
@@ -337,6 +340,8 @@ struct iavf_adapter {
 #define IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION		BIT_ULL(37)
 #define IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION	BIT_ULL(38)
 #define IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW		BIT_ULL(39)
+#define IAVF_FLAG_AQ_CFG_QUEUES_QUANTA_SIZE		BIT_ULL(40)
+#define IAVF_FLAG_AQ_GET_QOS_CAPS			BIT_ULL(41)
 
 	/* flags for processing extended capability messages during
 	 * __IAVF_INIT_EXTENDED_CAPS. Each capability exchange requires
@@ -407,6 +412,8 @@ struct iavf_adapter {
 			       VIRTCHNL_VF_OFFLOAD_FDIR_PF)
 #define ADV_RSS_SUPPORT(_a) ((_a)->vf_res->vf_cap_flags & \
 			     VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF)
+#define QOS_ALLOWED(_a) ((_a)->vf_res->vf_cap_flags & \
+			 VIRTCHNL_VF_OFFLOAD_QOS)
 	struct virtchnl_vf_resource *vf_res; /* incl. all VSIs */
 	struct virtchnl_vsi_resource *vsi_res; /* our LAN VSI */
 	struct virtchnl_version_info pf_version;
@@ -415,6 +422,7 @@ struct iavf_adapter {
 	struct virtchnl_vlan_caps vlan_v2_caps;
 	u16 msg_enable;
 	struct iavf_eth_stats current_stats;
+	struct virtchnl_qos_cap_list *qos_caps;
 	struct iavf_vsi vsi;
 	u32 aq_wait_count;
 	/* RSS stuff */
@@ -554,6 +562,8 @@ int iavf_config_rss(struct iavf_adapter *adapter);
 int iavf_lan_add_device(struct iavf_adapter *adapter);
 int iavf_lan_del_device(struct iavf_adapter *adapter);
 void iavf_cfg_queues_bw(struct iavf_adapter *adapter);
+void iavf_cfg_queues_quanta_size(struct iavf_adapter *adapter);
+void iavf_get_qos_caps(struct iavf_adapter *adapter);
 void iavf_enable_channels(struct iavf_adapter *adapter);
 void iavf_disable_channels(struct iavf_adapter *adapter);
 void iavf_add_cloud_filter(struct iavf_adapter *adapter);
drivers/net/ethernet/intel/iavf/iavf_main.c +44 -2

82 lines not on this page · raw message

drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +91 -1

116 lines not on this page · raw message

Signature · 1 line
2.45.2
author maintainer
#2 lore raw

Re: [PATCH RFC v2 01/11] netlink: spec: add shaper YAML spec

In reply to Paolo Abeni
Quoted 11 lines · Paolo Abeni

Define the user-space visible interface to query, configure and delete network shapers via yaml definition.

Add dummy implementations for the relevant NL callbacks.

set() and delete() operations touch a single shaper creating/updating or deleting it.
The group() operation creates a shaper's group, nesting multiple input shapers under the specified output shaper.

Quoted 8 lines

---
RFC v1 -> RFC v2:
- u64 -> uint
- net_shapers -> net-shapers
- documented all the attributes
- dropped [ admin-perm ] for get() op
- group op
- set/delete touch a single shaper

FWIW, the CI told me I forgot entirely about the generated user-space code, and the naming here is not supported by the current tooling. I'll send something alike the following when net-next re-open.
---

tools/net/ynl/ynl-gen-c.py +4 -2
index 51529fabd517..717530bc9c52 100755
@@ -2668,13 +2668,15 @@ def main():
         cw.p('#define ' + hdr_prot)
         cw.nl()
 
+    hdr_file=os.path.basename(args.out_file[:-2]) + ".h"
+
     if args.mode == 'kernel':
         cw.p('#include <net/netlink.h>')
         cw.p('#include <net/genetlink.h>')
         cw.nl()
         if not args.header:
             if args.out_file:
-                cw.p(f'#include "{os.path.basename(args.out_file[:-2])}.h"')
+                cw.p(f'#include "{hdr_file}"')
             cw.nl()
         headers = ['uapi/' + parsed.uapi_header]
         headers += parsed.kernel_family.get('headers', [])
@@ -2686,7 +2688,7 @@ def main():
             if family_contains_bitfield32(parsed):
                 cw.p('#include <linux/netlink.h>')
         else:
-            cw.p(f'#include "{parsed.name}-user.h"')
+            cw.p(f'#include "{hdr_file}"')
             cw.p('#include "ynl.h"')
         headers = [parsed.uapi_header]
     for definition in parsed['definitions']:
#15 lore raw
In reply to Paolo Abeni
Quoted 5 lines · Wed, Jul 24, 2024 at 10:24:46PM CEST, [email protected] wrote:

We have a plurality of shaping-related drivers API, but none flexible enough to meet existing demand from vendors[1].

This series introduces new device APIs to configure in a flexible way TX shaping H/W offload. The new functionality is exposed via a newly

From what I understand, and please correct me if I'm wrong, this patchset is about HW shaper configuration. Basically it provides new UAPI to configure the HW shaper. So Why you say "offload"? I don't see anything being offloaded here.

Also, from the previous discussions, I gained impression that the goal of this work is to replace multiple driver apis for the shaper and consolidate it under new one. I don't see anything like this in this patchset. Do you plan it as a follow-up? Do you have RFC for that step as well?

Quoted 148 lines

defined generic netlink interface and includes introspection capabilities. Some self-tests are included, on top of a dummy netdevsim implementation, and a basic implementation for the iavf driver.

Some usage examples:

* Configure shaping on a given queue:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do set --json '{"ifindex":'$IFINDEX',
			"shaper": {"handle":
				{"scope": "queue", "id":'$QUEUEID' },
			"bw-max": 2000000 }}'

* Container B/W sharing

The orchestration infrastructure wants to group the container-related queues under a RR scheduling and limit the aggregate bandwidth:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do group --json '{"ifindex":'$IFINDEX', 
			"inputs": [ 
			  {"handle": {"scope": "queue", "id":'$QID1' },
			   "weight": '$W1'}, 
			  {"handle": {"scope": "queue", "id":'$QID2' },
			   "weight": '$W2'}], 
			  {"handle": {"scope": "queue", "id":'$QID3' },
			   "weight": '$W3'}], 
			"output": { "handle": {"scope":"netdev"},
			"output": { "handle": {"scope":"netdev"},
			   "bw-max": 10000000}}'

{'handle': {'id': 0, 'scope': 'netdev'}}

* Delegation

A container wants to set a B/W limit on 2 of its own queues:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do group --json '{"ifindex":'$IFINDEX', 
			"inputs": [ 
			  {"handle": {"scope": "queue", "id":'$QID1' },
			   "weight": '$W1'}, 
			  {"handle": {"scope": "queue", "id":'$QID2' },
			   "weight": '$W2'}], 
			"output": { "handle": {"scope":"detached"},
			   "bw-max": 5000000}}'

{'handle': {'id': 0, 'scope': 'detached'}}

* Cleanup:

Deleting a single queue shaper:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do delete --json \
	'{"ifindex":'$IFINDEX', 
	  "handle": {"scope": "queue", "id":'$QID1' }}'

Deleting the last shaper under a group deletes the group, too:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do delete --json \
	'{"ifindex":'$IFINDEX', 
	  "handle": {"scope": "queue", "id":'$QID2' }}'
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
       --do get --json '{"ifindex":'$IF', 
			  "handle": { "scope": "detached", "id": 0}}'
Netlink error: Invalid argument
nl_len = 80 (64) nl_flags = 0x300 nl_type = 2
	error: -22
	extack: {'msg': "Can't find shaper for handle 10000000"}

Changes from RFC v1:
- set() and delete() ops operate on a single shaper
- added group() op to allow grouping and nesting
- split the NL implementation into multiple patches to help reviewing

RFC v1: https://lore.kernel.org/netdev/[email protected]/

[1] https://lore.kernel.org/netdev/[email protected]/

Paolo Abeni (7):
netlink: spec: add shaper YAML spec
net-shapers: implement NL get operation
net-shapers: implement NL set and delete operations
net-shapers: implement NL group operation
netlink: spec: add shaper introspection support
net: shaper: implement introspection support
testing: net-drv: add basic shaper test

Sudheer Mogilappagari (2):
iavf: Add net_shaper_ops support
iavf: add support to exchange qos capabilities

Wenjun Wu (2):
virtchnl: support queue rate limit and quanta size configuration
ice: Support VF queue rate limit and quanta size configuration

Documentation/netlink/specs/shaper.yaml       | 337 ++++++
Documentation/networking/kapi.rst             |   3 +
MAINTAINERS                                   |   1 +
drivers/net/Kconfig                           |   1 +
drivers/net/ethernet/intel/Kconfig            |   1 +
drivers/net/ethernet/intel/iavf/iavf.h        |  13 +
drivers/net/ethernet/intel/iavf/iavf_main.c   | 215 +++-
drivers/net/ethernet/intel/iavf/iavf_txrx.h   |   2 +
.../net/ethernet/intel/iavf/iavf_virtchnl.c   | 157 ++-
drivers/net/ethernet/intel/ice/ice.h          |   2 +
drivers/net/ethernet/intel/ice/ice_base.c     |   2 +
drivers/net/ethernet/intel/ice/ice_common.c   |  21 +
.../net/ethernet/intel/ice/ice_hw_autogen.h   |   8 +
drivers/net/ethernet/intel/ice/ice_txrx.h     |   1 +
drivers/net/ethernet/intel/ice/ice_type.h     |   1 +
drivers/net/ethernet/intel/ice/ice_vf_lib.h   |   8 +
drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++
drivers/net/ethernet/intel/ice/ice_virtchnl.h |  11 +
.../intel/ice/ice_virtchnl_allowlist.c        |   6 +
drivers/net/netdevsim/netdev.c                |  37 +
include/linux/avf/virtchnl.h                  | 119 +++
include/linux/netdevice.h                     |  17 +
include/net/net_shaper.h                      | 169 +++
include/uapi/linux/net_shaper.h               |  91 ++
net/Kconfig                                   |   3 +
net/Makefile                                  |   1 +
net/core/dev.c                                |   2 +
net/core/dev.h                                |   6 +
net/shaper/Makefile                           |   9 +
net/shaper/shaper.c                           | 962 ++++++++++++++++++
net/shaper/shaper_nl_gen.c                    | 142 +++
net/shaper/shaper_nl_gen.h                    |  30 +
tools/testing/selftests/drivers/net/Makefile  |   1 +
tools/testing/selftests/drivers/net/shaper.py | 267 +++++
.../testing/selftests/net/lib/py/__init__.py  |   1 +
tools/testing/selftests/net/lib/py/ynl.py     |   5 +
36 files changed, 2983 insertions(+), 2 deletions(-)

create mode 100644 Documentation/netlink/specs/shaper.yaml create mode 100644 include/net/net_shaper.h
create mode 100644 include/uapi/linux/net_shaper.h create mode 100644 net/shaper/Makefile
create mode 100644 net/shaper/shaper.c
create mode 100644 net/shaper/shaper_nl_gen.c
create mode 100644 net/shaper/shaper_nl_gen.h
create mode 100755 tools/testing/selftests/drivers/net/shaper.py

Signature · 1 line
2.45.2
author maintainer
#16 lore raw
Quoted 4 lines · Jiri Pirko

From what I understand, and please correct me if I'm wrong, this
patchset is about HW shaper configuration. Basically it provides new UAPI to configure the HW shaper. So Why you say "offload"? I don't see anything being offloaded here.

The offload part comes from the initial, very old tentative solution. I guess we can change the title to "net: introduce TX H/W shaping API"

Quoted 5 lines

Also, from the previous discussions, I gained impression that the goal of this work is to replace multiple driver apis for the shaper and consolidate it under new one. I don't see anything like this in this patchset. Do you plan it as a follow-up? Do you have RFC for that step as well?

The general idea is, with time, to leverage this API to replace others H/W shaping related in-kernel interfaces.

At least ndo_set_tx_maxrate() should be quite straight-forward, after that the relevant device drivers have implemented (very limited) support for this API.

The latter will need some effort from the drivers' owners.

Thanks,

Paolo

#17 lore raw
Quoted 15 lines · Mon, Jul 29, 2024 at 04:42:19PM CEST, [email protected] wrote:
Quoted 4 lines · Jiri Pirko

From what I understand, and please correct me if I'm wrong, this
patchset is about HW shaper configuration. Basically it provides new UAPI to configure the HW shaper. So Why you say "offload"? I don't see anything being offloaded here.

The offload part comes from the initial, very old tentative solution. I guess we can change the title to "net: introduce TX H/W shaping API"

Quoted 5 lines

Also, from the previous discussions, I gained impression that the goal of this work is to replace multiple driver apis for the shaper and consolidate it under new one. I don't see anything like this in this patchset. Do you plan it as a follow-up? Do you have RFC for that step as well?

Quoted 6 lines

The general idea is, with time, to leverage this API to replace others H/W shaping related in-kernel interfaces.

At least ndo_set_tx_maxrate() should be quite straight-forward, after that the relevant device drivers have implemented (very limited) support for this API.

Could you try to draft at least one example per each user? I mean, this is likely to be the tricky part of this work, would be great to make that click from very beginning.

Quoted 2 lines

The latter will need some effort from the drivers' owners.

Let me know what you need exactly. Will try to do my best to help.

Thanks!

Quoted 5 lines

Thanks,

Paolo

author maintainer
#18 lore raw
Quoted 11 lines · Jiri Pirko
Quoted 6 lines · Mon, Jul 29, 2024 at 04:42:19PM CEST, [email protected] wrote:

The general idea is, with time, to leverage this API to replace others H/W shaping related in-kernel interfaces.

At least ndo_set_tx_maxrate() should be quite straight-forward, after that the relevant device drivers have implemented (very limited) support for this API.

Could you try to draft at least one example per each user? I mean, this is likely to be the tricky part of this work, would be great to make that click from very beginning.

I think we need to clarify that we are not going to replace all the existing in-kernel interfaces that somewhat configure shapers on the network devices.

Trying to achieve the above would be a (not so) nice way to block this effort forever.

ndo_set_tx_maxrate() is IMHO a good, feasible example. Others could be ieee_setmaxrate() or ndo_set_vf_rate() - ignoring the “obsoleted” argument.

Quoted 3 lines
Quoted 1 line

The latter will need some effort from the drivers' owners.

Let me know what you need exactly. Will try to do my best to help.

Something like what is implemented in this series for the iavf driver would suffice.

Thanks,

Paolo

#13 lore raw

Re: [PATCH RFC v2 10/11] iavf: Add net_shaper_ops support

In reply to Paolo Abeni
Quoted 1 line · Wed, Jul 24, 2024 at 10:24:56PM CEST, [email protected] wrote:

From: Sudheer Mogilappagari <[email protected]>

[...]

Quoted 12 lines
drivers/net/ethernet/intel/iavf/iavf_main.c +12 -0
+static int
+iavf_shaper_set(struct net_device *dev,
+		const struct net_shaper_info *shaper,
+		struct netlink_ext_ack *extack)
+{
+	struct iavf_adapter *adapter = netdev_priv(dev);
+	bool need_cfg_update = false;
+	enum net_shaper_scope scope;
+	int id, ret = 0;
+
+	ret = iavf_verify_shaper_info(dev, shaper, extack);
+	if (ret)

drivers/net/ethernet/intel/iavf/iavf_main.c static netdev_features_t iavf_fix_features(struct net_device *netdev, · line 4847

Quoted 10 lines
drivers/net/ethernet/intel/iavf/iavf_main.c +10 -0
+		return ret;
+
+	scope = net_shaper_handle_scope(shaper->handle);
+	id = net_shaper_handle_id(shaper->handle);
+
+	if (scope == NET_SHAPER_SCOPE_QUEUE) {
+		struct iavf_ring *tx_ring = &adapter->tx_rings[id];
+
+		tx_ring->q_shaper.bw_min = div_u64(shaper->bw_min, 1000);
+		tx_ring->q_shaper.bw_max = div_u64(shaper->bw_max, 1000);

Isn't this more or less what ndo_set_tx_maxrate() does? I mean, the only difference I see is ability to set min/max. Do I read this correctly?

If yes, could we have the ndo_set_tx_maxrate() consolidation included in this patchset (or at least to post follow-up RFC)? I mean, without that, there are 2 uapis to set queue rate limit for different drivers (net-shaper for iavf, sysfs tx_maxrate for the rest), which looks messy.

Quoted 9 lines
drivers/net/ethernet/intel/iavf/iavf_main.c +9 -0
+		tx_ring->q_shaper_update = true;
+		need_cfg_update = true;
+	}
+
+	if (need_cfg_update)
+		adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW;
+
+	return 0;
+}

[...]

#5 lore raw

Re: [PATCH RFC v2 03/11] net-shapers: implement NL set and delete operations

In reply to Paolo Abeni
Quoted 16 lines · Wed, Jul 24, 2024 at 10:24:49PM CEST, [email protected] wrote:

Both NL operations directly map on the homonymous device shaper callbacks and update accordingly the shapers cache. Implement the cache modification helpers to additionally deal with DETACHED scope shaper. That will be needed by the group() operation implemented in the next patch.

---
net/shaper/shaper.c | 323 +++++++++++++++++++++++++++++++++++++++++++-

1 file changed, 321 insertions(+), 2 deletions(-)
net/shaper/shaper.c +0 -0
index 93dd491ac7c2..7802c9ba6d9c 100644

net/shaper/shaper.c struct net_shaper_nl_ctx { · line 24

Quoted 6 lines
net/shaper/shaper.c +3 -0
@@ -19,6 +19,35 @@ struct net_shaper_nl_ctx {
 	u32 start_handle;
 };
 
+static u32 default_parent(u32 handle)
+{
+	enum net_shaper_scope parent, scope = net_shaper_handle_scope(handle);

In the rest of the code you call this "pscope", could you rename?

net/shaper/shaper.c struct net_shaper_nl_ctx { · line 33

Quoted 9 lines
net/shaper/shaper.c +9 -0
+
+	switch (scope) {
+	case NET_SHAPER_SCOPE_PORT:
+	case NET_SHAPER_SCOPE_UNSPEC:
+		parent = NET_SHAPER_SCOPE_UNSPEC;
+		break;
+
+	case NET_SHAPER_SCOPE_QUEUE:
+	case NET_SHAPER_SCOPE_DETACHED:

Why default parent of detached is netdev? Detached means "node in tree" if IIUC. Can't port be parent?

Quoted 34 lines
net/shaper/shaper.c +27 -0
+		parent = NET_SHAPER_SCOPE_NETDEV;
+		break;
+
+	case NET_SHAPER_SCOPE_NETDEV:
+	case NET_SHAPER_SCOPE_VF:
+		parent = NET_SHAPER_SCOPE_PORT;
+		break;
+	}
+
+	return net_shaper_make_handle(parent, 0);
+}
+
+static bool is_detached(u32 handle)
+{
+	return net_shaper_handle_scope(handle) == NET_SHAPER_SCOPE_DETACHED;
+}
+
 static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
 		       const struct genl_info *info)
 {
@@ -117,6 +146,115 @@ static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle)
 	return xa ? xa_load(xa, handle) : NULL;
 }
 
+/* allocate on demand the per device shaper's cache */
+static struct xarray *__sc_init(struct net_device *dev,
+				struct netlink_ext_ack *extack)
+{
+	if (!dev->net_shaper_data) {
+		dev->net_shaper_data = kmalloc(sizeof(*dev->net_shaper_data),
+					       GFP_KERNEL);
+		if (!dev->net_shaper_data) {
+			NL_SET_ERR_MSG(extack, "Can't allocate memory for shaper data");
+			return NULL;

net/shaper/shaper.c static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle) · line 168

Quoted 10 lines
net/shaper/shaper.c +10 -0
+		}
+
+		xa_init(&dev->net_shaper_data->shapers);
+		idr_init(&dev->net_shaper_data->detached_ids);
+	}
+	return &dev->net_shaper_data->shapers;
+}
+
+/* prepare the cache to actually insert the given shaper, doing
+ * in advance the needed allocations

In comments (all), would it make sense to start the sentence with capital letter and end it with "."?

Quoted 8 lines
net/shaper/shaper.c +8 -0
+ */
+static int sc_prepare_insert(struct net_device *dev, u32 *handle,
+			     struct netlink_ext_ack *extack)
+{
+	enum net_shaper_scope scope = net_shaper_handle_scope(*handle);
+	struct xarray *xa = __sc_init(dev, extack);
+	struct net_shaper_info *prev, *cur;
+	bool id_allocated = false;

net/shaper/shaper.c static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle) · line 186

Quoted 10 lines
net/shaper/shaper.c +10 -0
+	int ret, id;
+
+	if (!xa)
+		return -ENOMEM;
+
+	cur = xa_load(xa, *handle);
+	if (cur)
+		return 0;
+
+	/* allocated a new id, if needed */

s/allocated/allocate/ perhaps ?

Quoted 17 lines
net/shaper/shaper.c +17 -0
+	if (scope == NET_SHAPER_SCOPE_DETACHED &&
+	    net_shaper_handle_id(*handle) == NET_SHAPER_ID_UNSPEC) {
+		xa_lock(xa);
+		id = idr_alloc(&dev->net_shaper_data->detached_ids, NULL,
+			       0, NET_SHAPER_ID_UNSPEC, GFP_ATOMIC);
+		xa_unlock(xa);
+
+		if (id < 0) {
+			NL_SET_ERR_MSG(extack, "Can't allocate new id for detached shaper");
+			return id;
+		}
+
+		*handle = net_shaper_make_handle(scope, id);
+		id_allocated = true;
+	}
+
+	cur = kmalloc(sizeof(*cur), GFP_KERNEL | __GFP_ZERO);

net/shaper/shaper.c static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle) · line 213

Quoted 10 lines
net/shaper/shaper.c +10 -0
+	if (!cur) {
+		NL_SET_ERR_MSG(extack, "Can't allocate memory for cached shaper");
+		ret = -ENOMEM;
+		goto free_id;
+	}
+
+	/* mark 'tentative' shaper inside the cache */
+	xa_lock(xa);
+	prev = __xa_store(xa, *handle, cur, GFP_KERNEL);
+	__xa_set_mark(xa, *handle, XA_MARK_0);

I think it is nice to actually have a define for the mark that indicates what a certain mark represents.

Also, this patch introduces a bug in net_shaper_nl_get_dumpit()/net_shaper_nl_get_doit() which now work with uncommitted shapers. You have to check the mark.

net/shaper/shaper.c static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle) · line 225

Quoted 12 lines
net/shaper/shaper.c +12 -0
+	xa_unlock(xa);
+	if (xa_err(prev)) {
+		NL_SET_ERR_MSG(extack, "Can't insert shaper into cache");
+		kfree(cur);
+		ret = xa_err(prev);
+		goto free_id;
+	}
+	return 0;
+
+free_id:
+	if (id_allocated) {
+		xa_lock(xa);

May be worth mentioning you protect detached_ids with the xa lock.

Quoted 36 lines
net/shaper/shaper.c +32 -0
+		idr_remove(&dev->net_shaper_data->detached_ids,
+			   net_shaper_handle_id(*handle));
+		xa_unlock(xa);
+	}
+	return ret;
+}
+
+/* commit the tentative insert with the actual values.
+ * Must be called only after a successful sc_prepare_insert()
+ */
+static void sc_commit(struct net_device *dev, int nr_shapers,
+		      const struct net_shaper_info *shapers)
+{
+	struct xarray *xa = __sc_container(dev);
+	struct net_shaper_info *cur;
+	int i;
+
+	xa_lock(xa);
+	for (i = 0; i < nr_shapers; ++i) {
+		cur = xa_load(xa, shapers[i].handle);
+		if (WARN_ON_ONCE(!cur))
+			continue;
+
+		/* successful update: drop the tentative mark
+		 * and update the cache
+		 */
+		__xa_clear_mark(xa, shapers[i].handle, XA_MARK_0);
+		*cur = shapers[i];
+	}
+	xa_unlock(xa);
+}
+
 static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
 			u32 *handle)
 {

net/shaper/shaper.c static int parse_handle(const struct nlattr *attr, const struct genl_info *info, · line 295

Quoted 4 lines
net/shaper/shaper.c +1 -0
@@ -154,6 +292,68 @@ static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
 	return 0;
 }
 
+static int __parse_shaper(struct net_device *dev, struct nlattr **tb,

In general, "__" prefix indicates that reader should be careful for any reason, locking for example. Here, It's a simple helper, isn't it? Why "__" make sense here?

Quoted 69 lines
net/shaper/shaper.c +62 -0
+			  const struct genl_info *info,
+			  struct net_shaper_info *shaper)
+{
+	struct net_shaper_info *old;
+	int ret;
+
+	/* the shaper handle is the only mandatory attribute */
+	if (NL_REQ_ATTR_CHECK(info->extack, NULL, tb, NET_SHAPER_A_HANDLE))
+		return -EINVAL;
+
+	ret = parse_handle(tb[NET_SHAPER_A_HANDLE], info, &shaper->handle);
+	if (ret)
+		return ret;
+
+	/* fetch existing data, if any, so that user provide info will
+	 * incrementally update the existing shaper configuration
+	 */
+	old = sc_lookup(dev, shaper->handle);
+	if (old)
+		*shaper = *old;
+	else
+		shaper->parent = default_parent(shaper->handle);
+
+	if (tb[NET_SHAPER_A_METRIC])
+		shaper->metric = nla_get_u32(tb[NET_SHAPER_A_METRIC]);
+
+	if (tb[NET_SHAPER_A_BW_MIN])
+		shaper->bw_min = nla_get_uint(tb[NET_SHAPER_A_BW_MIN]);
+
+	if (tb[NET_SHAPER_A_BW_MAX])
+		shaper->bw_max = nla_get_uint(tb[NET_SHAPER_A_BW_MAX]);
+
+	if (tb[NET_SHAPER_A_BURST])
+		shaper->burst = nla_get_uint(tb[NET_SHAPER_A_BURST]);
+
+	if (tb[NET_SHAPER_A_PRIORITY])
+		shaper->priority = nla_get_u32(tb[NET_SHAPER_A_PRIORITY]);
+
+	if (tb[NET_SHAPER_A_WEIGHT])
+		shaper->weight = nla_get_u32(tb[NET_SHAPER_A_WEIGHT]);
+	return 0;
+}
+
+/* fetch the cached shaper info and update them with the user-provided
+ * attributes
+ */
+static int parse_shaper(struct net_device *dev, const struct nlattr *attr,
+			const struct genl_info *info,
+			struct net_shaper_info *shaper)
+{
+	struct nlattr *tb[NET_SHAPER_A_WEIGHT + 1];
+	int ret;
+
+	ret = nla_parse_nested(tb, NET_SHAPER_A_WEIGHT, attr,
+			       net_shaper_ns_info_nl_policy, info->extack);
+	if (ret < 0)
+		return ret;
+
+	return __parse_shaper(dev, tb, info, shaper);
+}
+
 int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
 {
 	struct net_shaper_info *shaper;
@@ -239,14 +439,133 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb,
 	return ret;
 }
 
+/* Update the H/W and on success update the local cache, too */

net/shaper/shaper.c int net_shaper_nl_get_dumpit(struct sk_buff *skb, · line 452

Quoted 10 lines
net/shaper/shaper.c +10 -0
+static int net_shaper_set(struct net_device *dev,
+			  const struct net_shaper_info *shaper,
+			  struct netlink_ext_ack *extack)
+{
+	enum net_shaper_scope scope;
+	u32 handle = shaper->handle;
+	int ret;
+
+	scope = net_shaper_handle_scope(handle);
+	if (scope == NET_SHAPER_SCOPE_PORT ||

NET_SHAPER_SCOPE_PORT is not really used in this set. Why do you have it? Same applies tol NET_SHAPER_SCOPE_NETDEV.

Quoted 7 lines
net/shaper/shaper.c +7 -0
+	    scope == NET_SHAPER_SCOPE_UNSPEC) {
+		NL_SET_ERR_MSG_FMT(extack, "Can't set shaper %x with scope %d",
+				   handle, scope);
+		return -EINVAL;
+	}
+	if (scope == NET_SHAPER_SCOPE_DETACHED && !sc_lookup(dev, handle)) {
+		NL_SET_ERR_MSG_FMT(extack, "Shaper %x with detached scope does not exist",

net/shaper/shaper.c int net_shaper_nl_get_dumpit(struct sk_buff *skb, · line 469

Quoted 10 lines
net/shaper/shaper.c +10 -0
+				   handle);
+		return -EINVAL;
+	}
+
+	ret = sc_prepare_insert(dev, &handle, extack);
+	if (ret)
+		return ret;
+
+	ret = dev->netdev_ops->net_shaper_ops->set(dev, shaper, extack);
+	sc_commit(dev, 1, shaper);

Missing rollback?

Quoted 14 lines
net/shaper/shaper.c +11 -1
+	return ret;
+}
+
 int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
 {
-	return -EOPNOTSUPP;
+	struct net_shaper_info shaper;
+	struct net_device *dev;
+	struct nlattr *attr;
+	int ret;
+
+	if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_SHAPER))
+		return -EINVAL;
+

net/shaper/shaper.c int net_shaper_nl_get_dumpit(struct sk_buff *skb, · line 492

Quoted 10 lines
net/shaper/shaper.c +10 -0
+	ret = fetch_dev(info, &dev);
+	if (ret)
+		return ret;
+
+	attr = info->attrs[NET_SHAPER_A_SHAPER];
+	ret = parse_shaper(dev, attr, info, &shaper);
+	if (ret)
+		goto put;
+
+	ret = net_shaper_set(dev, &shaper, info->extack);

Hmm, if I don't miss anything, this is lockless (.parallel_ops==true). What's stopping user from performing the set action in parallel on a single shaper? The locking scheme seems to be missing. Another thing to describe in documentation, perhaps.

Quoted 26 lines
net/shaper/shaper.c +26 -0
+
+put:
+	dev_put(dev);
+	return ret;
+}
+
+static int net_shaper_delete(struct net_device *dev, u32 handle,
+			     struct netlink_ext_ack *extack)
+{
+	struct net_shaper_info *parent, *shaper = sc_lookup(dev, handle);
+	struct xarray *xa = __sc_container(dev);
+	enum net_shaper_scope pscope;
+	u32 parent_handle;
+	int ret;
+
+	if (!xa || !shaper) {
+		NL_SET_ERR_MSG_FMT(extack, "Shaper %x not found", handle);
+		return -EINVAL;
+	}
+
+	if (is_detached(handle) && shaper->children > 0) {
+		NL_SET_ERR_MSG_FMT(extack, "Can't delete detached shaper %d with %d child nodes",
+				   handle, shaper->children);
+		return -EINVAL;
+	}
+

net/shaper/shaper.c int net_shaper_nl_get_dumpit(struct sk_buff *skb, · line 528

Quoted 10 lines
net/shaper/shaper.c +10 -0
+	while (shaper) {
+		parent_handle = shaper->parent;
+		pscope = net_shaper_handle_scope(parent_handle);
+
+		ret = dev->netdev_ops->net_shaper_ops->delete(dev, handle, extack);
+		if (ret < 0)
+			return ret;
+
+		xa_lock(xa);
+		__xa_erase(xa, handle);

Hmm, did you think about free of dev->net_shaper_data in case this is the last item? I think it would be correct to do that.

Quoted 47 lines
net/shaper/shaper.c +36 -1
+		if (is_detached(handle))
+			idr_remove(&dev->net_shaper_data->detached_ids,
+				   net_shaper_handle_id(handle));
+		xa_unlock(xa);
+		kfree(shaper);
+		shaper = NULL;
+
+		if (pscope == NET_SHAPER_SCOPE_DETACHED) {
+			parent = sc_lookup(dev, parent_handle);
+			if (parent && !--parent->children) {
+				shaper = parent;
+				handle = parent_handle;
+			}
+		}
+	}
+	return 0;
 }
 
 int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
 {
-	return -EOPNOTSUPP;
+	struct net_device *dev;
+	u32 handle;
+	int ret;
+
+	if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
+		return -EINVAL;
+
+	ret = fetch_dev(info, &dev);
+	if (ret)
+		return ret;
+
+	ret = parse_handle(info->attrs[NET_SHAPER_A_HANDLE], info, &handle);
+	if (ret)
+		goto put;
+
+	ret = net_shaper_delete(dev, handle, info->extack);
+
+put:
+	dev_put(dev);
+	return ret;
 }
 
 int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
Signature · 1 line
2.45.2