devlink: rate objects API
Subsystems (6)
v3 cover devlink: rate objects API
From: Dmytro Linkin <[email protected]>
Resending without RFC.
Currently kernel provides a way to change tx rate of single VF in switchdev mode via tc-police action. When lots of VFs are configured management of theirs rates becomes non-trivial task and some grouping mechanism is required. Implementing such grouping in tc-police will bring flow related limitations and unwanted complications, like:
- tc-police is a policer and there is a user request for a traffic shaper, so shared tc-police action is not suitable;
- flows requires net device to be placed on, means "groups" wouldn't have net device instance itself. Taking into the account previous point was reviewed a sollution, when representor have a policer and the driver use a shaper if qdisc contains group of VFs - such approach ugly, compilated and misleading;
- TC is ingress only, while configuring "other" side of the wire looks more like a "real" picture where shaping is outside of the steering world, similar to "ip link" command;
According to that devlink is the most appropriate place.
This series introduces devlink API for managing tx rate of single devlink port or of a group by invoking callbacks (see below) of corresponding driver. Also devlink port or a group can be added to the parent group, where driver responsible to handle rates of a group elements. To achieve all of that new rate object is added. It can be one of the two types:
- leaf - represents a single devlink port; created/destroyed by the driver and bound to the devlink port. As example, some driver may create leaf rate object for every devlink port associated with VF. Since leaf have 1to1 mapping to it's devlink port, in user space it is referred as pci/<bus_addr>/<port_index>;
- node - represents a group of rate objects; created/deleted by request from the userspace; initially empty (no rate objects added). In userspace it is referred as pci/<bus_addr>/<node_name>, where node name can be any, except decimal number, to avoid collisions with leafs.
devlink_ops extended with following callbacks:
- rate_{leaf|node}_tx_{share|max}_set
- rate_node_{new|del}
- rate_{leaf|node}_parent_set
KAPI provides:
- creation/destruction of the leaf rate object associated with devlink port
- destruction of rate nodes to allow a vendor driver to free allocated resources on driver removal or due to the other reasons when nodes destruction required
UAPI provides:
- dumping all or single rate objects
- setting tx_{share|max} of rate object of any type
- creating/deleting node rate object
- setting/unsetting parent of any rate object
Added devlink rate object support for netdevsim driver
Issues/open questions:
- Does user need DEVLINK_CMD_RATE_DEL_ALL_CHILD command to clean all children of particular parent node? For example:
$ devlink port function rate flush netdevsim/netdevsim10/group - priv pointer passed to the callbacks is a source of bugs; in leaf case driver can embed rate object into internal structure and use container_of() on it; in node case it cannot be done since nodes are created from userspace
v1->v2:
- fixed kernel-doc for devlink_rate_leaf_{create|destroy}()
- s/func/function/ for all devlink port command occurences
v2->v3:
- devlink:
- added devlink_rate_nodes_destroy() function
- netdevsim:
- added call of devlink_rate_nodes_destroy() function
Dmytro Linkin (18):
netdevsim: Add max_vfs to bus_dev
netdevsim: Disable VFs on nsim_dev_reload_destroy() call
netdevsim: Implement port types and indexing
netdevsim: Implement VFs
netdevsim: Implement legacy/switchdev mode for VFs
devlink: Introduce rate object
netdevsim: Register devlink rate leaf objects per VF
selftest: netdevsim: Add devlink rate test
devlink: Allow setting tx rate for devlink rate leaf objects
netdevsim: Implement devlink rate leafs tx rate support
selftest: netdevsim: Add devlink port shared/max tx rate test
devlink: Introduce rate nodes
netdevsim: Implement support for devlink rate nodes
selftest: netdevsim: Add devlink rate nodes test
devlink: Allow setting parent node of rate objects
netdevsim: Allow setting parent node of rate objects
selftest: netdevsim: Add devlink rate grouping test
Documentation: devlink rate objects
Documentation/networking/devlink/devlink-port.rst | 35 ++ Documentation/networking/devlink/netdevsim.rst | 26 + drivers/net/netdevsim/bus.c | 131 +++- drivers/net/netdevsim/dev.c | 396 ++++++++++++- drivers/net/netdevsim/netdev.c | 95 ++- drivers/net/netdevsim/netdevsim.h | 48 ++ include/net/devlink.h | 48 ++ include/uapi/linux/devlink.h | 17 + net/core/devlink.c | 660 ++++++++++++++++++++- .../selftests/drivers/net/netdevsim/devlink.sh | 167 +++++- 10 files changed, 1565 insertions(+), 58 deletions(-)
Signature · 1 line
1.8.3.1
PATCH RESEND v3 1/18 netdevsim: Add max_vfs to bus_dev
From: Dmytro Linkin <[email protected]>
Currently there is no limit to the number of VFs netdevsim can enable. In a real systems this value exist and used by the driver. Fore example, some features might need to consider this value when allocating memory.
Expose max_vfs variable to debugfs as configurable resource. If are VFs configured (num_vfs != 0) then changing of max_vfs not allowed.
--- drivers/net/netdevsim/bus.c | 99 +++++++++++++++++++++++++++++++++++---- drivers/net/netdevsim/dev.c | 13 +++++ drivers/net/netdevsim/netdevsim.h | 10 ++++ 3 files changed, 113 insertions(+), 9 deletions(-)
drivers/net/netdevsim/bus.c +90 -9
@@ -27,9 +27,9 @@ static struct nsim_bus_dev *to_nsim_bus_dev(struct device *dev)static int nsim_bus_dev_vfs_enable (struct nsim_bus_dev *nsim_bus_dev,unsigned int num_vfs) { - nsim_bus_dev->vfconfigs =kcalloc (num_vfs, -sizeof (struct nsim_vf_config ), -GFP_KERNEL | __GFP_NOWARN); +if (nsim_bus_dev->max_vfs < num_vfs) +return -ENOMEM ; +if (!nsim_bus_dev->vfconfigs)return -ENOMEM ; nsim_bus_dev->num_vfs = num_vfs;
@@ -39,8 +39,6 @@ static int nsim_bus_dev_vfs_enable(struct nsim_bus_dev *nsim_bus_dev,static void nsim_bus_dev_vfs_disable (struct nsim_bus_dev *nsim_bus_dev) { -kfree (nsim_bus_dev->vfconfigs); - nsim_bus_dev->vfconfigs =NULL ; nsim_bus_dev->num_vfs =0 ; }
@@ -56,7 +54,7 @@ static void nsim_bus_dev_vfs_disable(struct nsim_bus_dev *nsim_bus_dev)if (ret)return ret; -rtnl_lock (); +mutex_lock (&nsim_bus_dev->vfs_lock);if (nsim_bus_dev->num_vfs == num_vfs) goto exit_good;if (nsim_bus_dev->num_vfs && num_vfs) {
@@ -74,7 +72,7 @@ static void nsim_bus_dev_vfs_disable(struct nsim_bus_dev *nsim_bus_dev) exit_good: ret = count; exit_unlock: -rtnl_unlock (); +mutex_unlock (&nsim_bus_dev->vfs_lock);return ret; }
@@ -92,6 +90,73 @@ static void nsim_bus_dev_vfs_disable(struct nsim_bus_dev *nsim_bus_dev)__ATTR (sriov_numvfs,0664 , nsim_bus_dev_numvfs_show, nsim_bus_dev_numvfs_store); +ssize_t nsim_bus_dev_max_vfs_read (struct file *file, +char __user *data, +size_t count,loff_t *ppos) +{ +struct nsim_bus_dev *nsim_bus_dev = file->private_data; +char buf[11 ]; +size_t len; + + len =snprintf (buf,sizeof (buf),"%u\n" , nsim_bus_dev->max_vfs); +if (len <0 ) +return len; + +return simple_read_from_buffer (data, count, ppos, buf, len); +} + +ssize_t nsim_bus_dev_max_vfs_write (struct file *file, +const char __user *data, +size_t count,loff_t *ppos) +{ +struct nsim_bus_dev *nsim_bus_dev = file->private_data; +struct nsim_vf_config *vfconfigs; +ssize_t ret; +char buf[10 ]; +u32 val; + +if (*ppos !=0 ) +return 0 ; + +if (count >=sizeof (buf)) +return -ENOSPC ; + +mutex_lock (&nsim_bus_dev->vfs_lock); +/* Reject if VFs are configured */ +if (nsim_bus_dev->num_vfs) { + ret = -EBUSY ; + goto unlock; + } + + ret =copy_from_user (buf, data, count); +if (ret) { + ret = -EFAULT ; + goto unlock; + } + + buf[count] ='\0' ; + ret =kstrtouint (buf,10 , &val); +if (ret) { + ret = -EIO ; + goto unlock; + } + + vfconfigs =kcalloc (val,sizeof (struct nsim_vf_config ),GFP_KERNEL | __GFP_NOWARN); +if (!vfconfigs) { + ret = -ENOMEM ; + goto unlock; + } + +kfree (nsim_bus_dev->vfconfigs); + nsim_bus_dev->vfconfigs = vfconfigs; + nsim_bus_dev->max_vfs = val; + *ppos += count; + ret = count; +unlock: +mutex_unlock (&nsim_bus_dev->vfs_lock); +return ret; +} +static ssize_t new_port_store (struct device *dev,struct device_attribute *attr,const char *buf,size_t count)
@@ -311,6 +376,8 @@ static int nsim_num_vf(struct device *dev) .num_vf = nsim_num_vf, }; +#define NSIM_BUS_DEV_MAX_VFS 4 +static struct nsim_bus_dev *nsim_bus_dev_new (unsigned int id,unsigned int port_count) {
@@ -329,15 +396,28 @@ static int nsim_num_vf(struct device *dev) nsim_bus_dev->dev.type = &nsim_bus_dev_type; nsim_bus_dev->port_count = port_count; nsim_bus_dev->initial_net = current->nsproxy->net_ns; + nsim_bus_dev->max_vfs =NSIM_BUS_DEV_MAX_VFS ;mutex_init (&nsim_bus_dev->nsim_bus_reload_lock); +mutex_init (&nsim_bus_dev->vfs_lock);/* Disallow using nsim_bus_dev */ smp_store_release (&nsim_bus_dev->init, false); - err =device_register (&nsim_bus_dev->dev); -if (err) + nsim_bus_dev->vfconfigs =kcalloc (nsim_bus_dev->max_vfs, +sizeof (struct nsim_vf_config ), +GFP_KERNEL | __GFP_NOWARN); +if (!nsim_bus_dev->vfconfigs) { + err = -ENOMEM ; goto err_nsim_bus_dev_id_free; + } + + err =device_register (&nsim_bus_dev->dev); +if (err) + goto err_nsim_vfs_free; +return nsim_bus_dev; +err_nsim_vfs_free: +kfree (nsim_bus_dev->vfconfigs); err_nsim_bus_dev_id_free:ida_free (&nsim_bus_dev_ids, nsim_bus_dev->dev.id); err_nsim_bus_dev_free:
@@ -351,6 +431,7 @@ static void nsim_bus_dev_del(struct nsim_bus_dev *nsim_bus_dev)smp_store_release (&nsim_bus_dev->init, false);device_unregister (&nsim_bus_dev->dev);ida_free (&nsim_bus_dev_ids, nsim_bus_dev->dev.id); +kfree (nsim_bus_dev->vfconfigs);kfree (nsim_bus_dev); }
drivers/net/netdevsim/dev.c +13 -0
@@ -192,6 +192,14 @@ static ssize_t nsim_dev_trap_fa_cookie_write(struct file *file, .owner = THIS_MODULE, }; +static const struct file_operations nsim_dev_max_vfs_fops = { + .open = simple_open, + .read = nsim_bus_dev_max_vfs_read, + .write = nsim_bus_dev_max_vfs_write, + .llseek = generic_file_llseek, + .owner =THIS_MODULE , +}; +static int nsim_dev_debugfs_init (struct nsim_dev *nsim_dev) {char dev_ddir_name[sizeof (DRV_NAME ) +10 ];
@@ -231,6 +239,11 @@ static int nsim_dev_debugfs_init(struct nsim_dev *nsim_dev)debugfs_create_bool ("fail_trap_policer_counter_get" ,0600 , nsim_dev->ddir, &nsim_dev->fail_trap_policer_counter_get); + nsim_dev->max_vfs =debugfs_create_file ("max_vfs" , +0600 , + nsim_dev->ddir, + nsim_dev->nsim_bus_dev, + &nsim_dev_max_vfs_fops);nsim_udp_tunnels_debugfs_create (nsim_dev);return 0 ; }
drivers/net/netdevsim/netdevsim.h +10 -0
@@ -212,6 +212,7 @@ struct nsim_dev {struct dentry *ddir;struct dentry *ports_ddir;struct dentry *take_snapshot; +struct dentry *max_vfs;struct bpf_offload_dev *bpf_dev;bool bpf_bind_accept;bool bpf_bind_verifier_accept;
@@ -269,6 +270,13 @@ struct nsim_fib_data *nsim_fib_create(struct devlink *devlink,u64 nsim_fib_get_val (struct nsim_fib_data *fib_data,enum nsim_resource_id res_id,bool max); +ssize_t nsim_bus_dev_max_vfs_read (struct file *file, +char __user *data, +size_t count,loff_t *ppos); +ssize_t nsim_bus_dev_max_vfs_write (struct file *file, +const char __user *data, +size_t count,loff_t *ppos); +#if IS_ENABLED (CONFIG_XFRM_OFFLOAD )void nsim_ipsec_init (struct netdevsim *ns);void nsim_ipsec_teardown (struct netdevsim *ns);
@@ -308,7 +316,9 @@ struct nsim_bus_dev {struct net *initial_net;/* Purpose of this is to carry net pointer * during the probe time only. */ +unsigned int max_vfs;unsigned int num_vfs; +struct mutex vfs_lock;/* Protects vfconfigs */ struct nsim_vf_config *vfconfigs;/* Lock for devlink->reload_enabled in netdevsim module */ struct mutex nsim_bus_reload_lock;
Signature · 1 line
1.8.3.1
PATCH RESEND v3 2/18 netdevsim: Disable VFs on nsim_dev_reload_destroy() call
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Move VFs disabling from device release() to nsim_dev_reload_destroy() to make VFs disabling and ports removal simultaneous.
This is a requirement for VFs ports implemented in next patches.
--- drivers/net/netdevsim/bus.c | 5 +---- drivers/net/netdevsim/dev.c | 6 ++++++ drivers/net/netdevsim/netdevsim.h | 1 + 3 files changed, 8 insertions(+), 4 deletions(-)
drivers/net/netdevsim/bus.c +1 -4
@@ -37,7 +37,7 @@ static int nsim_bus_dev_vfs_enable(struct nsim_bus_dev *nsim_bus_dev,return 0 ; } -static void nsim_bus_dev_vfs_disable (struct nsim_bus_dev *nsim_bus_dev) +void nsim_bus_dev_vfs_disable (struct nsim_bus_dev *nsim_bus_dev) { nsim_bus_dev->num_vfs =0 ; }
@@ -233,9 +233,6 @@ ssize_t nsim_bus_dev_max_vfs_write(struct file *file,static void nsim_bus_dev_release (struct device *dev) { -struct nsim_bus_dev *nsim_bus_dev =to_nsim_bus_dev (dev); - -nsim_bus_dev_vfs_disable (nsim_bus_dev); }static struct device_type nsim_bus_dev_type = {
drivers/net/netdevsim/dev.c +6 -0
@@ -1182,6 +1182,12 @@ static void nsim_dev_reload_destroy(struct nsim_dev *nsim_dev)if (devlink_is_reload_failed (devlink))return ;debugfs_remove (nsim_dev->take_snapshot); + +mutex_lock (&nsim_dev->nsim_bus_dev->vfs_lock); +if (nsim_dev->nsim_bus_dev->num_vfs) +nsim_bus_dev_vfs_disable (nsim_dev->nsim_bus_dev); +mutex_unlock (&nsim_dev->nsim_bus_dev->vfs_lock); +nsim_dev_port_del_all (nsim_dev);nsim_dev_psample_exit (nsim_dev);nsim_dev_health_exit (nsim_dev);
drivers/net/netdevsim/netdevsim.h +1 -0
@@ -276,6 +276,7 @@ ssize_t nsim_bus_dev_max_vfs_read(struct file *file,ssize_t nsim_bus_dev_max_vfs_write (struct file *file,const char __user *data,size_t count,loff_t *ppos); +void nsim_bus_dev_vfs_disable (struct nsim_bus_dev *nsim_bus_dev);#if IS_ENABLED (CONFIG_XFRM_OFFLOAD )void nsim_ipsec_init (struct netdevsim *ns);
Signature · 1 line
1.8.3.1
PATCH RESEND v3 3/18 netdevsim: Implement port types and indexing
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Define type of ports, which netdevsim driver currently operates with as PF. Define new port type - VF, which will be implemented in following patches. Add helper functions to distinguish them. Add helper function to get VF index from port index.
Add port indexing logic where PFs' indexes starts from 0, VFs' - from NSIM_DEV_VF_PORT_INDEX_BASE.
All ports uses same index pool, which means that PF port may be created with index from VFs' indexes range.
Maximum number of VFs, which the driver can allocate, is limited by UINT_MAX - BASE.
--- drivers/net/netdevsim/bus.c | 10 ++++++++-- drivers/net/netdevsim/dev.c | 42 +++++++++++++++++++++++++++++---------- drivers/net/netdevsim/netdevsim.h | 20 +++++++++++++++++++ 3 files changed, 60 insertions(+), 12 deletions(-)
drivers/net/netdevsim/bus.c +8 -2
@@ -141,6 +141,12 @@ ssize_t nsim_bus_dev_max_vfs_write(struct file *file, goto unlock; } +/* max_vfs limited by the maximum number of provided port indexes */ +if (val >NSIM_DEV_VF_PORT_INDEX_MAX -NSIM_DEV_VF_PORT_INDEX_BASE ) { + ret = -ERANGE ; + goto unlock; + } + vfconfigs =kcalloc (val,sizeof (struct nsim_vf_config ),GFP_KERNEL | __GFP_NOWARN);if (!vfconfigs) { ret = -ENOMEM ;
@@ -178,7 +184,7 @@ ssize_t nsim_bus_dev_max_vfs_write(struct file *file,mutex_lock (&nsim_bus_dev->nsim_bus_reload_lock);devlink_reload_disable (devlink); - ret =nsim_dev_port_add (nsim_bus_dev, port_index); + ret =nsim_dev_port_add (nsim_bus_dev,NSIM_DEV_PORT_TYPE_PF , port_index);devlink_reload_enable (devlink);mutex_unlock (&nsim_bus_dev->nsim_bus_reload_lock);return ret ? ret : count;
@@ -207,7 +213,7 @@ ssize_t nsim_bus_dev_max_vfs_write(struct file *file,mutex_lock (&nsim_bus_dev->nsim_bus_reload_lock);devlink_reload_disable (devlink); - ret =nsim_dev_port_del (nsim_bus_dev, port_index); + ret =nsim_dev_port_del (nsim_bus_dev,NSIM_DEV_PORT_TYPE_PF , port_index);devlink_reload_enable (devlink);mutex_unlock (&nsim_bus_dev->nsim_bus_reload_lock);return ret ? ret : count;
drivers/net/netdevsim/dev.c +32 -10
105 lines not on this page · raw message
drivers/net/netdevsim/netdevsim.h +20 -0
@@ -197,10 +197,19 @@ static inline void nsim_dev_psample_exit(struct nsim_dev *nsim_dev) }#endif +enum nsim_dev_port_type { +NSIM_DEV_PORT_TYPE_PF , +NSIM_DEV_PORT_TYPE_VF , +}; + +#define NSIM_DEV_VF_PORT_INDEX_BASE 128 +#define NSIM_DEV_VF_PORT_INDEX_MAX UINT_MAX +struct nsim_dev_port {struct list_head list;struct devlink_port devlink_port;unsigned int port_index; +enum nsim_dev_port_type port_type;struct dentry *ddir;struct netdevsim *ns; };
@@ -260,8 +269,10 @@ static inline struct net *nsim_dev_net(struct nsim_dev *nsim_dev)int nsim_dev_probe (struct nsim_bus_dev *nsim_bus_dev);void nsim_dev_remove (struct nsim_bus_dev *nsim_bus_dev);int nsim_dev_port_add (struct nsim_bus_dev *nsim_bus_dev, +enum nsim_dev_port_type type,unsigned int port_index);int nsim_dev_port_del (struct nsim_bus_dev *nsim_bus_dev, +enum nsim_dev_port_type type,unsigned int port_index); struct nsim_fib_data *nsim_fib_create(struct devlink *devlink,
@@ -278,6 +289,15 @@ ssize_t nsim_bus_dev_max_vfs_write(struct file *file, size_t count, loff_t *ppos);void nsim_bus_dev_vfs_disable (struct nsim_bus_dev *nsim_bus_dev); +static inline bool nsim_dev_port_is_pf (struct nsim_dev_port *nsim_dev_port) +{ +return nsim_dev_port->port_type ==NSIM_DEV_PORT_TYPE_PF ; +} + +static inline bool nsim_dev_port_is_vf (struct nsim_dev_port *nsim_dev_port) +{ +return nsim_dev_port->port_type ==NSIM_DEV_PORT_TYPE_VF ; +}#if IS_ENABLED (CONFIG_XFRM_OFFLOAD )void nsim_ipsec_init (struct netdevsim *ns);void nsim_ipsec_teardown (struct netdevsim *ns);
Signature · 1 line
1.8.3.1
PATCH RESEND v3 4/18 netdevsim: Implement VFs
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Allow creation of netdevsim ports for VFs along with allocations of corresponding net devices and devlink ports.
Add enums and helpers to distinguish PFs' ports from VFs' ports.
Ports creation/deletion debugfs API intended to be used with physical ports only.
VFs instantiation will be done in one of the next patches.
--- drivers/net/netdevsim/dev.c | 14 ++++++- drivers/net/netdevsim/netdev.c | 90 ++++++++++++++++++++++++++++++------------ 2 files changed, 77 insertions(+), 27 deletions(-)
drivers/net/netdevsim/dev.c +12 -2
@@ -945,11 +945,15 @@ static int nsim_dev_devlink_trap_init(struct devlink *devlink,static int __nsim_dev_port_add (struct nsim_dev *nsim_dev,enum nsim_dev_port_type type,unsigned int port_index) { +struct nsim_bus_dev *nsim_bus_dev = nsim_dev->nsim_bus_dev;struct devlink_port_attrs attrs = {};struct nsim_dev_port *nsim_dev_port;struct devlink_port *devlink_port;int err; +if (type ==NSIM_DEV_PORT_TYPE_VF && !nsim_bus_dev->num_vfs) +return -EINVAL ; + nsim_dev_port =kzalloc (sizeof (*nsim_dev_port),GFP_KERNEL );if (!nsim_dev_port)return -ENOMEM ;
@@ -957,8 +961,14 @@ static int __nsim_dev_port_add(struct nsim_dev *nsim_dev, enum nsim_dev_port_typ nsim_dev_port->port_type = type; devlink_port = &nsim_dev_port->devlink_port; - attrs.flavour =DEVLINK_PORT_FLAVOUR_PHYSICAL ; - attrs.phys.port_number = port_index +1 ; +if (nsim_dev_port_is_pf (nsim_dev_port)) { + attrs.flavour =DEVLINK_PORT_FLAVOUR_PHYSICAL ; + attrs.phys.port_number = port_index +1 ; + }else { + attrs.flavour =DEVLINK_PORT_FLAVOUR_PCI_VF ; + attrs.pci_vf.pf =0 ; + attrs.pci_vf.vf = port_index; + }memcpy (attrs.switch_id.id, nsim_dev->switch_id.id, nsim_dev->switch_id.id_len); attrs.switch_id.id_len = nsim_dev->switch_id.id_len;devlink_port_attrs_set (devlink_port, &attrs);
drivers/net/netdevsim/netdev.c +65 -25
120 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH RESEND v3 5/18 netdevsim: Implement legacy/switchdev mode for VFs
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Implement callbacks to set/get eswitch mode value. Add helpers to check current mode.
Instantiate VFs' net devices and devlink ports on switchdev enabling and remove them on legacy enabling. Changing number of VFs while in switchdev mode triggers VFs creation/deletion.
Also disable NDO API callback to set VF rate, since it's legacy API. Switchdev API to set VF rate will be implemented in one of the next patches.
--- drivers/net/netdevsim/bus.c | 17 +++++++++- drivers/net/netdevsim/dev.c | 69 +++++++++++++++++++++++++++++++++++++++ drivers/net/netdevsim/netdev.c | 5 +++ drivers/net/netdevsim/netdevsim.h | 14 ++++++++ 4 files changed, 104 insertions(+), 1 deletion(-)
drivers/net/netdevsim/bus.c +16 -1
@@ -27,6 +27,9 @@ static struct nsim_bus_dev *to_nsim_bus_dev(struct device *dev)static int nsim_bus_dev_vfs_enable (struct nsim_bus_dev *nsim_bus_dev,unsigned int num_vfs) { +struct nsim_dev *nsim_dev; +int err =0 ; +if (nsim_bus_dev->max_vfs < num_vfs)return -ENOMEM ;
@@ -34,12 +37,24 @@ static int nsim_bus_dev_vfs_enable(struct nsim_bus_dev *nsim_bus_dev,return -ENOMEM ; nsim_bus_dev->num_vfs = num_vfs; -return 0 ; + nsim_dev =dev_get_drvdata (&nsim_bus_dev->dev); +if (nsim_esw_mode_is_switchdev (nsim_dev)) { + err =nsim_esw_switchdev_enable (nsim_dev,NULL ); +if (err) + nsim_bus_dev->num_vfs =0 ; + } + +return err; }void nsim_bus_dev_vfs_disable (struct nsim_bus_dev *nsim_bus_dev) { +struct nsim_dev *nsim_dev; + nsim_bus_dev->num_vfs =0 ; + nsim_dev =dev_get_drvdata (&nsim_bus_dev->dev); +if (nsim_esw_mode_is_switchdev (nsim_dev)) +nsim_esw_legacy_enable (nsim_dev,NULL ); } static ssize_t
drivers/net/netdevsim/dev.c +69 -0
87 lines not on this page · raw message
drivers/net/netdevsim/netdev.c +5 -0
@@ -113,6 +113,11 @@ static int nsim_set_vf_rate(struct net_device *dev, int vf, int min, int max)struct netdevsim *ns =netdev_priv (dev);struct nsim_bus_dev *nsim_bus_dev = ns->nsim_bus_dev; +if (nsim_esw_mode_is_switchdev (ns->nsim_dev)) { +pr_err ("Not supported in switchdev mode. Please use devlink API.\n" ); +return -EOPNOTSUPP ; + } +if (vf >= nsim_bus_dev->num_vfs)return -EINVAL ;
drivers/net/netdevsim/netdevsim.h +14 -0
@@ -257,8 +257,22 @@ struct nsim_dev {u32 sleep; } udp_ports;struct nsim_dev_psample *psample; +u16 esw_mode; }; +int nsim_esw_legacy_enable (struct nsim_dev *nsim_dev,struct netlink_ext_ack *extack); +int nsim_esw_switchdev_enable (struct nsim_dev *nsim_dev,struct netlink_ext_ack *extack); + +static inline bool nsim_esw_mode_is_legacy (struct nsim_dev *nsim_dev) +{ +return nsim_dev->esw_mode ==DEVLINK_ESWITCH_MODE_LEGACY ; +} + +static inline bool nsim_esw_mode_is_switchdev (struct nsim_dev *nsim_dev) +{ +return nsim_dev->esw_mode ==DEVLINK_ESWITCH_MODE_SWITCHDEV ; +} +static inline struct net *nsim_dev_net (struct nsim_dev *nsim_dev) {return devlink_net (priv_to_devlink (nsim_dev));
Signature · 1 line
1.8.3.1
PATCH RESEND v3 6/18 devlink: Introduce rate object
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Allow registering rate object for devlink ports with dedicated devlink_rate_leaf_{create|destroy}() API. Implement new netlink DEVLINK_CMD_RATE_GET command that is used to retrieve rate object info. Add new DEVLINK_CMD_RATE_{NEW|DEL} commands that are used for notifications when creating/deleting leaf rate object.
Rate API is intended to be used for rate limiting of individual devlink ports (leafs) and their aggregates (nodes).
Example:
$ devlink port show pci/0000:03:00.0/0 pci/0000:03:00.0/1
$ devlink port function rate show pci/0000:03:00.0/0: type leaf pci/0000:03:00.0/1: type leaf
---
Notes:
v1->v2:
- s/func/function/ in commit message
- fixes kernel-doc for devlink_rate_leaf_{create|destroy}()
include/net/devlink.h | 14 +++ include/uapi/linux/devlink.h | 11 +++ net/core/devlink.c | 229 ++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 253 insertions(+), 1 deletion(-)
include/net/devlink.h +14 -0
@@ -34,6 +34,7 @@ struct devlink_dev_stats {struct devlink {struct list_head list;struct list_head port_list; +struct list_head rate_list;struct list_head sb_list;struct list_head dpipe_table_list;struct list_head resource_list;
@@ -133,6 +134,15 @@ struct devlink_port_attrs { }; }; +struct devlink_rate { +struct list_head list; +enum devlink_rate_type type; +struct devlink *devlink; +void *priv; + +struct devlink_port *devlink_port; +}; +struct devlink_port {struct list_head list;struct list_head param_list;
@@ -152,6 +162,8 @@ struct devlink_port {struct delayed_work type_warn_dw;struct list_head reporter_list;struct mutex reporters_lock;/* Protects reporter_list */ + +struct devlink_rate *devlink_rate; };struct devlink_port_new_attrs {
@@ -1512,6 +1524,8 @@ void devlink_port_attrs_pci_vf_set(struct devlink_port *devlink_port, u32 controvoid devlink_port_attrs_pci_sf_set (struct devlink_port *devlink_port,u32 controller,u16 pf,u32 sf,bool external); +int devlink_rate_leaf_create (struct devlink_port *port,void *priv); +void devlink_rate_leaf_destroy (struct devlink_port *devlink_port); int devlink_sb_register(struct devlink *devlink, unsigned int sb_index, u32 size, u16 ingress_pools_count, u16 egress_pools_count, u16 ingress_tc_count,
include/uapi/linux/devlink.h +11 -0
@@ -126,6 +126,11 @@ enum devlink_command {DEVLINK_CMD_HEALTH_REPORTER_TEST , +DEVLINK_CMD_RATE_GET ,/* can dump */ +DEVLINK_CMD_RATE_SET , +DEVLINK_CMD_RATE_NEW , +DEVLINK_CMD_RATE_DEL , +/* add new commands above here */ __DEVLINK_CMD_MAX,DEVLINK_CMD_MAX = __DEVLINK_CMD_MAX -1
@@ -206,6 +211,10 @@ enum devlink_port_flavour {*/ }; +enum devlink_rate_type { +DEVLINK_RATE_TYPE_LEAF , +}; +enum devlink_param_cmode {DEVLINK_PARAM_CMODE_RUNTIME ,DEVLINK_PARAM_CMODE_DRIVERINIT ,
@@ -534,6 +543,8 @@ enum devlink_attr {DEVLINK_ATTR_RELOAD_ACTION_STATS ,/* nested */ DEVLINK_ATTR_PORT_PCI_SF_NUMBER ,/* u32 */ + +DEVLINK_ATTR_RATE_TYPE ,/* u16 */ /* add new attributes above here, update the policy in devlink.c */ __DEVLINK_ATTR_MAX,
net/core/devlink.c +228 -1
294 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH RESEND v3 7/18 netdevsim: Register devlink rate leaf objects per VF
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Register devlink rate leaf objects per VF.
--- drivers/net/netdevsim/dev.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
drivers/net/netdevsim/dev.c +11 -0
@@ -1055,11 +1055,20 @@ static int __nsim_dev_port_add(struct nsim_dev *nsim_dev, enum nsim_dev_port_typ goto err_port_debugfs_exit; } +if (nsim_dev_port_is_vf (nsim_dev_port)) { + err =devlink_rate_leaf_create (&nsim_dev_port->devlink_port, + nsim_dev_port); +if (err) + goto err_nsim_destroy; + } +devlink_port_type_eth_set (devlink_port, nsim_dev_port->ns->netdev);list_add (&nsim_dev_port->list, &nsim_dev->port_list);return 0 ; +err_nsim_destroy: +nsim_destroy (nsim_dev_port->ns); err_port_debugfs_exit:nsim_dev_port_debugfs_exit (nsim_dev_port); err_dl_port_unregister:
@@ -1074,6 +1083,8 @@ static void __nsim_dev_port_del(struct nsim_dev_port *nsim_dev_port)struct devlink_port *devlink_port = &nsim_dev_port->devlink_port;list_del (&nsim_dev_port->list); +if (nsim_dev_port_is_vf (nsim_dev_port)) +devlink_rate_leaf_destroy (&nsim_dev_port->devlink_port);devlink_port_type_clear (devlink_port);nsim_destroy (nsim_dev_port->ns);nsim_dev_port_debugfs_exit (nsim_dev_port);
Signature · 1 line
1.8.3.1
PATCH RESEND v3 8/18 selftest: netdevsim: Add devlink rate test
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Test verifies that all netdevsim VF ports have rate leaf object created by default.
---
Notes:
v1->v2:
- s/func/function in devlink command
.../selftests/drivers/net/netdevsim/devlink.sh | 25 +++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-)
tools/testing/selftests/drivers/net/netdevsim/devlink.sh +24 -1
@@ -5,12 +5,13 @@ lib_dir=$(dirname $0)/../../../net/forwarding ALL_TESTS="fw_flash_test params_test regions_test reload_test \ netns_reload_test resource_test dev_info_test \ - empty_reporter_test dummy_reporter_test" + empty_reporter_test dummy_reporter_test rate_test" NUM_NETIFS=0 source $lib_dir/lib.sh BUS_ADDR=10 PORT_COUNT=4 +VF_COUNT=4 DEV_NAME=netdevsim$BUS_ADDR SYSFS_NET_DIR=/sys/bus/netdevsim/devices/$DEV_NAME/net/ DEBUGFS_DIR=/sys/kernel/debug/netdevsim/$DEV_NAME/
@@ -507,6 +508,28 @@ dummy_reporter_test() log_test "dummy reporter test" } +rate_leafs_get() +{ + local handle=$1 + + cmd_jq "devlink port function rate show -j" \ + '.[] | to_entries | .[] | select(.value.type == "leaf") | .key | select(contains("'$handle'"))' +} + +rate_test() +{ + RET=0 + + echo $VF_COUNT > /sys/bus/netdevsim/devices/$DEV_NAME/sriov_numvfs + devlink dev eswitch set $DL_HANDLE mode switchdev + local leafs=`rate_leafs_get $DL_HANDLE` + local num_leafs=`echo $leafs | wc -w` + [ "$num_leafs" == "$VF_COUNT" ] + check_err $? "Expected $VF_COUNT rate leafs but got $num_leafs" + + log_test "rate test" +} + setup_prepare() { modprobe netdevsim
Signature · 1 line
1.8.3.1
PATCH RESEND v3 9/18 devlink: Allow setting tx rate for devlink rate leaf objects
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Implement support for DEVLINK_CMD_RATE_SET command with new attributes DEVLINK_ATTR_RATE_TX_{SHARE|MAX} that are used to set devlink rate shared/max tx rate values. Extend devlink ops with new callbacks rate_leaf_tx_{share|max}_set() to allow supporting drivers to implement rate control through devlink.
New attributes are optional. Driver implementations are allowed to support either or both of them.
Shared rate example:
$ devlink port function rate set netdevsim/netdevsim10/0 tx_share 10mbit
$ devlink port function rate show netdevsim/netdevsim10/0 netdevsim/netdevsim10/0: type leaf tx_share 10mbit
Max rate example:
$ devlink port function rate set netdevsim/netdevsim10/0 tx_max 100mbit
$ devlink port function rate show netdevsim/netdevsim10/0 netdevsim/netdevsim10/0: type leaf tx_max 100mbit
---
Notes:
v1->v2:
- s/func/function/ in devlink commands of commit message
include/net/devlink.h | 10 ++++++ include/uapi/linux/devlink.h | 2 ++ net/core/devlink.c | 86 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+)
include/net/devlink.h +10 -0
@@ -139,6 +139,8 @@ struct devlink_rate {enum devlink_rate_type type;struct devlink *devlink;void *priv; +u64 tx_share; +u64 tx_max;struct devlink_port *devlink_port; };
@@ -1465,6 +1467,14 @@ struct devlink_ops {struct devlink_port *port, enum devlink_port_fn_state state, struct netlink_ext_ack *extack); + +/** +* Rate control callbacks. +*/ +int (*rate_leaf_tx_share_set)(struct devlink_rate *devlink_rate,void *priv, +u64 tx_share,struct netlink_ext_ack *extack); +int (*rate_leaf_tx_max_set)(struct devlink_rate *devlink_rate,void *priv, +u64 tx_max,struct netlink_ext_ack *extack); };static inline void *devlink_priv (struct devlink *devlink)
include/uapi/linux/devlink.h +2 -0
@@ -545,6 +545,8 @@ enum devlink_attr {DEVLINK_ATTR_PORT_PCI_SF_NUMBER ,/* u32 */ DEVLINK_ATTR_RATE_TYPE ,/* u16 */ +DEVLINK_ATTR_RATE_TX_SHARE ,/* u64 */ +DEVLINK_ATTR_RATE_TX_MAX ,/* u64 */ /* add new attributes above here, update the policy in devlink.c */ __DEVLINK_ATTR_MAX,
net/core/devlink.c +86 -0
110 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH RESEND v3 10/18 netdevsim: Implement devlink rate leafs tx rate support
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Implement new devlink ops that allow shared and max tx rate control for devlink port rate objects (leafs) through devlink API.
Expose rate values of VF ports to netdevsim debugfs.
--- drivers/net/netdevsim/dev.c | 78 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 3 deletions(-)
drivers/net/netdevsim/dev.c +75 -3
@@ -276,17 +276,26 @@ static void nsim_dev_debugfs_exit(struct nsim_dev *nsim_dev)static int nsim_dev_port_debugfs_init (struct nsim_dev *nsim_dev,struct nsim_dev_port *nsim_dev_port) { +struct nsim_bus_dev *nsim_bus_dev = nsim_dev->nsim_bus_dev; +unsigned int port_index = nsim_dev_port->port_index;char port_ddir_name[16 ];char dev_link_name[32 ]; -sprintf (port_ddir_name,"%u" , nsim_dev_port->port_index); +sprintf (port_ddir_name,"%u" , port_index); nsim_dev_port->ddir =debugfs_create_dir (port_ddir_name, nsim_dev->ports_ddir);if (IS_ERR (nsim_dev_port->ddir))return PTR_ERR (nsim_dev_port->ddir); -sprintf (dev_link_name,"../../../" DRV_NAME "%u" , - nsim_dev->nsim_bus_dev->dev.id); +sprintf (dev_link_name,"../../../" DRV_NAME "%u" , nsim_bus_dev->dev.id); +if (nsim_dev_port_is_vf (nsim_dev_port)) { +unsigned int vf_id =nsim_dev_port_index_to_vf_index (port_index); + +debugfs_create_u16 ("tx_share" ,0400 , nsim_dev_port->ddir, + &nsim_bus_dev->vfconfigs[vf_id].min_tx_rate); +debugfs_create_u16 ("tx_max" ,0400 , nsim_dev_port->ddir, + &nsim_bus_dev->vfconfigs[vf_id].max_tx_rate); + }debugfs_create_symlink ("dev" , nsim_dev_port->ddir, dev_link_name);return 0 ;
@@ -990,6 +999,67 @@ static int nsim_dev_devlink_trap_init(struct devlink *devlink,return 0 ; } +#define NSIM_LINK_SPEED_MAX 5000/* Mbps */ +#define NSIM_LINK_SPEED_UNIT 125000/* 1 Mbps given in bytes/sec to avoid +* u64 overflow during conversion from +* bytes to bits. +*/ + +static int nsim_rate_bytes_to_units (char *name,u64 *rate,struct netlink_ext_ack *extack) +{ +u64 val; +u32 rem; + + val =div_u64_rem (*rate,NSIM_LINK_SPEED_UNIT , &rem); +if (rem) { +pr_err ("%s rate value %lluBps not in link speed units of 1Mbps.\n" , + name, *rate); +NL_SET_ERR_MSG_MOD (extack,"TX rate value not in link speed units of 1Mbps." ); +return -EINVAL ; + } + +if (val >NSIM_LINK_SPEED_MAX ) { +pr_err ("%s rate value %lluMbps exceed link maximum speed 5000Mbps.\n" , + name, val); +NL_SET_ERR_MSG_MOD (extack,"TX rate value exceed link maximum speed 5000Mbps." ); +return -EINVAL ; + } + *rate = val; +return 0 ; +} + +static int nsim_leaf_tx_share_set (struct devlink_rate *devlink_rate,void *priv, +u64 tx_share,struct netlink_ext_ack *extack) +{ +struct nsim_dev_port *nsim_dev_port = priv; +struct nsim_bus_dev *nsim_bus_dev = nsim_dev_port->ns->nsim_bus_dev; +int vf_id =nsim_dev_port_index_to_vf_index (nsim_dev_port->port_index); +int err; + + err =nsim_rate_bytes_to_units ("tx_share" , &tx_share, extack); +if (err) +return err; + + nsim_bus_dev->vfconfigs[vf_id].min_tx_rate = tx_share; +return 0 ; +} + +static int nsim_leaf_tx_max_set (struct devlink_rate *devlink_rate,void *priv, +u64 tx_max,struct netlink_ext_ack *extack) +{ +struct nsim_dev_port *nsim_dev_port = priv; +struct nsim_bus_dev *nsim_bus_dev = nsim_dev_port->ns->nsim_bus_dev; +int vf_id =nsim_dev_port_index_to_vf_index (nsim_dev_port->port_index); +int err; + + err =nsim_rate_bytes_to_units ("tx_max" , &tx_max, extack); +if (err) +return err; + + nsim_bus_dev->vfconfigs[vf_id].max_tx_rate = tx_max; +return 0 ; +} +static const struct devlink_ops nsim_dev_devlink_ops = { .eswitch_mode_set = nsim_devlink_eswitch_mode_set, .eswitch_mode_get = nsim_devlink_eswitch_mode_get,
@@ -1005,6 +1075,8 @@ static int nsim_dev_devlink_trap_init(struct devlink *devlink, .trap_group_set = nsim_dev_devlink_trap_group_set, .trap_policer_set = nsim_dev_devlink_trap_policer_set, .trap_policer_counter_get = nsim_dev_devlink_trap_policer_counter_get, + .rate_leaf_tx_share_set = nsim_leaf_tx_share_set, + .rate_leaf_tx_max_set = nsim_leaf_tx_max_set, };#define NSIM_DEV_MAX_MACS_DEFAULT 32
Signature · 1 line
1.8.3.1
PATCH RESEND v3 11/18 selftest: netdevsim: Add devlink port shared/max tx rate test
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Test verifies that netdevsim VFs can set and retrieve shared/max tx rate through new devlink API.
---
Notes:
v1->v2:
- s/func/function/ in devlink commands
.../selftests/drivers/net/netdevsim/devlink.sh | 55 ++++++++++++++++++++++ 1 file changed, 55 insertions(+)
tools/testing/selftests/drivers/net/netdevsim/devlink.sh +55 -0
@@ -516,6 +516,45 @@ rate_leafs_get() '.[] | to_entries | .[] | select(.value.type == "leaf") | .key | select(contains("'$handle'"))' } +rate_attr_set() +{ + local handle=$1 + local name=$2 + local value=$3 + local units=$4 + + devlink port function rate set $handle $name $value$units +} + +rate_attr_get() +{ + local handle=$1 + local name=$2 + + cmd_jq "devlink port function rate show $handle -j" '.[][].'$name +} + +rate_attr_tx_rate_check() +{ + local handle=$1 + local name=$2 + local rate=$3 + local debug_file=$4 + + rate_attr_set $handle $name $rate mbit + check_err $? "Failed to set $name value" + + local debug_value=$(cat $debug_file) + check_err $? "Failed to read $name value from debugfs" + [ "$debug_value" == "$rate" ] + check_err $? "Unexpected $name debug value $debug_value != $rate" + + local api_value=$(( $(rate_attr_get $handle $name) * 8 / 1000000 )) + check_err $? "Failed to get $name attr value" + [ "$api_value" == "$rate" ] + check_err $? "Unexpected $name attr value $api_value != $rate" +} + rate_test() { RET=0
@@ -527,6 +566,22 @@ rate_test() [ "$num_leafs" == "$VF_COUNT" ] check_err $? "Expected $VF_COUNT rate leafs but got $num_leafs" + rate=10 + for r_obj in $leafs + do + rate_attr_tx_rate_check $r_obj tx_share $rate \ + $DEBUGFS_DIR/ports/${r_obj##*/}/tx_share + rate=$(($rate+10)) + done + + rate=100 + for r_obj in $leafs + do + rate_attr_tx_rate_check $r_obj tx_max $rate \ + $DEBUGFS_DIR/ports/${r_obj##*/}/tx_max + rate=$(($rate+100)) + done + log_test "rate test" }
Signature · 1 line
1.8.3.1
PATCH RESEND v3 12/18 devlink: Introduce rate nodes
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Implement support for DEVLINK_CMD_RATE_{NEW|DEL} commands that are used to create and delete devlink rate nodes. Add new attribute DEVLINK_ATTR_RATE_NODE_NAME that specify node name string. The node name is an alphanumeric identifier. No valid node name can be a devlink port index, eg. decimal number. Extend devlink ops with new callbacks rate_node_{new|del}() and rate_node_tx_{share|max}_set() to allow supporting drivers to implement ports rate grouping and setting tx rate of rate nodes through devlink.
Expose devlink_rate_nodes_destroy() function to allow vendor driver do proper cleanup of internally allocated resources for the nodes if the driver goes down or due to any other reasons which requires nodes to be destroyed.
Disallow moving device from switchdev to legacy mode if any node exists on that device. User must explicitly delete nodes before switching mode.
Example:
$ devlink port function rate add netdevsim/netdevsim10/group1
$ devlink port function rate set netdevsim/netdevsim10/group1 \
tx_share 10mbit tx_max 100mbitAdd + set command can be combined:
$ devlink port function rate add netdevsim/netdevsim10/group1 \
tx_share 10mbit tx_max 100mbit$ devlink port function rate show netdevsim/netdevsim10/group1 netdevsim/netdevsim10/group1: type node tx_share 10mbit tx_max 100mbit
$ devlink port function rate del netdevsim/netdevsim10/group1
---
Notes:
v1->v2:
- s/func/function/ at commit message
v2->v3:
- added devlink_rate_nodes_destroy()
include/net/devlink.h | 14 ++- include/uapi/linux/devlink.h | 3 + net/core/devlink.c | 238 +++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 247 insertions(+), 8 deletions(-)
include/net/devlink.h +13 -1
@@ -142,7 +142,10 @@ struct devlink_rate {u64 tx_share;u64 tx_max; -struct devlink_port *devlink_port; +union { +struct devlink_port *devlink_port; +char *name; + }; };struct devlink_port {
@@ -1475,6 +1478,14 @@ struct devlink_ops { u64 tx_share, struct netlink_ext_ack *extack);int (*rate_leaf_tx_max_set)(struct devlink_rate *devlink_rate,void *priv,u64 tx_max,struct netlink_ext_ack *extack); +int (*rate_node_tx_share_set)(struct devlink_rate *devlink_rate,void *priv, +u64 tx_share,struct netlink_ext_ack *extack); +int (*rate_node_tx_max_set)(struct devlink_rate *devlink_rate,void *priv, +u64 tx_max,struct netlink_ext_ack *extack); +int (*rate_node_new)(struct devlink_rate *rate_node,void **priv, +struct netlink_ext_ack *extack); +int (*rate_node_del)(struct devlink_rate *rate_node,void *priv, +struct netlink_ext_ack *extack); };static inline void *devlink_priv (struct devlink *devlink)
@@ -1536,6 +1547,7 @@ void devlink_port_attrs_pci_sf_set(struct devlink_port *devlink_port, bool external);int devlink_rate_leaf_create (struct devlink_port *port,void *priv);void devlink_rate_leaf_destroy (struct devlink_port *devlink_port); +void devlink_rate_nodes_destroy (struct devlink *devlink); int devlink_sb_register(struct devlink *devlink, unsigned int sb_index, u32 size, u16 ingress_pools_count, u16 egress_pools_count, u16 ingress_tc_count,
include/uapi/linux/devlink.h +3 -0
@@ -213,6 +213,7 @@ enum devlink_port_flavour {enum devlink_rate_type {DEVLINK_RATE_TYPE_LEAF , +DEVLINK_RATE_TYPE_NODE , };enum devlink_param_cmode {
@@ -547,6 +548,8 @@ enum devlink_attr {DEVLINK_ATTR_RATE_TYPE ,/* u16 */ DEVLINK_ATTR_RATE_TX_SHARE ,/* u64 */ DEVLINK_ATTR_RATE_TX_MAX ,/* u64 */ +DEVLINK_ATTR_RATE_NODE_NAME ,/* string */ +/* add new attributes above here, update the policy in devlink.c */ __DEVLINK_ATTR_MAX,
net/core/devlink.c +231 -7
336 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH RESEND v3 13/18 netdevsim: Implement support for devlink rate nodes
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Implement new devlink ops that allow creation, deletion and setting of shared/max tx rate of devlink rate nodes through devlink API. Expose rate node and it's tx rates to netdevsim debugfs.
---
Notes:
v2->v3:
- added devlink_rate_nodes_destroy() call
drivers/net/netdevsim/dev.c | 80 +++++++++++++++++++++++++++++++++++++++ drivers/net/netdevsim/netdevsim.h | 1 + 2 files changed, 81 insertions(+)
drivers/net/netdevsim/dev.c +80 -0
@@ -263,12 +263,16 @@ static int nsim_dev_debugfs_init(struct nsim_dev *nsim_dev) nsim_dev->ddir, nsim_dev->nsim_bus_dev, &nsim_dev_max_vfs_fops); + nsim_dev->nodes_ddir =debugfs_create_dir ("rate_nodes" , nsim_dev->ddir); +if (IS_ERR (nsim_dev->nodes_ddir)) +return PTR_ERR (nsim_dev->nodes_ddir);nsim_udp_tunnels_debugfs_create (nsim_dev);return 0 ; }static void nsim_dev_debugfs_exit (struct nsim_dev *nsim_dev) { +debugfs_remove_recursive (nsim_dev->nodes_ddir);debugfs_remove_recursive (nsim_dev->ports_ddir);debugfs_remove_recursive (nsim_dev->ddir); }
@@ -451,8 +455,10 @@ static void nsim_dev_dummy_region_exit(struct nsim_dev *nsim_dev)static void __nsim_dev_port_del (struct nsim_dev_port *nsim_dev_port);int nsim_esw_legacy_enable (struct nsim_dev *nsim_dev,struct netlink_ext_ack *extack) { +struct devlink *devlink =priv_to_devlink (nsim_dev);struct nsim_dev_port *nsim_dev_port, *tmp; +devlink_rate_nodes_destroy (devlink);mutex_lock (&nsim_dev->port_list_lock);list_for_each_entry_safe (nsim_dev_port, tmp, &nsim_dev->port_list, list)if (nsim_dev_port_is_vf (nsim_dev_port))
@@ -1060,6 +1066,76 @@ static int nsim_leaf_tx_max_set(struct devlink_rate *devlink_rate, void *priv,return 0 ; } +struct nsim_rate_node { +struct dentry *ddir; +u16 tx_share; +u16 tx_max; +}; + +static int nsim_node_tx_share_set (struct devlink_rate *devlink_rate,void *priv, +u64 tx_share,struct netlink_ext_ack *extack) +{ +struct nsim_rate_node *nsim_node = priv; +int err; + + err =nsim_rate_bytes_to_units ("tx_share" , &tx_share, extack); +if (err) +return err; + + nsim_node->tx_share = tx_share; +return 0 ; +} + +static int nsim_node_tx_max_set (struct devlink_rate *devlink_rate,void *priv, +u64 tx_max,struct netlink_ext_ack *extack) +{ +struct nsim_rate_node *nsim_node = priv; +int err; + + err =nsim_rate_bytes_to_units ("tx_max" , &tx_max, extack); +if (err) +return err; + + nsim_node->tx_max = tx_max; +return 0 ; +} + +static int nsim_rate_node_new (struct devlink_rate *node,void **priv, +struct netlink_ext_ack *extack) +{ +struct nsim_dev *nsim_dev =devlink_priv (node->devlink); +struct nsim_rate_node *nsim_node; + +if (!nsim_esw_mode_is_switchdev (nsim_dev)) { +NL_SET_ERR_MSG_MOD (extack,"Node creation allowed only in switchdev mode." ); +return -EOPNOTSUPP ; + } + + nsim_node =kzalloc (sizeof (*nsim_node),GFP_KERNEL ); +if (!nsim_node) +return -ENOMEM ; + + nsim_node->ddir =debugfs_create_dir (node->name, nsim_dev->nodes_ddir); +if (!nsim_node->ddir) { +kfree (nsim_node); +return -ENOMEM ; + } +debugfs_create_u16 ("tx_share" ,0400 , nsim_node->ddir, &nsim_node->tx_share); +debugfs_create_u16 ("tx_max" ,0400 , nsim_node->ddir, &nsim_node->tx_max); + *priv = nsim_node; +return 0 ; +} + +static int nsim_rate_node_del (struct devlink_rate *node,void *priv, +struct netlink_ext_ack *extack) +{ +struct nsim_rate_node *nsim_node = priv; + +debugfs_remove_recursive (nsim_node->ddir); +kfree (nsim_node); +return 0 ; +} +static const struct devlink_ops nsim_dev_devlink_ops = { .eswitch_mode_set = nsim_devlink_eswitch_mode_set, .eswitch_mode_get = nsim_devlink_eswitch_mode_get,
@@ -1077,6 +1153,10 @@ static int nsim_leaf_tx_max_set(struct devlink_rate *devlink_rate, void *priv, .trap_policer_counter_get = nsim_dev_devlink_trap_policer_counter_get, .rate_leaf_tx_share_set = nsim_leaf_tx_share_set, .rate_leaf_tx_max_set = nsim_leaf_tx_max_set, + .rate_node_tx_share_set = nsim_node_tx_share_set, + .rate_node_tx_max_set = nsim_node_tx_max_set, + .rate_node_new = nsim_rate_node_new, + .rate_node_del = nsim_rate_node_del, };#define NSIM_DEV_MAX_MACS_DEFAULT 32
drivers/net/netdevsim/netdevsim.h +1 -0
@@ -222,6 +222,7 @@ struct nsim_dev {struct dentry *ports_ddir;struct dentry *take_snapshot;struct dentry *max_vfs; +struct dentry *nodes_ddir;struct bpf_offload_dev *bpf_dev;bool bpf_bind_accept;bool bpf_bind_verifier_accept;
Signature · 1 line
1.8.3.1
PATCH RESEND v3 14/18 selftest: netdevsim: Add devlink rate nodes test
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Test verifies that it is possible to create, delete and set min/max tx rate of devlink rate node on netdevsim VF.
---
Notes:
v1->v2:
- s/func/function/ in devlink commands
.../selftests/drivers/net/netdevsim/devlink.sh | 45 ++++++++++++++++++++++ 1 file changed, 45 insertions(+)
tools/testing/selftests/drivers/net/netdevsim/devlink.sh +45 -0
@@ -516,6 +516,14 @@ rate_leafs_get() '.[] | to_entries | .[] | select(.value.type == "leaf") | .key | select(contains("'$handle'"))' } +rate_nodes_get() +{ + local handle=$1 + + cmd_jq "devlink port function rate show -j" \ + '.[] | to_entries | .[] | select(.value.type == "node") | .key | select(contains("'$handle'"))' +} + rate_attr_set() { local handle=$1
@@ -555,6 +563,20 @@ rate_attr_tx_rate_check() check_err $? "Unexpected $name attr value $api_value != $rate" } +rate_node_add() +{ + local handle=$1 + + devlink port function rate add $handle +} + +rate_node_del() +{ + local handle=$1 + + devlink port function rate del $handle +} + rate_test() { RET=0
@@ -582,6 +604,29 @@ rate_test() rate=$(($rate+100)) done + local node1_name='group1' + local node1="$DL_HANDLE/$node1_name" + rate_node_add "$node1" + check_err $? "Failed to add node $node1" + + local num_nodes=`rate_nodes_get $DL_HANDLE | wc -w` + [ $num_nodes == 1 ] + check_err $? "Expected 1 rate node in output but got $num_nodes" + + local node_tx_share=10 + rate_attr_tx_rate_check $node1 tx_share $node_tx_share \ + $DEBUGFS_DIR/rate_nodes/${node1##*/}/tx_share + + local node_tx_max=100 + rate_attr_tx_rate_check $node1 tx_max $node_tx_max \ + $DEBUGFS_DIR/rate_nodes/${node1##*/}/tx_max + + rate_node_del "$node1" + check_err $? "Failed to delete node $node1" + local num_nodes=`rate_nodes_get $DL_HANDLE | wc -w` + [ $num_nodes == 0 ] + check_err $? "Expected 0 rate node but got $num_nodes" + log_test "rate test" }
Signature · 1 line
1.8.3.1
PATCH RESEND v3 15/18 devlink: Allow setting parent node of rate objects
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Refactor DEVLINK_CMD_RATE_{GET|SET} command handlers to support setting a node as a parent for another rate object (leaf or node) by means of new attribute DEVLINK_ATTR_RATE_PARENT_NODE_NAME. Extend devlink ops with new callbacks rate_{leaf|node}_parent_set() to set node as a parent for rate object to allow supporting drivers to implement rate grouping through devlink. Driver implementations are allowed to support leafs or node children only. Invoking callback with NULL as parent should be threated by the driver as unset parent action.
Extend rate object struct with reference counter to disallow deleting a node with any child pointing to it. User should unset parent for the child explicitly.
Example:
$ devlink port function rate add netdevsim/netdevsim10/group1
$ devlink port function rate add netdevsim/netdevsim10/group2
$ devlink port function rate set netdevsim/netdevsim10/group1 parent group2
$ devlink port function rate show netdevsim/netdevsim10/group1 netdevsim/netdevsim10/group1: type node parent group2
$ devlink port function rate set netdevsim/netdevsim10/group1 noparent
---
Notes:
v1->v2:
- s/func/function/ at commit message
v2->v3:
- unset parents for all rate objects at devlink_rate_nodes_destroy()
include/net/devlink.h | 14 ++++- include/uapi/linux/devlink.h | 1 + net/core/devlink.c | 125 ++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 137 insertions(+), 3 deletions(-)
include/net/devlink.h +13 -1
@@ -142,9 +142,13 @@ struct devlink_rate {u64 tx_share;u64 tx_max; +struct devlink_rate *parent;union {struct devlink_port *devlink_port; -char *name; +struct { +char *name; +refcount_t refcnt; + }; }; };
@@ -1486,6 +1490,14 @@ struct devlink_ops { struct netlink_ext_ack *extack);int (*rate_node_del)(struct devlink_rate *rate_node,void *priv,struct netlink_ext_ack *extack); +int (*rate_leaf_parent_set)(struct devlink_rate *child, +struct devlink_rate *parent, +void *priv_child,void *priv_parent, +struct netlink_ext_ack *extack); +int (*rate_node_parent_set)(struct devlink_rate *child, +struct devlink_rate *parent, +void *priv_child,void *priv_parent, +struct netlink_ext_ack *extack); };static inline void *devlink_priv (struct devlink *devlink)
include/uapi/linux/devlink.h +1 -0
@@ -549,6 +549,7 @@ enum devlink_attr {DEVLINK_ATTR_RATE_TX_SHARE ,/* u64 */ DEVLINK_ATTR_RATE_TX_MAX ,/* u64 */ DEVLINK_ATTR_RATE_NODE_NAME ,/* string */ + DEVLINK_ATTR_RATE_PARENT_NODE_NAME, /* string *//* add new attributes above here, update the policy in devlink.c */
net/core/devlink.c +123 -2
197 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH RESEND v3 16/18 netdevsim: Allow setting parent node of rate objects
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Implement new devlink ops that allow setting rate node as a parent for devlink port (leaf) or another devlink node through devlink API. Expose parent names to netdevsim debugfs in read only mode.
--- drivers/net/netdevsim/dev.c | 91 +++++++++++++++++++++++++++++++++++++-- drivers/net/netdevsim/netdevsim.h | 2 + 2 files changed, 89 insertions(+), 4 deletions(-)
drivers/net/netdevsim/dev.c +87 -4
@@ -222,6 +222,7 @@ static ssize_t nsim_dev_trap_fa_cookie_write(struct file *file,static int nsim_dev_debugfs_init (struct nsim_dev *nsim_dev) {char dev_ddir_name[sizeof (DRV_NAME ) +10 ]; +int err;sprintf (dev_ddir_name,DRV_NAME "%u" , nsim_dev->nsim_bus_dev->dev.id); nsim_dev->ddir =debugfs_create_dir (dev_ddir_name, nsim_dev_ddir);
@@ -264,10 +265,17 @@ static int nsim_dev_debugfs_init(struct nsim_dev *nsim_dev) nsim_dev->nsim_bus_dev, &nsim_dev_max_vfs_fops); nsim_dev->nodes_ddir =debugfs_create_dir ("rate_nodes" , nsim_dev->ddir); -if (IS_ERR (nsim_dev->nodes_ddir)) -return PTR_ERR (nsim_dev->nodes_ddir); +if (IS_ERR (nsim_dev->nodes_ddir)) { + err =PTR_ERR (nsim_dev->nodes_ddir); + goto err_out; + }nsim_udp_tunnels_debugfs_create (nsim_dev);return 0 ; + +err_out: +debugfs_remove_recursive (nsim_dev->ports_ddir); +debugfs_remove_recursive (nsim_dev->ddir); +return err; }static void nsim_dev_debugfs_exit (struct nsim_dev *nsim_dev)
@@ -277,6 +285,27 @@ static void nsim_dev_debugfs_exit(struct nsim_dev *nsim_dev)debugfs_remove_recursive (nsim_dev->ddir); } +static ssize_t nsim_dev_rate_parent_read (struct file *file, +char __user *data, +size_t count,loff_t *ppos) +{ +char **name_ptr = file->private_data; +size_t len; + +if (!*name_ptr) +return 0 ; + + len =strlen (*name_ptr); +return simple_read_from_buffer (data, count, ppos, *name_ptr, len); +} + +static const struct file_operations nsim_dev_rate_parent_fops = { + .open = simple_open, + .read = nsim_dev_rate_parent_read, + .llseek = generic_file_llseek, + .owner =THIS_MODULE , +}; +static int nsim_dev_port_debugfs_init (struct nsim_dev *nsim_dev,struct nsim_dev_port *nsim_dev_port) {
@@ -299,6 +328,11 @@ static int nsim_dev_port_debugfs_init(struct nsim_dev *nsim_dev, &nsim_bus_dev->vfconfigs[vf_id].min_tx_rate);debugfs_create_u16 ("tx_max" ,0400 , nsim_dev_port->ddir, &nsim_bus_dev->vfconfigs[vf_id].max_tx_rate); + nsim_dev_port->rate_parent =debugfs_create_file ("rate_parent" , +0400 , + nsim_dev_port->ddir, + &nsim_dev_port->parent_name, + &nsim_dev_rate_parent_fops); }debugfs_create_symlink ("dev" , nsim_dev_port->ddir, dev_link_name);
@@ -1068,6 +1102,8 @@ static int nsim_leaf_tx_max_set(struct devlink_rate *devlink_rate, void *priv,struct nsim_rate_node {struct dentry *ddir; +struct dentry *rate_parent; +char *parent_name;u16 tx_share;u16 tx_max; };
@@ -1105,6 +1141,7 @@ static int nsim_rate_node_new(struct devlink_rate *node, void **priv, {struct nsim_dev *nsim_dev =devlink_priv (node->devlink);struct nsim_rate_node *nsim_node; +int err;if (!nsim_esw_mode_is_switchdev (nsim_dev)) {NL_SET_ERR_MSG_MOD (extack,"Node creation allowed only in switchdev mode." );
@@ -1117,13 +1154,28 @@ static int nsim_rate_node_new(struct devlink_rate *node, void **priv, nsim_node->ddir =debugfs_create_dir (node->name, nsim_dev->nodes_ddir);if (!nsim_node->ddir) { -kfree (nsim_node); -return -ENOMEM ; + err = -ENOMEM ; + goto err_node; }debugfs_create_u16 ("tx_share" ,0400 , nsim_node->ddir, &nsim_node->tx_share);debugfs_create_u16 ("tx_max" ,0400 , nsim_node->ddir, &nsim_node->tx_max); + nsim_node->rate_parent =debugfs_create_file ("rate_parent" ,0400 , + nsim_node->ddir, + &nsim_node->parent_name, + &nsim_dev_rate_parent_fops); +if (IS_ERR (nsim_node->rate_parent)) { + err =PTR_ERR (nsim_node->rate_parent); + goto err_ddir; + } + *priv = nsim_node;return 0 ; + +err_ddir: +debugfs_remove_recursive (nsim_node->ddir); +err_node: +kfree (nsim_node); +return err; } static int nsim_rate_node_del(struct devlink_rate *node, void *priv,
@@ -1131,11 +1183,40 @@ static int nsim_rate_node_del(struct devlink_rate *node, void *priv, {struct nsim_rate_node *nsim_node = priv; +debugfs_remove (nsim_node->rate_parent);debugfs_remove_recursive (nsim_node->ddir);kfree (nsim_node);return 0 ; } +static int nsim_rate_leaf_parent_set (struct devlink_rate *child, +struct devlink_rate *parent, +void *priv_child,void *priv_parent, +struct netlink_ext_ack *extack) +{ +struct nsim_dev_port *nsim_dev_port = priv_child; + +if (parent) + nsim_dev_port->parent_name = parent->name; +else + nsim_dev_port->parent_name =NULL ; +return 0 ; +} + +static int nsim_rate_node_parent_set (struct devlink_rate *child, +struct devlink_rate *parent, +void *priv_child,void *priv_parent, +struct netlink_ext_ack *extack) +{ +struct nsim_rate_node *nsim_node = priv_child; + +if (parent) + nsim_node->parent_name = parent->name; +else + nsim_node->parent_name =NULL ; +return 0 ; +} +static const struct devlink_ops nsim_dev_devlink_ops = { .eswitch_mode_set = nsim_devlink_eswitch_mode_set, .eswitch_mode_get = nsim_devlink_eswitch_mode_get,
@@ -1157,6 +1238,8 @@ static int nsim_rate_node_del(struct devlink_rate *node, void *priv, .rate_node_tx_max_set = nsim_node_tx_max_set, .rate_node_new = nsim_rate_node_new, .rate_node_del = nsim_rate_node_del, + .rate_leaf_parent_set = nsim_rate_leaf_parent_set, + .rate_node_parent_set = nsim_rate_node_parent_set, };#define NSIM_DEV_MAX_MACS_DEFAULT 32
drivers/net/netdevsim/netdevsim.h +2 -0
@@ -211,6 +211,8 @@ struct nsim_dev_port {unsigned int port_index;enum nsim_dev_port_type port_type;struct dentry *ddir; +struct dentry *rate_parent; +char *parent_name;struct netdevsim *ns; };
Signature · 1 line
1.8.3.1
PATCH RESEND v3 17/18 selftest: netdevsim: Add devlink rate grouping test
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Test verifies that netdevsim correctly implements devlink ops callbacks that set node as a parent of devlink leaf or node rate object.
--- .../selftests/drivers/net/netdevsim/devlink.sh | 42 ++++++++++++++++++++++ 1 file changed, 42 insertions(+)
tools/testing/selftests/drivers/net/netdevsim/devlink.sh +42 -0
@@ -563,6 +563,26 @@ rate_attr_tx_rate_check() check_err $? "Unexpected $name attr value $api_value != $rate" } +rate_attr_parent_check() +{ + local handle=$1 + local parent=$2 + local debug_file=$3 + + rate_attr_set $handle parent $parent + check_err $? "Failed to set parent" + + debug_value=$(cat $debug_file) + check_err $? "Failed to get parent debugfs value" + [ "$debug_value" == "$parent" ] + check_err $? "Unexpected parent debug value $debug_value != $parent" + + api_value=$(rate_attr_get $r_obj parent) + check_err $? "Failed to get parent attr value" + [ "$api_value" == "$parent" ] + check_err $? "Unexpected parent attr value $api_value != $parent" +} + rate_node_add() { local handle=$1
@@ -627,6 +647,28 @@ rate_test() [ $num_nodes == 0 ] check_err $? "Expected 0 rate node but got $num_nodes" + local node1_name='group1' + local node1="$DL_HANDLE/$node1_name" + rate_node_add "$node1" + check_err $? "Failed to add node $node1" + + rate_attr_parent_check $r_obj $node1_name \ + $DEBUGFS_DIR/ports/${r_obj##*/}/rate_parent + + local node2_name='group2' + local node2="$DL_HANDLE/$node2_name" + rate_node_add "$node2" + check_err $? "Failed to add node $node2" + + rate_attr_parent_check $node2 $node1_name \ + $DEBUGFS_DIR/rate_nodes/$node2_name/rate_parent + rate_node_del "$node2" + check_err $? "Failed to delete node $node2" + rate_attr_set "$r_obj" noparent + check_err $? "Failed to unset $r_obj parent node" + rate_node_del "$node1" + check_err $? "Failed to delete node $node1" + log_test "rate test" }
Signature · 1 line
1.8.3.1
PATCH RESEND v3 18/18 Documentation: devlink rate objects
In reply to [email protected]From: Dmytro Linkin <[email protected]>
Add devlink rate objects section at devlink port documentation. Add devlink rate support info at netdevsim devlink documentation.
--- Documentation/networking/devlink/devlink-port.rst | 35 +++++++++++++++++++++++ Documentation/networking/devlink/netdevsim.rst | 26 +++++++++++++++++ 2 files changed, 61 insertions(+)
Documentation/networking/devlink/devlink-port.rst +35 -0
@@ -164,6 +164,41 @@ device to instantiate the subfunction device on particular PCI function. A subfunction device is created on the :ref:`Documentation/driver-api/auxiliary_bus.rst <auxiliary_bus>`. At this point a matching subfunction driver binds to the subfunction's auxiliary device. +Rate object management +====================== + +Devlink provides API to manage tx rates of single devlink port or a group. +This is done through rate objects, which can be one of the two types: + +``leaf`` + Represents a single devlink port; created/destroyed by the driver. Since leaf + have 1to1 mapping to its devlink port, in user space it is referred as + ``pci/<bus_addr>/<port_index>``; + +``node`` + Represents a group of rate objects (leafs and/or nodes); created/deleted by + request from the userspace; initially empty (no rate objects added). In + userspace it is referred as ``pci/<bus_addr>/<node_name>``, where + ``node_name`` can be any identifier, except decimal number, to avoid + collisions with leafs. + +API allows to configure following rate object's parameters: + +``tx_share`` + Minimum TX rate value shared among all other rate objects, or rate objects + that parts of the parent group, if it is a part of the same group. + +``tx_max`` + Maximum TX rate value. + +``parent`` + Parent node name. Parent node rate limits are considered as additional limits + to all node children limits. ``tx_max`` is an upper limit for children. + ``tx_share`` is a total bandwidth distributed among children. + +Driver implementations are allowed to support both or either rate object types +and setting methods of their parameters. + Terms and Definitions =====================
Documentation/networking/devlink/netdevsim.rst +26 -0
@@ -57,6 +57,32 @@ entries, FIB rule entries and nexthops that the driver will allow. $ devlink resource set netdevsim/netdevsim0 path /nexthops size 16 $ devlink dev reload netdevsim/netdevsim0 +Rate objects +============ + +The ``netdevsim`` driver supports rate objects management, which includes: + +- registerging/unregistering leaf rate objects per VF devlink port; +- creation/deletion node rate objects; +- setting tx_share and tx_max rate values for any rate object type; +- setting parent node for any rate object type. + +Rate nodes and it's parameters are exposed in ``netdevsim`` debugfs in RO mode. +For example created rate node with name ``some_group``: + +.. code:: shell + + $ ls /sys/kernel/debug/netdevsim/netdevsim0/rate_groups/some_group + rate_parent tx_max tx_share + +Same parameters are exposed for leaf objects in corresponding ports directories. +For ex.: + +.. code:: shell + + $ ls /sys/kernel/debug/netdevsim/netdevsim0/ports/1 + dev ethtool rate_parent tx_max tx_share + Driver-specific Traps =====================
Signature · 1 line
1.8.3.1
cover devlink rate support
In reply to [email protected]Resending without RFC.
Serries implements devlink rate commands, which are:
- Dump particular or all rate objects (JSON or non-JSON)
- Add/Delete node rate object
- Set tx rate share/max values for rate object
- Set/Unset parent rate object for other rate object
Examples:
Display all rate objects:
# devlink port function rate show
pci/0000:03:00.0/1 type leaf parent some_group
pci/0000:03:00.0/2 type leaf tx_share 12Mbit
pci/0000:03:00.0/some_group type node tx_share 1Gbps tx_max 5GbpsDisplay leaf rate object bound to the 1st devlink port of the pci/0000:03:00.0 device:
# devlink port function rate show pci/0000:03:00.0/1
pci/0000:03:00.0/1 type leafDisplay node rate object with name some_group of the pci/0000:03:00.0 device:
# devlink port function rate show pci/0000:03:00.0/some_group
pci/0000:03:00.0/some_group type nodeDisplay leaf rate object rate values using IEC units:
# devlink -i port function rate show pci/0000:03:00.0/2
pci/0000:03:00.0/2 type leaf 11718KibitDisplay pci/0000:03:00.0/2 leaf rate object as pretty JSON output:
# devlink -jp port function rate show pci/0000:03:00.0/2
{
"rate": {
"pci/0000:03:00.0/2": {
"type": "leaf",
"tx_share": 1500000
}
}
}Create node rate object with name "1st_group" on pci/0000:03:00.0 device:
# devlink port function rate add pci/0000:03:00.0/1st_group
Create node rate object with specified parameters:
# devlink port function rate add pci/0000:03:00.0/2nd_group \
tx_share 10Mbit tx_max 30Mbit parent 1st_groupSet parameters to the specified leaf rate object:
# devlink port function rate set pci/0000:03:00.0/1 \
tx_share 2Mbit tx_max 10MbitSet leaf's parent to "1st_group":
# devlink port function rate set pci/0000:03:00.0/1 parent 1st_group
Unset leaf's parent:
# devlink port function rate set pci/0000:03:00.0/1 noparent
Delete node rate object:
# devlink port function rate del pci/0000:03:00.0/2nd_group
Rate values can be specified in bits or bytes per second (bit|bps), with any SI (k, m, g, t) or IEC (ki, mi, gi, ti) prefix. Bare number means bits per second. Units also printed in "show" command output, but not necessarily the same which were specified with "set" or "add" command. -i/--iec switch force output in IEC units. JSON output always print values as bytes per sec.
Dmytro Linkin (4):
uapi: update devlink kernel header
devlink: Add helper function to validate object handler
devlink: Add port func rate support
devlink: Add ISO/IEC switch
devlink/devlink.c | 527 ++++++++++++++++++++++++++++++++++++++++--- include/uapi/linux/devlink.h | 17 ++ man/man8/devlink-port.8 | 8 + man/man8/devlink-rate.8 | 270 ++++++++++++++++++++++ man/man8/devlink.8 | 4 + 5 files changed, 797 insertions(+), 29 deletions(-) create mode 100644 man/man8/devlink-rate.8
Signature · 1 line
1.8.3.1
PATCH RESEND 1/4 uapi: update devlink kernel header
--- include/uapi/linux/devlink.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
include/uapi/linux/devlink.h +17 -0
@@ -126,6 +126,11 @@ enum devlink_command {DEVLINK_CMD_HEALTH_REPORTER_TEST , +DEVLINK_CMD_RATE_GET ,/* can dump */ +DEVLINK_CMD_RATE_SET , +DEVLINK_CMD_RATE_NEW , +DEVLINK_CMD_RATE_DEL , +/* add new commands above here */ __DEVLINK_CMD_MAX,DEVLINK_CMD_MAX = __DEVLINK_CMD_MAX -1
@@ -206,6 +211,11 @@ enum devlink_port_flavour {*/ }; +enum devlink_rate_type { +DEVLINK_RATE_TYPE_LEAF , +DEVLINK_RATE_TYPE_NODE , +}; +enum devlink_param_cmode {DEVLINK_PARAM_CMODE_RUNTIME ,DEVLINK_PARAM_CMODE_DRIVERINIT ,
@@ -534,6 +544,13 @@ enum devlink_attr {DEVLINK_ATTR_RELOAD_ACTION_STATS ,/* nested */ DEVLINK_ATTR_PORT_PCI_SF_NUMBER ,/* u32 */ + +DEVLINK_ATTR_RATE_TYPE ,/* u16 */ +DEVLINK_ATTR_RATE_TX_SHARE ,/* u64 */ +DEVLINK_ATTR_RATE_TX_MAX ,/* u64 */ +DEVLINK_ATTR_RATE_NODE_NAME ,/* string */ +DEVLINK_ATTR_RATE_PARENT_NODE_NAME ,/* string */ +/* add new attributes above here, update the policy in devlink.c */ __DEVLINK_ATTR_MAX,
Signature · 1 line
1.8.3.1
PATCH RESEND 2/4 devlink: Add helper function to validate object handler
In reply to Dmytro LinkinEvery handler argument validated in two steps, first of which, form checking, expects identifier is few words separated by slashes. For device and region handlers just checked if identifier have expected number of slashes.
Add generic function to do that and make code cleaner & consistent.
--- devlink/devlink.c | 38 +++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-)
devlink/devlink.c +21 -17
@@ -915,6 +915,19 @@ static int strtobool(const char *str, bool *p_val)return 0 ; } +static int ident_str_validate (char *str,unsigned int expected) +{ +if (!str) +return -EINVAL ; + +if (get_str_char_count (str,'/' ) != expected) { +pr_err ("Wrong identification string format.\n" ); +return -EINVAL ; + } + +return 0 ; +} +static int __dl_argv_handle (char *str,char **p_bus_name,char **p_dev_name) {str_split_by_char (str, p_bus_name, p_dev_name,'/' );
@@ -924,15 +937,12 @@ static int __dl_argv_handle(char *str, char **p_bus_name, char **p_dev_name)static int dl_argv_handle (struct dl *dl,char **p_bus_name,char **p_dev_name) {char *str =dl_argv_next (dl); +int err; -if (!str) { + err =ident_str_validate (str,1 ); +if (err) {pr_err ("Devlink identification (\"bus_name/dev_name\") expected\n" ); -return -EINVAL ; - } -if (get_str_char_count (str,'/' ) !=1 ) { -pr_err ("Wrong devlink identification string format.\n" ); -pr_err ("Expected \"bus_name/dev_name\".\n" ); -return -EINVAL ; +return err; }return __dl_argv_handle (str, p_bus_name, p_dev_name); }
@@ -1067,18 +1077,12 @@ static int dl_argv_handle_region(struct dl *dl, char **p_bus_name, char **p_dev_name, char **p_region) { char *str = dl_argv_next(dl); -unsigned int slash_count; +int err; -if (!str) { -pr_err ("Expected \"bus_name/dev_name/region\" identification.\n" ); -return -EINVAL ; - } - - slash_count =get_str_char_count (str,'/' ); -if (slash_count !=2 ) { -pr_err ("Wrong region identification string format.\n" ); + err =ident_str_validate (str,2 ); +if (err) {pr_err ("Expected \"bus_name/dev_name/region\" identification.\n"".\n" ); -return -EINVAL ; +return err; }return __dl_argv_handle_region (str, p_bus_name, p_dev_name, p_region);
Signature · 1 line
1.8.3.1
PATCH RESEND 3/4 devlink: Add port func rate support
In reply to Dmytro LinkinImplement user commands to manage devlink port func rate objects. List all rate commands:
$ devlink port func rate help
or just
$ devlink port func rate
To list all OR particular rate object:
$ devlink port func rate show
pci/0000:03:00.0/some_group: type node
pci/0000:03:00.0/0: type leaf
pci/0000:03:00.0/1: type leaf $ devlink prot func rate show pci/0000:03:00.0/1
pci/0000:03:00.0/0: type leaf $ devlink prot func rate show pci/0000:03:00.0/some_group
pci/0000:03:00.0/some_group: type nodeRate object of type "leaf" created by it's driver where name is the name of corresponding devlink port. Rate object of type "node" represents rate group created by the user using commands:
$ devlink port func rate add pci/0000:03:00.0/some_group
or with defining tx rate limits
$ devlink port func rate add pci/0000:03:00.0/some_group \
tx_shara 10kbit tx_max 100mbitNOTE: node name cannot be a decimal value because it conflicts with devlink port indexes.
To delete node object:
$ devlink port func rate del pci/0000:03:00.0/some_group
Set rate limits of existing rate object:
$ devlink prot func rate set pci/0000:03:00.0/0 \
tx_share 5MBps tx_max 25GBps $ devlink prot func rate set pci/0000:03:00.0/some_group \
tx_share 0Both SET and ADD commands accept any units of rates defined in IEC 60027-2 standard.
NOTE: rate value 0 means that rate is unlimited. Such value is also ommited in show command output.
NOTE: In SHOW command output rate values will be printed with suffixes as well, but in JSON output they are always units of Bps.
Set or unset parent of existing rate object:
$ devlink prot func rate set pci/0000:03:00.0/0 parent some_group
$ devlink port func rate set pci/0000:03:00.0/0 noparent
NOTE: Setting parent to empty ("") name due to kernel logic means unset parent and shouldn't be used to avoid unexpected parent unsets.
--- devlink/devlink.c | 489 ++++++++++++++++++++++++++++++++++++++++++++++-- man/man8/devlink-port.8 | 8 + man/man8/devlink-rate.8 | 256 +++++++++++++++++++++++++ 3 files changed, 738 insertions(+), 15 deletions(-) create mode 100644 man/man8/devlink-rate.8
devlink/devlink.c +474 -15
@@ -286,6 +286,11 @@ static void ifname_map_free(struct ifname_map *ifname_map)#define DL_OPT_PORT_PFNUMBER BIT(43)#define DL_OPT_PORT_SFNUMBER BIT(44)#define DL_OPT_PORT_FUNCTION_STATE BIT(45) +#define DL_OPT_PORT_FN_RATE_TYPE BIT(46) +#define DL_OPT_PORT_FN_RATE_TX_SHARE BIT(47) +#define DL_OPT_PORT_FN_RATE_TX_MAX BIT(48) +#define DL_OPT_PORT_FN_RATE_NODE_NAME BIT(49) +#define DL_OPT_PORT_FN_RATE_PARENT BIT(50)struct dl_opts {uint64_t present;/* flags of present items */
@@ -340,6 +345,11 @@ struct dl_opts {uint16_t port_flavour;uint16_t port_pfnumber;uint8_t port_fn_state; +uint16_t rate_type; +uint64_t rate_tx_share; +uint64_t rate_tx_max; +char *rate_node_name; +const char *rate_parent_node; };struct dl {
@@ -1054,38 +1064,103 @@ static int dl_argv_handle_both(struct dl *dl, char **p_bus_name,return 0 ; } -static int __dl_argv_handle_region (char *str,char **p_bus_name, -char **p_dev_name,char **p_region) +static int __dl_argv_handle_name (char *str,char **p_bus_name, +char **p_dev_name,char **p_name) {char *handlestr;int err; - err =str_split_by_char (str, &handlestr, p_region,'/' ); -if (err) { -pr_err ("Region identification \"%s\" is invalid\n" , str); + err =str_split_by_char (str, &handlestr, p_name,'/' ); +if (err)return err; - } - err =str_split_by_char (handlestr, p_bus_name, p_dev_name,'/' ); -if (err) { -pr_err ("Region identification \"%s\" is invalid\n" , str); -return err; - } -return 0 ; + +return str_split_by_char (handlestr, p_bus_name, p_dev_name,'/' ); }static int dl_argv_handle_region (struct dl *dl,char **p_bus_name, -char **p_dev_name,char **p_region) +char **p_dev_name,char **p_region) {char *str =dl_argv_next (dl);int err; err =ident_str_validate (str,2 );if (err) { -pr_err ("Expected \"bus_name/dev_name/region\" identification.\n"".\n" ); +pr_err ("Expected \"bus_name/dev_name/region\" identification.\n" );return err; } -return __dl_argv_handle_region (str, p_bus_name, p_dev_name, p_region); + err =__dl_argv_handle_name (str, p_bus_name, p_dev_name, p_region); +if (err) +pr_err ("Region identification \"%s\" is invalid\n" , str); +return err; +} + + +static int dl_argv_handle_rate_node (struct dl *dl,char **p_bus_name, +char **p_dev_name,char **p_node) +{ +char *str =dl_argv_next (dl); +int err; + + err =ident_str_validate (str,2 ); +if (err) { +pr_err ("Expected \"bus_name/dev_name/node\" identification.\n" ); +return err; + } + + err =__dl_argv_handle_name (str, p_bus_name, p_dev_name, p_node); +if (err) { +pr_err ("Node identification \"%s\" is invalid\n" , str); +return err; + } + +if (!**p_node ||strspn (*p_node,"0123456789" ) ==strlen (*p_node)) { + err = -EINVAL ; +pr_err ("Node name cannot be a devlink port index or empty.\n" ); + } + +return err; +} + +static int dl_argv_handle_rate (struct dl *dl,char **p_bus_name, +char **p_dev_name,uint32_t *p_port_index, +char **p_node_name,uint64_t *p_handle_bit) +{ +char *str =dl_argv_next (dl); +char *identifier; +int err; + + err =ident_str_validate (str,2 ); +if (err) { +pr_err ("Expected \"bus_name/dev_name/node\" or " +"\"bus_name/dev_name/port_index\" identification.\n" ); +return err; + } + + err =__dl_argv_handle_name (str, p_bus_name, p_dev_name, &identifier); +if (err) { +pr_err ("Identification \"%s\" is invalid\n" , str); +return err; + } + +if (!*identifier) { +pr_err ("Identifier cannot be empty" ); +return -EINVAL ; + } + +if (strspn (identifier,"0123456789" ) ==strlen (identifier)) { + err =strtouint32_t (identifier, p_port_index); +if (err) { +pr_err ("Port index \"%s\" is not a number" +" or not within range\n" , identifier); +return err; + } + *p_handle_bit =DL_OPT_HANDLEP ; + }else { + *p_handle_bit =DL_OPT_PORT_FN_RATE_NODE_NAME ; + *p_node_name = identifier; + } +return 0 ; }static int dl_argv_uint64_t (struct dl *dl,uint64_t *p_val)
@@ -1397,6 +1472,36 @@ static int port_fn_state_parse(const char *statestr, uint8_t *state)return 0 ; } +static int port_fn_rate_type_get (const char *typestr,uint16_t *type) +{ +if (!strcmp (typestr,"leaf" )) + *type =DEVLINK_RATE_TYPE_LEAF ; +else if (!strcmp (typestr,"node" )) + *type =DEVLINK_RATE_TYPE_NODE ; +else +return -EINVAL ; +return 0 ; +} + +static int port_fn_rate_value_get (struct dl *dl,uint64_t *rate) +{ +const char *ratestr; +__u64 rate64; +int err; + + err =dl_argv_str (dl, &ratestr); +if (err) +return err; + err =get_rate64 (&rate64, ratestr); +if (err) { +pr_err ("Invalid rate value: \"%s\"\n" , ratestr); +return -EINVAL ; + } + + *rate = rate64; +return 0 ; +} +struct dl_args_metadata {uint64_t o_flag;char err_msg[DL_ARGS_REQUIRED_MAX_ERR_LEN ];
@@ -1469,6 +1574,19 @@ static int dl_argv_parse(struct dl *dl, uint64_t o_required,return err; o_required &= ~(DL_OPT_HANDLE |DL_OPT_HANDLEP ) | handle_bit; o_found |= handle_bit; + }else if (o_required &DL_OPT_HANDLEP && + o_required &DL_OPT_PORT_FN_RATE_NODE_NAME ) { +uint64_t handle_bit; + + err =dl_argv_handle_rate (dl, &opts->bus_name, &opts->dev_name, + &opts->port_index, + &opts->rate_node_name, + &handle_bit); +if (err) +return err; + o_required &= ~(DL_OPT_HANDLEP |DL_OPT_PORT_FN_RATE_NODE_NAME ) | + handle_bit; + o_found |= handle_bit; }else if (o_required &DL_OPT_HANDLE ) { err =dl_argv_handle (dl, &opts->bus_name, &opts->dev_name);if (err)
@@ -1487,6 +1605,13 @@ static int dl_argv_parse(struct dl *dl, uint64_t o_required,if (err)return err; o_found |=DL_OPT_HANDLE_REGION ; + }else if (o_required &DL_OPT_PORT_FN_RATE_NODE_NAME ) { + err =dl_argv_handle_rate_node (dl, &opts->bus_name, + &opts->dev_name, + &opts->rate_node_name); +if (err) +return err; + o_found |=DL_OPT_PORT_FN_RATE_NODE_NAME ; }while (dl_argc (dl)) {
@@ -1884,6 +2009,44 @@ static int dl_argv_parse(struct dl *dl, uint64_t o_required,if (err)return err; o_found |=DL_OPT_PORT_SFNUMBER ; + }else if (dl_argv_match (dl,"type" ) && + (o_all &DL_OPT_PORT_FN_RATE_TYPE )) { +const char *typestr; + +dl_arg_inc (dl); + err =dl_argv_str (dl, &typestr); +if (err) +return err; + err =port_fn_rate_type_get (typestr, &opts->rate_type); +if (err) +return err; + o_found |=DL_OPT_PORT_FN_RATE_TYPE ; + }else if (dl_argv_match (dl,"tx_share" ) && + (o_all &DL_OPT_PORT_FN_RATE_TX_SHARE )) { +dl_arg_inc (dl); + err =port_fn_rate_value_get (dl, &opts->rate_tx_share); +if (err) +return err; + o_found |=DL_OPT_PORT_FN_RATE_TX_SHARE ; + }else if (dl_argv_match (dl,"tx_max" ) && + (o_all &DL_OPT_PORT_FN_RATE_TX_MAX )) { +dl_arg_inc (dl); + err =port_fn_rate_value_get (dl, &opts->rate_tx_max); +if (err) +return err; + o_found |=DL_OPT_PORT_FN_RATE_TX_MAX ; + }else if (dl_argv_match (dl,"parent" ) && + (o_all &DL_OPT_PORT_FN_RATE_PARENT )) { +dl_arg_inc (dl); + err =dl_argv_str (dl, &opts->rate_parent_node); +if (err) +return err; + o_found |=DL_OPT_PORT_FN_RATE_PARENT ; + }else if (dl_argv_match (dl,"noparent" ) && + (o_all &DL_OPT_PORT_FN_RATE_PARENT )) { +dl_arg_inc (dl); + opts->rate_parent_node ="" ; + o_found |=DL_OPT_PORT_FN_RATE_PARENT ; }else {pr_err ("Unknown option \"%s\"\n" ,dl_argv (dl));return -EINVAL ;
@@ -1956,6 +2119,11 @@ static void dl_opts_put(struct nlmsghdr *nlh, struct dl *dl)mnl_attr_put_strz (nlh,DEVLINK_ATTR_DEV_NAME , opts->dev_name);mnl_attr_put_strz (nlh,DEVLINK_ATTR_REGION_NAME , opts->region_name); + }else if (opts->present &DL_OPT_PORT_FN_RATE_NODE_NAME ) { +mnl_attr_put_strz (nlh,DEVLINK_ATTR_BUS_NAME , opts->bus_name); +mnl_attr_put_strz (nlh,DEVLINK_ATTR_DEV_NAME , opts->dev_name); +mnl_attr_put_strz (nlh,DEVLINK_ATTR_RATE_NODE_NAME , + opts->rate_node_name); }if (opts->present &DL_OPT_PORT_TYPE ) mnl_attr_put_u16(nlh, DEVLINK_ATTR_PORT_TYPE,
@@ -2077,6 +2245,18 @@ static void dl_opts_put(struct nlmsghdr *nlh, struct dl *dl)mnl_attr_put_u16 (nlh,DEVLINK_ATTR_PORT_PCI_PF_NUMBER , opts->port_pfnumber);if (opts->present &DL_OPT_PORT_SFNUMBER )mnl_attr_put_u32 (nlh,DEVLINK_ATTR_PORT_PCI_SF_NUMBER , opts->port_sfnumber); +if (opts->present &DL_OPT_PORT_FN_RATE_TYPE ) +mnl_attr_put_u16 (nlh,DEVLINK_ATTR_RATE_TYPE , + opts->rate_type); +if (opts->present &DL_OPT_PORT_FN_RATE_TX_SHARE ) +mnl_attr_put_u64 (nlh,DEVLINK_ATTR_RATE_TX_SHARE , + opts->rate_tx_share); +if (opts->present &DL_OPT_PORT_FN_RATE_TX_MAX ) +mnl_attr_put_u64 (nlh,DEVLINK_ATTR_RATE_TX_MAX , + opts->rate_tx_max); +if (opts->present &DL_OPT_PORT_FN_RATE_PARENT ) +mnl_attr_put_strz (nlh,DEVLINK_ATTR_RATE_PARENT_NODE_NAME , + opts->rate_parent_node); } static int dl_argv_parse_put(struct nlmsghdr *nlh, struct dl *dl,
@@ -3790,6 +3970,7 @@ static void cmd_port_help(void)pr_err (" devlink port split DEV/PORT_INDEX count COUNT\n" );pr_err (" devlink port unsplit DEV/PORT_INDEX\n" );pr_err (" devlink port function set DEV/PORT_INDEX [ hw_addr ADDR ] [ state STATE ]\n" ); +pr_err (" devlink port function rate { help | show | add | del | set }\n" );pr_err (" devlink port param set DEV/PORT_INDEX name PARAMETER value VALUE cmode { permanent | driverinit | runtime }\n" );pr_err (" devlink port param show [DEV/PORT_INDEX name PARAMETER]\n" );pr_err (" devlink port health show [ DEV/PORT_INDEX reporter REPORTER_NAME ]\n" );
@@ -4083,6 +4264,7 @@ static int cmd_port_param_show(struct dl *dl)static void cmd_port_function_help (void ) {pr_err ("Usage: devlink port function set DEV/PORT_INDEX [ hw_addr ADDR ] [ state STATE ]\n" ); +pr_err (" devlink port function rate { help | show | add | del | set }\n" ); }static int cmd_port_function_set (struct dl *dl)
@@ -4304,6 +4486,280 @@ static int cmd_port_param(struct dl *dl)return -ENOENT ; } +static void +pr_out_port_rate_handle_start (struct dl *dl,struct nlattr **tb,bool try_nice) +{ +const char *bus_name; +const char *dev_name; +const char *node_name; +static char buf[64 ]; + + bus_name =mnl_attr_get_str (tb[DEVLINK_ATTR_BUS_NAME ]); + dev_name =mnl_attr_get_str (tb[DEVLINK_ATTR_DEV_NAME ]); + node_name =mnl_attr_get_str (tb[DEVLINK_ATTR_RATE_NODE_NAME ]); +sprintf (buf,"%s/%s/%s" , bus_name, dev_name, node_name); +if (dl->json_output) +open_json_object (buf); +else +pr_out ("%s:" , buf); +} + +static char *port_rate_type_name (uint16_t type) +{ +switch (type) { +case DEVLINK_RATE_TYPE_LEAF : +return "leaf" ; +case DEVLINK_RATE_TYPE_NODE : +return "node" ; +default : +return "<unknown type>" ; + } +} + +static void pr_out_port_fn_rate (struct dl *dl,struct nlattr **tb) +{ + +if (!tb[DEVLINK_ATTR_RATE_NODE_NAME ]) +pr_out_port_handle_start (dl, tb, false); +else +pr_out_port_rate_handle_start (dl, tb, false); +check_indent_newline (dl); + +if (tb[DEVLINK_ATTR_RATE_TYPE ]) { +uint16_t type = +mnl_attr_get_u16 (tb[DEVLINK_ATTR_RATE_TYPE ]); + +print_string (PRINT_ANY ,"type" ,"type %s" , +port_rate_type_name (type)); + } +if (tb[DEVLINK_ATTR_RATE_TX_SHARE ]) { +uint64_t rate = +mnl_attr_get_u64 (tb[DEVLINK_ATTR_RATE_TX_SHARE ]); + +if (rate) +print_rate (false,PRINT_ANY ,"tx_share" , +" tx_share %s" , rate); + } +if (tb[DEVLINK_ATTR_RATE_TX_MAX ]) { +uint64_t rate = +mnl_attr_get_u64 (tb[DEVLINK_ATTR_RATE_TX_MAX ]); + +if (rate) +print_rate (false,PRINT_ANY ,"tx_max" , +" tx_max %s" , rate); + } +if (tb[DEVLINK_ATTR_RATE_PARENT_NODE_NAME ]) { +const char *parent = +mnl_attr_get_str (tb[DEVLINK_ATTR_RATE_PARENT_NODE_NAME ]); + +print_string (PRINT_ANY ,"parent" ," parent %s" , parent); + } + +pr_out_port_handle_end (dl); +} + +static int cmd_port_fn_rate_show_cb (const struct nlmsghdr *nlh,void *data) +{ +struct dl *dl = data; +struct nlattr *tb[DEVLINK_ATTR_MAX +1 ] = {}; +struct genlmsghdr *genl =mnl_nlmsg_get_payload (nlh); + +mnl_attr_parse (nlh,sizeof (*genl), attr_cb, tb); +if ((!tb[DEVLINK_ATTR_BUS_NAME ] || !tb[DEVLINK_ATTR_DEV_NAME ] || + !tb[DEVLINK_ATTR_PORT_INDEX ]) && + !tb[DEVLINK_ATTR_RATE_NODE_NAME ]) { +return MNL_CB_ERROR ; + } +pr_out_port_fn_rate (dl, tb); +return MNL_CB_OK ; +} + +static void cmd_port_fn_rate_help (void ) +{ +pr_err ("Usage: devlink port function rate help\n" ); +pr_err (" devlink port function rate show [ DEV/{ PORT_INDEX | NODE_NAME } ]\n" ); +pr_err (" devlink port function rate add DEV/NODE_NAME\n" ); +pr_err (" [ tx_share VAL ][ tx_max VAL ][ { parent NODE_NAME | noparent } ]\n" ); +pr_err (" devlink port function rate del DEV/NODE_NAME\n" ); +pr_err (" devlink port function rate set DEV/{ PORT_INDEX | NODE_NAME }\n" ); +pr_err (" [ tx_share VAL ][ tx_max VAL ][ { parent NODE_NAME | noparent } ]\n\n" ); +pr_err (" VAL - float or integer value in units of bits or bytes per second (bit|bps)\n" ); +pr_err (" and SI (k-, m-, g-, t-) or IEC (ki-, mi-, gi-, ti-) case-insensitive prefix.\n" ); +pr_err (" Bare number, means bits per second, is possible.\n\n" ); +pr_err (" For details refer to devlink-rate(8) man page.\n" ); +} + +static int cmd_port_fn_rate_show (struct dl *dl) +{ +struct nlmsghdr *nlh; +uint16_t flags =NLM_F_REQUEST |NLM_F_ACK ; +int err; + +if (dl_argc (dl) ==0 ) + flags |=NLM_F_DUMP ; + + nlh =mnlu_gen_socket_cmd_prepare (&dl->nlg,DEVLINK_CMD_RATE_GET , flags); + +if (dl_argc (dl) >0 ) { + err =dl_argv_parse_put (nlh, dl,DL_OPT_HANDLEP | +DL_OPT_PORT_FN_RATE_NODE_NAME ,0 ); +if (err) +return err; + } + +pr_out_section_start (dl,"rate" ); + err =mnlu_gen_socket_sndrcv (&dl->nlg, nlh, cmd_port_fn_rate_show_cb, dl); +pr_out_section_end (dl); +return err; +} + +static int port_fn_check_tx_rates (uint64_t min_rate,uint64_t max_rate) +{ +if (max_rate && min_rate > max_rate) { +pr_err ("Invalid. Expected tx_share <= tx_max or tx_share == 0.\n" ); +return -EINVAL ; + } +return 0 ; +} + +static int port_fn_get_and_check_tx_rates (struct dl_opts *reply, +struct dl_opts *request) +{ +uint64_t min = reply->rate_tx_share; +uint64_t max = reply->rate_tx_max; + +if (request->present &DL_OPT_PORT_FN_RATE_TX_SHARE ) +return port_fn_check_tx_rates (request->rate_tx_share, max); +return port_fn_check_tx_rates (min, request->rate_tx_max); +} + +static int cmd_port_fn_rate_add (struct dl *dl) +{ +struct nlmsghdr *nlh; +int err; + + nlh =mnlu_gen_socket_cmd_prepare (&dl->nlg,DEVLINK_CMD_RATE_NEW , +NLM_F_REQUEST |NLM_F_ACK ); + err =dl_argv_parse_put (nlh, dl,DL_OPT_PORT_FN_RATE_NODE_NAME , +DL_OPT_PORT_FN_RATE_TX_SHARE | +DL_OPT_PORT_FN_RATE_TX_MAX ); +if (err) +return err; + +if ((dl->opts.present &DL_OPT_PORT_FN_RATE_TX_SHARE ) && + (dl->opts.present &DL_OPT_PORT_FN_RATE_TX_MAX )) { + err =port_fn_check_tx_rates (dl->opts.rate_tx_share, + dl->opts.rate_tx_max); +if (err) +return err; + } + +return mnlu_gen_socket_sndrcv (&dl->nlg, nlh,NULL ,NULL ); +} + +static int cmd_port_fn_rate_del (struct dl *dl) +{ +struct nlmsghdr *nlh; +int err; + + nlh =mnlu_gen_socket_cmd_prepare (&dl->nlg,DEVLINK_CMD_RATE_DEL , +NLM_F_REQUEST |NLM_F_ACK ); + err =dl_argv_parse_put (nlh, dl,DL_OPT_PORT_FN_RATE_NODE_NAME ,0 ); +if (err) +return err; + +return mnlu_gen_socket_sndrcv (&dl->nlg, nlh,NULL ,NULL ); +} + +static int port_fn_get_rates_cb (const struct nlmsghdr *nlh,void *data) +{ +struct dl_opts *opts = data; +struct nlattr *tb[DEVLINK_ATTR_MAX +1 ] = {}; +struct genlmsghdr *genl =mnl_nlmsg_get_payload (nlh); + +mnl_attr_parse (nlh,sizeof (*genl), attr_cb, tb); +if ((!tb[DEVLINK_ATTR_BUS_NAME ] || !tb[DEVLINK_ATTR_DEV_NAME ] || + !tb[DEVLINK_ATTR_PORT_INDEX ]) && + !tb[DEVLINK_ATTR_RATE_NODE_NAME ]) { +return MNL_CB_ERROR ; + } + +if (tb[DEVLINK_ATTR_RATE_TX_SHARE ]) + opts->rate_tx_share = +mnl_attr_get_u64 (tb[DEVLINK_ATTR_RATE_TX_SHARE ]); +if (tb[DEVLINK_ATTR_RATE_TX_MAX ]) + opts->rate_tx_max = +mnl_attr_get_u64 (tb[DEVLINK_ATTR_RATE_TX_MAX ]); +return MNL_CB_OK ; +} + +static int cmd_port_fn_rate_set (struct dl *dl) +{ +struct dl_opts tmp_opts = {0 }; +struct nlmsghdr *nlh; +int err; + + err =dl_argv_parse (dl,DL_OPT_HANDLEP | +DL_OPT_PORT_FN_RATE_NODE_NAME , +DL_OPT_PORT_FN_RATE_TX_SHARE | +DL_OPT_PORT_FN_RATE_TX_MAX | +DL_OPT_PORT_FN_RATE_PARENT ); +if (err) +return err; + +if ((dl->opts.present &DL_OPT_PORT_FN_RATE_TX_SHARE ) && + (dl->opts.present &DL_OPT_PORT_FN_RATE_TX_MAX )) { + err =port_fn_check_tx_rates (dl->opts.rate_tx_share, + dl->opts.rate_tx_max); +if (err) +return err; + }else if (dl->opts.present & + (DL_OPT_PORT_FN_RATE_TX_SHARE |DL_OPT_PORT_FN_RATE_TX_MAX )) { + nlh =mnlu_gen_socket_cmd_prepare (&dl->nlg,DEVLINK_CMD_RATE_GET , +NLM_F_REQUEST |NLM_F_ACK ); + tmp_opts = dl->opts; + dl->opts.present &= ~(DL_OPT_PORT_FN_RATE_TX_SHARE | +DL_OPT_PORT_FN_RATE_TX_MAX | +DL_OPT_PORT_FN_RATE_PARENT ); +dl_opts_put (nlh, dl); + err =mnlu_gen_socket_sndrcv (&dl->nlg, nlh, port_fn_get_rates_cb, + &dl->opts); +if (err) +return err; + err =port_fn_get_and_check_tx_rates (&dl->opts, &tmp_opts); +if (err) +return err; + dl->opts = tmp_opts; + } + + nlh =mnlu_gen_socket_cmd_prepare (&dl->nlg,DEVLINK_CMD_RATE_SET , +NLM_F_REQUEST |NLM_F_ACK ); +dl_opts_put (nlh, dl); +return mnlu_gen_socket_sndrcv (&dl->nlg, nlh,NULL ,NULL ); +} + +static int cmd_port_function_rate (struct dl *dl) +{ +if (dl_argv_match (dl,"help" )) { +cmd_port_fn_rate_help (); +return 0 ; + }else if (dl_argv_match (dl,"show" ) ||dl_no_arg (dl)) { +dl_arg_inc (dl); +return cmd_port_fn_rate_show (dl); + }else if (dl_argv_match (dl,"add" )) { +dl_arg_inc (dl); +return cmd_port_fn_rate_add (dl); + }else if (dl_argv_match (dl,"del" )) { +dl_arg_inc (dl); +return cmd_port_fn_rate_del (dl); + }else if (dl_argv_match (dl,"set" )) { +dl_arg_inc (dl); +return cmd_port_fn_rate_set (dl); + } +pr_err ("Command \"%s\" not found\n" ,dl_argv (dl)); +return -ENOENT ; +} +static int cmd_port_function (struct dl *dl) {if (dl_argv_match (dl,"help" ) ||dl_no_arg (dl)) {
@@ -4312,6 +4768,9 @@ static int cmd_port_function(struct dl *dl) }else if (dl_argv_match (dl,"set" )) {dl_arg_inc (dl);return cmd_port_function_set (dl); + }else if (dl_argv_match (dl,"rate" )) { +dl_arg_inc (dl); +return cmd_port_function_rate (dl); }pr_err ("Command \"%s\" not found\n" ,dl_argv (dl));return -ENOENT ;
man/man8/devlink-port.8 +8 -0
@@ -71,6 +71,10 @@ devlink-port \- devlink port configuration .RI "STATE }" .ti -8 +.BR "devlink port function rate " +.RI "{ " show " | " set " | " add " | " del " | " help " }" + +.ti -8 .B devlink dev param set .I DEV/PORT_INDEX .B name
@@ -240,6 +244,10 @@ Configuration mode in which the new value is set. Specify parameter name to show. If this argument, as well as port index, are omitted - all parameters supported by devlink device ports are listed. +.SS devlink port function rate - manage devlink rate objects +Is an alias for +.BR devlink-rate (8). + .SH "EXAMPLES" .PP devlink port show
man/man8/devlink-rate.8 new file +256 -0
256 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH RESEND 4/4 devlink: Add ISO/IEC switch
In reply to Dmytro LinkinAdd -i/--iec switch to print rate values using binary prefixes. Update devlink(8) and devlink-rate(8) pages.
--- devlink/devlink.c | 12 +++++++++--- man/man8/devlink-rate.8 | 24 +++++++++++++++++++----- man/man8/devlink.8 | 4 ++++ 3 files changed, 32 insertions(+), 8 deletions(-)
devlink/devlink.c +9 -3
@@ -71,6 +71,8 @@ static bool g_indent_newline;#define INDENT_STR_MAXLEN 32static char g_indent_str[INDENT_STR_MAXLEN +1 ] ="" ; +static bool use_iec = false; +static void __attribute__((format (printf,1 ,2 )))pr_err (const char *fmt, ...) {
@@ -4537,7 +4539,7 @@ static void pr_out_port_fn_rate(struct dl *dl, struct nlattr **tb)mnl_attr_get_u64 (tb[DEVLINK_ATTR_RATE_TX_SHARE ]);if (rate) -print_rate (false,PRINT_ANY ,"tx_share" , +print_rate (use_iec,PRINT_ANY ,"tx_share" ," tx_share %s" , rate); }if (tb[DEVLINK_ATTR_RATE_TX_MAX ]) {
@@ -4545,7 +4547,7 @@ static void pr_out_port_fn_rate(struct dl *dl, struct nlattr **tb)mnl_attr_get_u64 (tb[DEVLINK_ATTR_RATE_TX_MAX ]);if (rate) -print_rate (false,PRINT_ANY ,"tx_max" , +print_rate (use_iec,PRINT_ANY ,"tx_max" ," tx_max %s" , rate); }if (tb[DEVLINK_ATTR_RATE_PARENT_NODE_NAME ]) {
@@ -9066,6 +9068,7 @@ int main(int argc, char **argv) { "verbose", no_argument, NULL, 'v' }, { "statistics", no_argument, NULL, 's' }, { "Netns", required_argument, NULL, 'N' }, + { "iec", no_argument, NULL, 'i' }, { NULL, 0, NULL, 0 } };const char *batch_file =NULL ;
@@ -9081,7 +9084,7 @@ int main(int argc, char **argv)return EXIT_FAILURE ; } -while ((opt =getopt_long (argc, argv,"Vfb:njpvsN:" , +while ((opt =getopt_long (argc, argv,"Vfb:njpvsN:i" , long_options,NULL )) >=0 ) {switch (opt) {
@@ -9116,6 +9119,9 @@ int main(int argc, char **argv) goto dl_free; }break ; +case 'i' : + use_iec = true; +break ;default :pr_err ("Unknown option.\n" );help ();
man/man8/devlink-rate.8 +19 -5
@@ -15,7 +15,7 @@ devlink-rate \- devlink rate management .ti -8 .IR OPTIONS " := { " -.BR -j [ \fIson "] | " -p [ \fIretty "] }" +.BR -j [ \fIson "] | " -p [ \fIretty "] | " -i [ \fIec "] }" .ti -8 .B devlink port function rate show
@@ -61,9 +61,10 @@ except decimal number, to avoid collisions with leafs. .PP Command output show rate object identifier, it's type and rate values along with parent node name. Rate values printed in SI units which are more suitable to -represent specific value. JSON (\fB-j\fR) output always print rate values in -bytes per second. Zero rate values means "unlimited" rates and ommited in -output, as well as parent node name. +represent specific value. To print values in IEC units \fB-i\fR switch is +used. JSON (\fB-j\fR) output always print rate values in bytes per second. Zero +rate values means "unlimited" rates and ommited in output, as well as parent +node name. .SS devlink port function rate set - set rate object parameters. Allows set rate object's parameters. If any parameter specified multiple times
@@ -85,7 +86,7 @@ rate group. .TP 8 .I VALUE These parameter accept a floating point number, possibly followed by either a -unit. +unit (both SI and IEC units supported). .RS .TP bit or a bare number
@@ -117,6 +118,9 @@ Gigabytes per second .TP tbps Terabytes per second +.P +To specify in IEC units, replace the SI prefix (k-, m-, g-, t-) with IEC prefix +(ki-, mi-, gi- and ti-) respectively. Input is case-insensitive. .RE .PP .BI parent " NODE_NAME \fR| " noparent
@@ -169,6 +173,16 @@ pci/0000:03:00.0/1 type leaf .RE .PP +\fB*\fR Display leaf rate object rate values using IEC units: +.RS 4 +.PP +# devlink -i port function rate show pci/0000:03:00.0/2 +.br +pci/0000:03:00.0/2 type leaf 11718Kibit +.br +.RE + +.PP \fB*\fR Display node rate object with name some_group of the pci/0000:03:00.0 device: .RS 4 .PP
man/man8/devlink.8 +4 -0
@@ -59,6 +59,10 @@ Output statistics. .BR "\-N", " \-Netns " <NETNSNAME> Switches to the specified network namespace. +.TP +.BR "\-i", " --iec" +Print human readable rates in IEC units (e.g. 1Ki = 1024). + .SS .I OBJECT
Signature · 1 line
1.8.3.1
Jakub Kicinski
maintainerQuoted 20 lines · On Wed, 2 Jun 2021 15:17:13 +0300 [email protected] wrote:
From: Dmytro Linkin <[email protected]>
Resending without RFC.
Currently kernel provides a way to change tx rate of single VF in switchdev mode via tc-police action. When lots of VFs are configured management of theirs rates becomes non-trivial task and some grouping mechanism is required. Implementing such grouping in tc-police will bring flow related limitations and unwanted complications, like:
- tc-police is a policer and there is a user request for a traffic shaper, so shared tc-police action is not suitable;
- flows requires net device to be placed on, means "groups" wouldn't have net device instance itself. Taking into the account previous point was reviewed a sollution, when representor have a policer and the driver use a shaper if qdisc contains group of VFs - such approach ugly, compilated and misleading;
- TC is ingress only, while configuring "other" side of the wire looks more like a "real" picture where shaping is outside of the steering world, similar to "ip link" command;
Quoted 1 line
According to that devlink is the most appropriate place.
I don't think you researched TC well enough. But whatever, I'm tired of being the only one who pushes back given I neither work on or use any of these features.
You need to provide a real implementation for this new uAPI, tho. netdevsim won't cut it.
Quoted 21 lines · Jakub Kicinski
Quoted 20 lines · On Wed, 2 Jun 2021 15:17:13 +0300 [email protected] wrote:
From: Dmytro Linkin <[email protected]>
Resending without RFC.
Currently kernel provides a way to change tx rate of single VF in switchdev mode via tc-police action. When lots of VFs are configured management of theirs rates becomes non-trivial task and some grouping mechanism is required. Implementing such grouping in tc-police will bring flow related limitations and unwanted complications, like:
- tc-police is a policer and there is a user request for a traffic shaper, so shared tc-police action is not suitable;
- flows requires net device to be placed on, means "groups" wouldn't have net device instance itself. Taking into the account previous point was reviewed a sollution, when representor have a policer and the driver use a shaper if qdisc contains group of VFs - such approach ugly, compilated and misleading;
- TC is ingress only, while configuring "other" side of the wire looks more like a "real" picture where shaping is outside of the steering world, similar to "ip link" command;
Quoted 9 lines
Quoted 1 line
According to that devlink is the most appropriate place.
I don't think you researched TC well enough. But whatever, I'm tired of being the only one who pushes back given I neither work on or use any of these features.
You need to provide a real implementation for this new uAPI, tho. netdevsim won't cut it.
+Saeed
The series is already big enough to add more patches to it and implementation (mlx5_core) must go through Saeed. How would You like to proceed?
Quoted 21 lines · Jakub Kicinski
Quoted 20 lines · On Wed, 2 Jun 2021 15:17:13 +0300 [email protected] wrote:
From: Dmytro Linkin <[email protected]>
Resending without RFC.
Currently kernel provides a way to change tx rate of single VF in switchdev mode via tc-police action. When lots of VFs are configured management of theirs rates becomes non-trivial task and some grouping mechanism is required. Implementing such grouping in tc-police will bring flow related limitations and unwanted complications, like:
- tc-police is a policer and there is a user request for a traffic shaper, so shared tc-police action is not suitable;
- flows requires net device to be placed on, means "groups" wouldn't have net device instance itself. Taking into the account previous point was reviewed a sollution, when representor have a policer and the driver use a shaper if qdisc contains group of VFs - such approach ugly, compilated and misleading;
- TC is ingress only, while configuring "other" side of the wire looks more like a "real" picture where shaping is outside of the steering world, similar to "ip link" command;
Quoted 5 lines
Quoted 1 line
According to that devlink is the most appropriate place.
I don't think you researched TC well enough. But whatever, I'm tired of being the only one who pushes back given I neither work on or use any of these features.
tc action offload feature used in [1] seems to solve the police action lifecycle problem?
And it seem to allow different flow to use the same action, I am not sure if different function can use the same action, it seems jianbo has mentioned about the same usecase?
1. https://lore.kernel.org/netdev/CALnP8ZaZQAbvm1girLUSLcFZTKV5MvBMEtN67OiA55OAvsO_1Q@mail.gmail.com/T/