Skip to contentSkip to conversation

devlink: Add SF add/delete devlink ops

PATCH Superseded Parav Pandit 6 participants 53 messages Sep 17, 2020 – Sep 22, 2020 3 messages not in the archive

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover devlink: Add SF add/delete devlink ops

Hi Dave, Jakub,

Similar to PCI VF, PCI SF represents portion of the device. PCI SF is represented using a new devlink port flavour.

This short series implements small part of the RFC described in detail at [1] and [2].

It extends
(a) devlink core to expose new devlink port flavour 'pcisf'.
(b) Expose new user interface to add/delete devlink port.
(c) Extends netdevsim driver to simulate PCI PF and SF ports
(d) Add port function state attribute

Patch summary:
Patch-1 Extends devlink to expose new PCI SF port flavour Patch-2 Extends devlink to let user add, delete devlink Port Patch-3 Prepare code to handle multiple port attributes Patch-4 Extends devlink to let user get, set function state Patch-5 Extends netdevsim driver to simulate PCI PF ports Patch-6 Extends netdevsim driver to simulate hw_addr get/set Patch-7 Extends netdevsim driver to simulate function state get/set Patch-8 Extends netdevsim driver to simulate PCI SF ports

[1] https://lore.kernel.org/netdev/20200519092258.GF4655@nanopsycho/
[2] https://marc.info/?l=linux-netdev&m=158555928517777&w=2

Parav Pandit (8):
devlink: Introduce PCI SF port flavour and port attribute
devlink: Support add and delete devlink port
devlink: Prepare code to fill multiple port function attributes
devlink: Support get and set state of port function
netdevsim: Add support for add and delete of a PCI PF port
netdevsim: Simulate get/set hardware address of a PCI port
netdevsim: Simulate port function state for a PCI port
netdevsim: Add support for add and delete PCI SF port

 drivers/net/netdevsim/Makefile        |   3 +-
 drivers/net/netdevsim/dev.c           |  14 +
 drivers/net/netdevsim/netdevsim.h     |  32 ++
 drivers/net/netdevsim/port_function.c | 498 ++++++++++++++++++++++++++
 include/net/devlink.h                 |  75 ++++
 include/uapi/linux/devlink.h          |  13 +
 net/core/devlink.c                    | 230 ++++++++++--
 7 files changed, 840 insertions(+), 25 deletions(-)
 create mode 100644 drivers/net/netdevsim/port_function.c
Signature · 1 line
2.26.2
#1 lore raw

PATCH 1/8 devlink: Introduce PCI SF port flavour and port attribute

A PCI sub-function (SF) represents a portion of the device similar to PCI VF.

In an eswitch, PCI SF may have port which is normally represented using a representor netdevice.
To have better visibility of eswitch port, its association with SF, and its representor netdevice, introduce a PCI SF port flavour.

When devlink port flavour is PCI SF, fill up PCI SF attributes of the port.

Extend port name creation using PCI PF and SF number scheme on best effort basis, so that vendor drivers can skip defining their own scheme.

An example view of a PCI SF port.

$ devlink port show netdevsim/netdevsim10/2
netdevsim/netdevsim10/2: type eth netdev eni10npf0sf44 flavour pcisf controller 0 pfnum 0 sfnum 44 external false splittable false
  function:
    hw_addr 00:00:00:00:00:00

devlink port show netdevsim/netdevsim10/2 -jp

{
    "port": {
        "netdevsim/netdevsim10/2": {
            "type": "eth",
            "netdev": "eni10npf0sf44",
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": false,
            "splittable": false,
            "function": {
                "hw_addr": "00:00:00:00:00:00"
            }
        }
    }
}
---
 include/net/devlink.h        | 17 +++++++++++++++++
 include/uapi/linux/devlink.h |  7 +++++++
 net/core/devlink.c           | 37 ++++++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+)
include/net/devlink.h +17 -0
index 48b1c1ef1ebd..1edb558125b0 100644
@@ -83,6 +83,20 @@ struct devlink_port_pci_vf_attrs {
 	u8 external:1;
 };
 
+/**
+ * struct devlink_port_pci_sf_attrs - devlink port's PCI SF attributes
+ * @controller: Associated controller number
+ * @pf: Associated PCI PF number for this port.
+ * @sf: Associated PCI SF for of the PCI PF for this port.
+ * @external: when set, indicates if a port is for an external controller
+ */
+struct devlink_port_pci_sf_attrs {
+	u32 controller;
+	u16 pf;
+	u32 sf;
+	u8 external:1;
+};
+
 /**
  * struct devlink_port_attrs - devlink port object
  * @flavour: flavour of the port
@@ -104,6 +118,7 @@ struct devlink_port_attrs {
 		struct devlink_port_phys_attrs phys;
 		struct devlink_port_pci_pf_attrs pci_pf;
 		struct devlink_port_pci_vf_attrs pci_vf;
+		struct devlink_port_pci_sf_attrs pci_sf;
 	};
 };
 
@@ -1230,6 +1245,8 @@ void devlink_port_attrs_pci_pf_set(struct devlink_port *devlink_port, u32 contro
 				   u16 pf, bool external);
 void devlink_port_attrs_pci_vf_set(struct devlink_port *devlink_port, u32 controller,
 				   u16 pf, u16 vf, bool external);
+void devlink_port_attrs_pci_sf_set(struct devlink_port *devlink_port, u32 controller,
+				   u16 pf, u32 sf, bool external);
 int devlink_sb_register(struct devlink *devlink, unsigned int sb_index,
 			u32 size, u16 ingress_pools_count,
 			u16 egress_pools_count, u16 ingress_tc_count,
include/uapi/linux/devlink.h +7 -0
index 631f5bdf1707..09c41b9ce407 100644
@@ -195,6 +195,11 @@ enum devlink_port_flavour {
 				      * port that faces the PCI VF.
 				      */
 	DEVLINK_PORT_FLAVOUR_VIRTUAL, /* Any virtual port facing the user. */
+
+	DEVLINK_PORT_FLAVOUR_PCI_SF, /* Represents eswitch port
+				      * for the PCI SF. It is an internal
+				      * port that faces the PCI SF.
+				      */
 };
 
 enum devlink_param_cmode {
@@ -462,6 +467,8 @@ enum devlink_attr {
 
 	DEVLINK_ATTR_PORT_EXTERNAL,		/* u8 */
 	DEVLINK_ATTR_PORT_CONTROLLER_NUMBER,	/* u32 */
+
+	DEVLINK_ATTR_PORT_PCI_SF_NUMBER,	/* u32 */
 	/* add new attributes above here, update the policy in devlink.c */
 
 	__DEVLINK_ATTR_MAX,
net/core/devlink.c +37 -0
index e5b71f3c2d4d..fada660fd515 100644
@@ -539,6 +539,15 @@ static int devlink_nl_port_attrs_put(struct sk_buff *msg,
 		if (nla_put_u8(msg, DEVLINK_ATTR_PORT_EXTERNAL, attrs->pci_vf.external))
 			return -EMSGSIZE;
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		if (nla_put_u32(msg, DEVLINK_ATTR_PORT_CONTROLLER_NUMBER,
+				attrs->pci_sf.controller) ||
+		    nla_put_u16(msg, DEVLINK_ATTR_PORT_PCI_PF_NUMBER, attrs->pci_sf.pf) ||
+		    nla_put_u32(msg, DEVLINK_ATTR_PORT_PCI_SF_NUMBER, attrs->pci_sf.sf))
+			return -EMSGSIZE;
+		if (nla_put_u8(msg, DEVLINK_ATTR_PORT_EXTERNAL, attrs->pci_vf.external))
+			return -EMSGSIZE;
+		break;
 	case DEVLINK_PORT_FLAVOUR_PHYSICAL:
 	case DEVLINK_PORT_FLAVOUR_CPU:
 	case DEVLINK_PORT_FLAVOUR_DSA:
@@ -7808,6 +7817,31 @@ void devlink_port_attrs_pci_vf_set(struct devlink_port *devlink_port, u32 contro
 }
 EXPORT_SYMBOL_GPL(devlink_port_attrs_pci_vf_set);
 
+/**
+ *	devlink_port_attrs_pci_sf_set - Set PCI SF port attributes
+ *
+ *	@devlink_port: devlink port
+ *	@controller: associated controller number for the devlink port instance
+ *	@pf: associated PF for the devlink port instance
+ *	@sf: associated SF of a PF for the devlink port instance
+ *	@external: indicates if the port is for an external controller
+ */
+void devlink_port_attrs_pci_sf_set(struct devlink_port *devlink_port, u32 controller,
+				   u16 pf, u32 sf, bool external)
+{
+	struct devlink_port_attrs *attrs = &devlink_port->attrs;
+	int ret;
+
+	ret = __devlink_port_attrs_set(devlink_port, DEVLINK_PORT_FLAVOUR_PCI_SF);
+	if (ret)
+		return;
+	attrs->pci_sf.controller = controller;
+	attrs->pci_sf.pf = pf;
+	attrs->pci_sf.sf = sf;
+	attrs->pci_sf.external = external;
+}
+EXPORT_SYMBOL_GPL(devlink_port_attrs_pci_sf_set);
+
 static int __devlink_port_phys_port_name_get(struct devlink_port *devlink_port,
 					     char *name, size_t len)
 {
@@ -7855,6 +7889,9 @@ static int __devlink_port_phys_port_name_get(struct devlink_port *devlink_port,
 		n = snprintf(name, len, "pf%uvf%u",
 			     attrs->pci_vf.pf, attrs->pci_vf.vf);
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		n = snprintf(name, len, "pf%usf%u", attrs->pci_sf.pf, attrs->pci_sf.sf);
+		break;
 	}
 
 	if (n >= len)
Signature · 1 line
2.26.2
#2 lore raw

PATCH 2/8 devlink: Support add and delete devlink port

In reply to Parav Pandit

Extended devlink interface for the user to add and delete port. Extend devlink to connect user requests to driver to add/delete such port in the device.

When driver routines are invoked, devlink instance lock is not held. This enables driver to perform several devlink objects registration, unregistration such as (port, health reporter, resource etc) by using exising devlink APIs.
This also helps to uniformly used the code for port registration during driver unload and during port deletion initiated by user.

Examples of add, show and delete commands:
Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device
$ devlink port show netdevsim/netdevsim10/0
netdevsim/netdevsim10/0: type eth netdev eni10np1 flavour physical port 1 splittable false
$ devlink port add netdevsim/netdevsim10 flavour pcipf pfnum 0
$ devlink port show netdevsim/netdevsim10/1
netdevsim/netdevsim10/1: type eth netdev eni10npf0 flavour pcipf controller 0 pfnum 0 external false splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port show netdevsim/netdevsim10/1 -jp
{
    "port": {
        "netdevsim/netdevsim10/1": {
            "type": "eth",
            "netdev": "eni10npf0",
            "flavour": "pcipf",
            "controller": 0,
            "pfnum": 0,
            "external": false,
            "splittable": false,
            "function": {
                "hw_addr": "00:00:00:00:00:00",
                "state": "inactive"
            }
        }
    }
}
$ devlink port del netdevsim/netdevsim10/1
---
 include/net/devlink.h | 38 ++++++++++++++++++++++++
 net/core/devlink.c    | 67 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 105 insertions(+)
include/net/devlink.h +38 -0
index 1edb558125b0..ebab2c0360d0 100644
@@ -142,6 +142,17 @@ struct devlink_port {
 	struct mutex reporters_lock; /* Protects reporter_list */
 };
 
+struct devlink_port_new_attrs {
+	enum devlink_port_flavour flavour;
+	unsigned int port_index;
+	u32 controller;
+	u32 sfnum;
+	u16 pfnum;
+	u8 port_index_valid:1,
+	   controller_valid:1,
+	   sfnum_valid:1;
+};
+
 struct devlink_sb_pool_info {
 	enum devlink_sb_pool_type pool_type;
 	u32 size;
@@ -1189,6 +1200,33 @@ struct devlink_ops {
 	int (*port_function_hw_addr_set)(struct devlink *devlink, struct devlink_port *port,
 					 const u8 *hw_addr, int hw_addr_len,
 					 struct netlink_ext_ack *extack);
+	/**
+	 * @port_new: Port add function.
+	 *
+	 * Should be used by device driver to let caller add new port of a specified flavour
+	 * with optional attributes.
+	 * Driver should return -EOPNOTSUPP if it doesn't support port addition of a specified
+	 * flavour or specified attributes. Driver should set extack error message in case of fail
+	 * to add the port.
+	 * devlink core does not hold a devlink instance lock when this callback is invoked.
+	 * Driver must ensures synchronization when adding or deleting a port. Driver must
+	 * register a port with devlink core.
+	 */
+	int (*port_new)(struct devlink *devlink, const struct devlink_port_new_attrs *attrs,
+			struct netlink_ext_ack *extack);
+	/**
+	 * @port_del: Port delete function.
+	 *
+	 * Should be used by device driver to let caller delete port which was previously created
+	 * using port_new() callback.
+	 * Driver should return -EOPNOTSUPP if it doesn't support port deletion.
+	 * Driver should set extack error message in case of fail to delete the port.
+	 * devlink core does not hold a devlink instance lock when this callback is invoked.
+	 * Driver must ensures synchronization when adding or deleting a port. Driver must
+	 * register a port with devlink core.
+	 */
+	int (*port_del)(struct devlink *devlink, unsigned int port_index,
+			struct netlink_ext_ack *extack);
 };
 
 static inline void *devlink_priv(struct devlink *devlink)
net/core/devlink.c +67 -0

85 lines not on this page · raw message

Signature · 1 line
2.26.2
#3 lore raw

PATCH 3/8 devlink: Prepare code to fill multiple port function attributes

In reply to Parav Pandit

Prepare code to fill zero or more port function optional attributes. Subsequent patch makes use of this to fill more port function attributes.

---
 net/core/devlink.c | 53 +++++++++++++++++++++++++---------------------
 1 file changed, 29 insertions(+), 24 deletions(-)
net/core/devlink.c +29 -24
index e93730065c57..d152489e48da 100644
@@ -570,6 +570,31 @@ static int devlink_nl_port_attrs_put(struct sk_buff *msg,
 	return 0;
 }
 
+static int
+devlink_port_function_hw_addr_fill(struct devlink *devlink, const struct devlink_ops *ops,
+				   struct devlink_port *port, struct sk_buff *msg,
+				   struct netlink_ext_ack *extack, bool *msg_updated)
+{
+	u8 hw_addr[MAX_ADDR_LEN];
+	int hw_addr_len;
+	int err;
+
+	if (!ops->port_function_hw_addr_get)
+		return 0;
+
+	err = ops->port_function_hw_addr_get(devlink, port, hw_addr, &hw_addr_len, extack);
+	if (err) {
+		if (err == -EOPNOTSUPP)
+			return 0;
+		return err;
+	}
+	err = nla_put(msg, DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR, hw_addr_len, hw_addr);
+	if (err)
+		return err;
+	*msg_updated = true;
+	return 0;
+}
+
 static int
 devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *port,
 				   struct netlink_ext_ack *extack)
@@ -577,36 +602,16 @@ devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *por
 	struct devlink *devlink = port->devlink;
 	const struct devlink_ops *ops;
 	struct nlattr *function_attr;
-	bool empty_nest = true;
-	int err = 0;
+	bool msg_updated = false;
+	int err;
 
 	function_attr = nla_nest_start_noflag(msg, DEVLINK_ATTR_PORT_FUNCTION);
 	if (!function_attr)
 		return -EMSGSIZE;
 
 	ops = devlink->ops;
-	if (ops->port_function_hw_addr_get) {
-		int hw_addr_len;
-		u8 hw_addr[MAX_ADDR_LEN];
-
-		err = ops->port_function_hw_addr_get(devlink, port, hw_addr, &hw_addr_len, extack);
-		if (err == -EOPNOTSUPP) {
-			/* Port function attributes are optional for a port. If port doesn't
-			 * support function attribute, returning -EOPNOTSUPP is not an error.
-			 */
-			err = 0;
-			goto out;
-		} else if (err) {
-			goto out;
-		}
-		err = nla_put(msg, DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR, hw_addr_len, hw_addr);
-		if (err)
-			goto out;
-		empty_nest = false;
-	}
-
-out:
-	if (err || empty_nest)
+	err = devlink_port_function_hw_addr_fill(devlink, ops, port, msg, extack, &msg_updated);
+	if (err || !msg_updated)
 		nla_nest_cancel(msg, function_attr);
 	else
 		nla_nest_end(msg, function_attr);
Signature · 1 line
2.26.2
#4 lore raw

PATCH 4/8 devlink: Support get and set state of port function

In reply to Parav Pandit

devlink port function can be in active or inactive state. Allow users to get and set port function's state.

Example of a PCI SF port which supports a port function:
Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device
$ devlink port show
netdevsim/netdevsim10/0: type eth netdev eni10np1 flavour physical port 1 splittable false
$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0 sfnum 44
$ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour pcisf controller 0 pfnum 0 sfnum 44 external false splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr 00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/11 -jp
{
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0sf44",
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": false,
            "splittable": false,
            "function": {
                "hw_addr": "00:11:22:33:44:55",
                "state": "active"
            }
        }
    }
}
---
 include/net/devlink.h        | 20 ++++++++++
 include/uapi/linux/devlink.h |  6 +++
 net/core/devlink.c           | 77 +++++++++++++++++++++++++++++++++++-
 3 files changed, 101 insertions(+), 2 deletions(-)
include/net/devlink.h +20 -0
index ebab2c0360d0..500c22835686 100644
@@ -1200,6 +1200,26 @@ struct devlink_ops {
 	int (*port_function_hw_addr_set)(struct devlink *devlink, struct devlink_port *port,
 					 const u8 *hw_addr, int hw_addr_len,
 					 struct netlink_ext_ack *extack);
+	/**
+	 * @port_function_state_get: Port function's state get function.
+	 *
+	 * Should be used by device drivers to report the state of a function managed
+	 * by the devlink port. Driver should return -EOPNOTSUPP if it doesn't support port
+	 * function handling for a particular port.
+	 */
+	int (*port_function_state_get)(struct devlink *devlink, struct devlink_port *port,
+				       enum devlink_port_function_state *state,
+				       struct netlink_ext_ack *extack);
+	/**
+	 * @port_function_state_set: Port function's state set function.
+	 *
+	 * Should be used by device drivers to set the state of a function managed
+	 * by the devlink port. Driver should return -EOPNOTSUPP if it doesn't support port
+	 * function handling for a particular port.
+	 */
+	int (*port_function_state_set)(struct devlink *devlink, struct devlink_port *port,
+				       enum devlink_port_function_state state,
+				       struct netlink_ext_ack *extack);
 	/**
 	 * @port_new: Port add function.
 	 *
include/uapi/linux/devlink.h +6 -0
index 09c41b9ce407..8e513f1cd638 100644
@@ -518,9 +518,15 @@ enum devlink_resource_unit {
 enum devlink_port_function_attr {
 	DEVLINK_PORT_FUNCTION_ATTR_UNSPEC,
 	DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR,	/* binary */
+	DEVLINK_PORT_FUNCTION_ATTR_STATE,	/* u8 */
 
 	__DEVLINK_PORT_FUNCTION_ATTR_MAX,
 	DEVLINK_PORT_FUNCTION_ATTR_MAX = __DEVLINK_PORT_FUNCTION_ATTR_MAX - 1
 };
 
+enum devlink_port_function_state {
+	DEVLINK_PORT_FUNCTION_STATE_INACTIVE,
+	DEVLINK_PORT_FUNCTION_STATE_ACTIVE,
+};
+
 #endif /* _UAPI_LINUX_DEVLINK_H_ */
net/core/devlink.c +75 -2

108 lines not on this page · raw message

Signature · 1 line
2.26.2
#5 lore raw

PATCH 5/8 netdevsim: Add support for add and delete of a PCI PF port

In reply to Parav Pandit

Simulate PCI PF ports. Allow user to create one or more PCI PF ports.

Examples:

Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device
$ devlink port show
netdevsim/netdevsim10/0: type eth netdev eni10np1 flavour physical port 1 splittable false

Add and show devlink port of flavour 'pcipf' for PF number 0.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port show netdevsim/netdevsim10/10
netdevsim/netdevsim10/10: type eth netdev eni10npf0 flavour pcipf controller 0 pfnum 0 external false splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive

Delete newly added devlink port

$ devlink port add netdevsim/netdevsim10/10
---
 drivers/net/netdevsim/Makefile        |   3 +-
 drivers/net/netdevsim/dev.c           |  10 +
 drivers/net/netdevsim/netdevsim.h     |  19 ++
 drivers/net/netdevsim/port_function.c | 337 ++++++++++++++++++++++++++
 4 files changed, 368 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/netdevsim/port_function.c
drivers/net/netdevsim/Makefile +2 -1
index ade086eed955..e69e895af62c 100644
@@ -3,7 +3,8 @@
 obj-$(CONFIG_NETDEVSIM) += netdevsim.o
 
 netdevsim-objs := \
-	netdev.o dev.o ethtool.o fib.o bus.o health.o udp_tunnels.o
+	netdev.o dev.o ethtool.o fib.o bus.o health.o udp_tunnels.o \
+	port_function.o
 
 ifeq ($(CONFIG_BPF_SYSCALL),y)
 netdevsim-objs += \
drivers/net/netdevsim/dev.c +10 -0
index 32f339fedb21..e3b81c8b5125 100644
@@ -884,6 +884,8 @@ static const struct devlink_ops nsim_dev_devlink_ops = {
 	.trap_group_set = nsim_dev_devlink_trap_group_set,
 	.trap_policer_set = nsim_dev_devlink_trap_policer_set,
 	.trap_policer_counter_get = nsim_dev_devlink_trap_policer_counter_get,
+	.port_new = nsim_dev_devlink_port_new,
+	.port_del = nsim_dev_devlink_port_del,
 };
 
 #define NSIM_DEV_MAX_MACS_DEFAULT 32
@@ -1017,6 +1019,8 @@ static int nsim_dev_reload_create(struct nsim_dev *nsim_dev,
 						      nsim_dev->ddir,
 						      nsim_dev,
 						&nsim_dev_take_snapshot_fops);
+
+	nsim_dev_port_function_enable(nsim_dev);
 	return 0;
 
 err_health_exit:
@@ -1050,6 +1054,7 @@ int nsim_dev_probe(struct nsim_bus_dev *nsim_bus_dev)
 	nsim_dev->max_macs = NSIM_DEV_MAX_MACS_DEFAULT;
 	nsim_dev->test1 = NSIM_DEV_TEST1_DEFAULT;
 	spin_lock_init(&nsim_dev->fa_cookie_lock);
+	nsim_dev_port_function_init(nsim_dev);
 
 	dev_set_drvdata(&nsim_bus_dev->dev, nsim_dev);
 
@@ -1097,6 +1102,7 @@ int nsim_dev_probe(struct nsim_bus_dev *nsim_bus_dev)
 	if (err)
 		goto err_bpf_dev_exit;
 
+	nsim_dev_port_function_enable(nsim_dev);
 	devlink_params_publish(devlink);
 	devlink_reload_enable(devlink);
 	return 0;
@@ -1131,6 +1137,9 @@ static void nsim_dev_reload_destroy(struct nsim_dev *nsim_dev)
 
 	if (devlink_is_reload_failed(devlink))
 		return;
+
+	/* Disable and destroy any user created devlink ports */
+	nsim_dev_port_function_disable(nsim_dev);
 	debugfs_remove(nsim_dev->take_snapshot);
 	nsim_dev_port_del_all(nsim_dev);
 	nsim_dev_health_exit(nsim_dev);
@@ -1155,6 +1164,7 @@ void nsim_dev_remove(struct nsim_bus_dev *nsim_bus_dev)
 				  ARRAY_SIZE(nsim_devlink_params));
 	devlink_unregister(devlink);
 	devlink_resources_unregister(devlink, NULL);
+	nsim_dev_port_function_exit(nsim_dev);
 	devlink_free(devlink);
 }
 
drivers/net/netdevsim/netdevsim.h +19 -0
index 0c86561e6d8d..aec3c4d5fda7 100644
@@ -213,6 +213,16 @@ struct nsim_dev {
 		bool ipv4_only;
 		u32 sleep;
 	} udp_ports;
+	struct {
+		refcount_t refcount; /* refcount along with disable_complete serializes
+				      * port operations with port function disablement
+				      * during driver unload.
+				      */
+		struct completion disable_complete;
+		struct list_head head;
+		struct ida ida;
+		struct ida pfnum_ida;
+	} port_functions;
 };
 
 static inline struct net *nsim_dev_net(struct nsim_dev *nsim_dev)
@@ -283,3 +293,12 @@ struct nsim_bus_dev {
 
 int nsim_bus_init(void);
 void nsim_bus_exit(void);
+
+void nsim_dev_port_function_init(struct nsim_dev *nsim_dev);
+void nsim_dev_port_function_exit(struct nsim_dev *nsim_dev);
+void nsim_dev_port_function_enable(struct nsim_dev *nsim_dev);
+void nsim_dev_port_function_disable(struct nsim_dev *nsim_dev);
+int nsim_dev_devlink_port_new(struct devlink *devlink, const struct devlink_port_new_attrs *attrs,
+			      struct netlink_ext_ack *extack);
+int nsim_dev_devlink_port_del(struct devlink *devlink, unsigned int port_index,
+			      struct netlink_ext_ack *extack);
drivers/net/netdevsim/port_function.c new file +337 -0

337 lines not on this page · raw message

Signature · 1 line
2.26.2
#6 lore raw

Re: [PATCH net-next 5/8] netdevsim: Add support for add and delete of a PCI PF port

Thank you for the patch! Perhaps something to improve:

Hi Parav,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on net-next/master]

url:    https://github.com/0day-ci/linux/commits/Parav-Pandit/devlink-Add-SF-add-delete-devlink-ops/20200917-162417
base:   https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git b948577b984a01d24d401d2264efbccc7f0146c1
config: i386-randconfig-c003-20200917 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0

If you fix the issue, kindly add following tag as appropriate

coccinelle warnings: (new ones prefixed by >>)

>> drivers/net/netdevsim/port_function.c:122:2-3: Unneeded semicolon

drivers/net/netdevsim/port_function.c:140:2-3: Unneeded semicolon

Please review and possibly fold the followup patch.

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#7 lore raw

RE: [PATCH net-next 5/8] netdevsim: Add support for add and delete of a PCI PF port

Quoted 22 lines

From: kernel test robot <[email protected]>
Sent: Thursday, September 17, 2020 4:46 PM

Hi Parav,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on net-next/master]

url: https://github.com/0day-ci/linux/commits/Parav-Pandit/devlink-Add-
SF-add-delete-devlink-ops/20200917-162417
base: https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git
b948577b984a01d24d401d2264efbccc7f0146c1
config: i386-randconfig-c003-20200917 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0

If you fix the issue, kindly add following tag as appropriate

Quoted 7 lines

coccinelle warnings: (new ones prefixed by >>)

Quoted 1 line
Quoted 1 line

drivers/net/netdevsim/port_function.c:122:2-3: Unneeded semicolon

drivers/net/netdevsim/port_function.c:140:2-3: Unneeded semicolon

Please review and possibly fold the followup patch.

Sending v2 containing the fix.
Thanks.

#8 lore raw

[PATCH] netdevsim: fix semicolon.cocci warnings

In reply to Parav Pandit
drivers/net/netdevsim/port_function.c:122:2-3: Unneeded semicolon drivers/net/netdevsim/port_function.c:140:2-3: Unneeded semicolon

From: kernel test robot <[email protected]>

drivers/net/netdevsim/port_function.c:122:2-3: Unneeded semicolon drivers/net/netdevsim/port_function.c:140:2-3: Unneeded semicolon

Remove unneeded semicolon.

Generated by: scripts/coccinelle/misc/semicolon.cocci

---

url:    https://github.com/0day-ci/linux/commits/Parav-Pandit/devlink-Add-SF-add-delete-devlink-ops/20200917-162417
base:   https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git b948577b984a01d24d401d2264efbccc7f0146c1
 port_function.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/netdevsim/port_function.c +2 -2
@@ -119,7 +119,7 @@ nsim_devlink_port_function_alloc(struct
 		break;
 	default:
 		break;
-	};
+	}
 	return port;
 
 fn_ida_err:
@@ -137,7 +137,7 @@ static void nsim_devlink_port_function_f
 		break;
 	default:
 		break;
-	};
+	}
 	ida_simple_remove(&dev->port_functions.ida, port->port_index);
 	free_netdev(port->netdev);
 }
#9 lore raw

PATCH 6/8 netdevsim: Simulate get/set hardware address of a PCI port

In reply to Parav Pandit

Allow users to get/set hardware address for the PCI port.

Below example creates one devlink port, queries a port, sets a hardware address.

Example of a PCI SF port which supports a port function hw_addr set:
Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device
$ devlink port show
netdevsim/netdevsim10/0: type eth netdev eni10np1 flavour physical port 1 splittable false
$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port show netdevsim/netdevsim10/10
netdevsim/netdevsim10/10: type eth netdev eni10npf0 flavour pcipf controller 0 pfnum 0 external false splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/10 hw_addr 00:11:22:33:44:55
$ devlink port show netdevsim/netdevsim10/10 -jp
{
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0",
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": true,
            "splittable": false,
            "function": {
                "hw_addr": "00:11:22:33:44:55"
            }
        }
    }
}
---
 drivers/net/netdevsim/dev.c           |  2 ++
 drivers/net/netdevsim/netdevsim.h     |  6 ++++
 drivers/net/netdevsim/port_function.c | 44 +++++++++++++++++++++++++++
 3 files changed, 52 insertions(+)
drivers/net/netdevsim/dev.c +2 -0
index e3b81c8b5125..ef2e293f358b 100644
@@ -886,6 +886,8 @@ static const struct devlink_ops nsim_dev_devlink_ops = {
 	.trap_policer_counter_get = nsim_dev_devlink_trap_policer_counter_get,
 	.port_new = nsim_dev_devlink_port_new,
 	.port_del = nsim_dev_devlink_port_del,
+	.port_function_hw_addr_get = nsim_dev_port_function_hw_addr_get,
+	.port_function_hw_addr_set = nsim_dev_port_function_hw_addr_set,
 };
 
 #define NSIM_DEV_MAX_MACS_DEFAULT 32
drivers/net/netdevsim/netdevsim.h +6 -0
index aec3c4d5fda7..8dc8f4e5dcd8 100644
@@ -302,3 +302,9 @@ int nsim_dev_devlink_port_new(struct devlink *devlink, const struct devlink_port
 			      struct netlink_ext_ack *extack);
 int nsim_dev_devlink_port_del(struct devlink *devlink, unsigned int port_index,
 			      struct netlink_ext_ack *extack);
+int nsim_dev_port_function_hw_addr_get(struct devlink *devlink, struct devlink_port *port,
+				       u8 *hw_addr, int *hw_addr_len,
+				       struct netlink_ext_ack *extack);
+int nsim_dev_port_function_hw_addr_set(struct devlink *devlink, struct devlink_port *port,
+				       const u8 *hw_addr, int hw_addr_len,
+				       struct netlink_ext_ack *extack);
drivers/net/netdevsim/port_function.c +44 -0

53 lines not on this page · raw message

Signature · 1 line
2.26.2
#10 lore raw

PATCH 7/8 netdevsim: Simulate port function state for a PCI port

In reply to Parav Pandit

Simulate port function state of a PCI port.
This enables users to get and set the state of the PCI port function.

Example of a PCI SF port which supports a port function:
Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device
$ devlink port show
netdevsim/netdevsim10/0: type eth netdev eni10np1 flavour physical port 1 splittable false
$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port function set netdevsim/netdevsim10/10 hw_addr 00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/10
netdevsim/netdevsim10/10: type eth netdev eni10npf0 flavour pcipf controller 0 pfnum 0 external true splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/10 hw_addr 00:11:22:33:44:55 state active
$ devlink port show  netdevsim/netdevsim10/10 -jp
{
    "port": {
        "netdevsim/netdevsim10/10": {
            "type": "eth",
            "netdev": "eni10npf0",
            "flavour": "pcipf",
            "controller": 0,
            "pfnum": 0,
            "external": false,
            "splittable": false,
            "function": {
                "hw_addr": "00:11:22:33:44:55",
                "state": "active"
            }
        }
    }
}
---
 drivers/net/netdevsim/dev.c           |  2 ++
 drivers/net/netdevsim/netdevsim.h     |  6 ++++++
 drivers/net/netdevsim/port_function.c | 30 +++++++++++++++++++++++++++
 3 files changed, 38 insertions(+)
drivers/net/netdevsim/dev.c +2 -0
index ef2e293f358b..ec1e5dc74be1 100644
@@ -888,6 +888,8 @@ static const struct devlink_ops nsim_dev_devlink_ops = {
 	.port_del = nsim_dev_devlink_port_del,
 	.port_function_hw_addr_get = nsim_dev_port_function_hw_addr_get,
 	.port_function_hw_addr_set = nsim_dev_port_function_hw_addr_set,
+	.port_function_state_get = nsim_dev_port_function_state_get,
+	.port_function_state_set = nsim_dev_port_function_state_set,
 };
 
 #define NSIM_DEV_MAX_MACS_DEFAULT 32
drivers/net/netdevsim/netdevsim.h +6 -0
index 8dc8f4e5dcd8..0ea9705eda38 100644
@@ -308,3 +308,9 @@ int nsim_dev_port_function_hw_addr_get(struct devlink *devlink, struct devlink_p
 int nsim_dev_port_function_hw_addr_set(struct devlink *devlink, struct devlink_port *port,
 				       const u8 *hw_addr, int hw_addr_len,
 				       struct netlink_ext_ack *extack);
+int nsim_dev_port_function_state_get(struct devlink *devlink, struct devlink_port *port,
+				     enum devlink_port_function_state *state,
+				     struct netlink_ext_ack *extack);
+int nsim_dev_port_function_state_set(struct devlink *devlink, struct devlink_port *port,
+				     enum devlink_port_function_state state,
+				     struct netlink_ext_ack *extack);
drivers/net/netdevsim/port_function.c +30 -0
index 0053f6f6d530..01587b54f0e0 100644
@@ -16,6 +16,7 @@ struct nsim_port_function {
 	u16 pfnum;
 	struct nsim_port_function *pf_port; /* Valid only for SF port */
 	u8 hw_addr[ETH_ALEN];
+	u8 state; /* enum devlink_port_function_state */
 };
 
 void nsim_dev_port_function_init(struct nsim_dev *nsim_dev)
@@ -196,6 +197,7 @@ static int nsim_devlink_port_function_add(struct devlink *devlink, struct nsim_d
 
 	list_add(&port->list, &nsim_dev->port_functions.head);
 
+	port->state = DEVLINK_PORT_FUNCTION_STATE_INACTIVE;
 	err = devlink_port_register(devlink, &port->dl_port, port->port_index);
 	if (err)
 		goto reg_err;
@@ -379,3 +381,31 @@ int nsim_dev_port_function_hw_addr_set(struct devlink *devlink, struct devlink_p
 	memcpy(port->hw_addr, hw_addr, ETH_ALEN);
 	return 0;
 }
+
+int nsim_dev_port_function_state_get(struct devlink *devlink, struct devlink_port *dl_port,
+				     enum devlink_port_function_state *state,
+				     struct netlink_ext_ack *extack)
+{
+	struct nsim_dev *nsim_dev = devlink_priv(devlink);
+	struct nsim_port_function *port;
+
+	port = nsim_dev_to_port_function(nsim_dev, dl_port);
+	if (IS_ERR(port))
+		return PTR_ERR(port);
+	*state = port->state;
+	return 0;
+}
+
+int nsim_dev_port_function_state_set(struct devlink *devlink, struct devlink_port *dl_port,
+				     enum devlink_port_function_state state,
+				     struct netlink_ext_ack *extack)
+{
+	struct nsim_dev *nsim_dev = devlink_priv(devlink);
+	struct nsim_port_function *port;
+
+	port = nsim_dev_to_port_function(nsim_dev, dl_port);
+	if (IS_ERR(port))
+		return PTR_ERR(port);
+	port->state = state;
+	return 0;
+}
Signature · 1 line
2.26.2
#11 lore raw

v2 cover devlink: Add SF add/delete devlink ops

Hi Dave, Jakub,

Similar to PCI VF, PCI SF represents portion of the device. PCI SF is represented using a new devlink port flavour.

This short series implements small part of the RFC described in detail at [1] and [2].

It extends
(a) devlink core to expose new devlink port flavour 'pcisf'.
(b) Expose new user interface to add/delete devlink port.
(c) Extends netdevsim driver to simulate PCI PF and SF ports
(d) Add port function state attribute

Patch summary:
Patch-1 Extends devlink to expose new PCI SF port flavour Patch-2 Extends devlink to let user add, delete devlink Port Patch-3 Prepare code to handle multiple port attributes Patch-4 Extends devlink to let user get, set function state Patch-5 Extends netdevsim driver to simulate PCI PF ports Patch-6 Extends netdevsim driver to simulate hw_addr get/set Patch-7 Extends netdevsim driver to simulate function state get/set Patch-8 Extends netdevsim driver to simulate PCI SF ports

[1] https://lore.kernel.org/netdev/20200519092258.GF4655@nanopsycho/
[2] https://marc.info/?l=linux-netdev&m=158555928517777&w=2

---
Changelog:
v1->v2:
- Fixed extra semicolon at end of switch case reportec by coccinelle

Parav Pandit (8):
devlink: Introduce PCI SF port flavour and port attribute
devlink: Support add and delete devlink port
devlink: Prepare code to fill multiple port function attributes
devlink: Support get and set state of port function
netdevsim: Add support for add and delete of a PCI PF port
netdevsim: Simulate get/set hardware address of a PCI port
netdevsim: Simulate port function state for a PCI port
netdevsim: Add support for add and delete PCI SF port

 drivers/net/netdevsim/Makefile        |   3 +-
 drivers/net/netdevsim/dev.c           |  14 +
 drivers/net/netdevsim/netdevsim.h     |  32 ++
 drivers/net/netdevsim/port_function.c | 498 ++++++++++++++++++++++++++
 include/net/devlink.h                 |  75 ++++
 include/uapi/linux/devlink.h          |  13 +
 net/core/devlink.c                    | 230 ++++++++++--
 7 files changed, 840 insertions(+), 25 deletions(-)
 create mode 100644 drivers/net/netdevsim/port_function.c
Signature · 1 line
2.26.2
#12 lore raw

PATCH v2 1/8 devlink: Introduce PCI SF port flavour and port attribute

A PCI sub-function (SF) represents a portion of the device similar to PCI VF.

In an eswitch, PCI SF may have port which is normally represented using a representor netdevice.
To have better visibility of eswitch port, its association with SF, and its representor netdevice, introduce a PCI SF port flavour.

When devlink port flavour is PCI SF, fill up PCI SF attributes of the port.

Extend port name creation using PCI PF and SF number scheme on best effort basis, so that vendor drivers can skip defining their own scheme.

An example view of a PCI SF port.

$ devlink port show netdevsim/netdevsim10/2
netdevsim/netdevsim10/2: type eth netdev eni10npf0sf44 flavour pcisf controller 0 pfnum 0 sfnum 44 external false splittable false
  function:
    hw_addr 00:00:00:00:00:00

devlink port show netdevsim/netdevsim10/2 -jp

{
    "port": {
        "netdevsim/netdevsim10/2": {
            "type": "eth",
            "netdev": "eni10npf0sf44",
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": false,
            "splittable": false,
            "function": {
                "hw_addr": "00:00:00:00:00:00"
            }
        }
    }
}
---
 include/net/devlink.h        | 17 +++++++++++++++++
 include/uapi/linux/devlink.h |  7 +++++++
 net/core/devlink.c           | 37 ++++++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+)
include/net/devlink.h +17 -0
index 48b1c1ef1ebd..1edb558125b0 100644
@@ -83,6 +83,20 @@ struct devlink_port_pci_vf_attrs {
 	u8 external:1;
 };
 
+/**
+ * struct devlink_port_pci_sf_attrs - devlink port's PCI SF attributes
+ * @controller: Associated controller number
+ * @pf: Associated PCI PF number for this port.
+ * @sf: Associated PCI SF for of the PCI PF for this port.
+ * @external: when set, indicates if a port is for an external controller
+ */
+struct devlink_port_pci_sf_attrs {
+	u32 controller;
+	u16 pf;
+	u32 sf;
+	u8 external:1;
+};
+
 /**
  * struct devlink_port_attrs - devlink port object
  * @flavour: flavour of the port
@@ -104,6 +118,7 @@ struct devlink_port_attrs {
 		struct devlink_port_phys_attrs phys;
 		struct devlink_port_pci_pf_attrs pci_pf;
 		struct devlink_port_pci_vf_attrs pci_vf;
+		struct devlink_port_pci_sf_attrs pci_sf;
 	};
 };
 
@@ -1230,6 +1245,8 @@ void devlink_port_attrs_pci_pf_set(struct devlink_port *devlink_port, u32 contro
 				   u16 pf, bool external);
 void devlink_port_attrs_pci_vf_set(struct devlink_port *devlink_port, u32 controller,
 				   u16 pf, u16 vf, bool external);
+void devlink_port_attrs_pci_sf_set(struct devlink_port *devlink_port, u32 controller,
+				   u16 pf, u32 sf, bool external);
 int devlink_sb_register(struct devlink *devlink, unsigned int sb_index,
 			u32 size, u16 ingress_pools_count,
 			u16 egress_pools_count, u16 ingress_tc_count,
include/uapi/linux/devlink.h +7 -0
index 631f5bdf1707..09c41b9ce407 100644
@@ -195,6 +195,11 @@ enum devlink_port_flavour {
 				      * port that faces the PCI VF.
 				      */
 	DEVLINK_PORT_FLAVOUR_VIRTUAL, /* Any virtual port facing the user. */
+
+	DEVLINK_PORT_FLAVOUR_PCI_SF, /* Represents eswitch port
+				      * for the PCI SF. It is an internal
+				      * port that faces the PCI SF.
+				      */
 };
 
 enum devlink_param_cmode {
@@ -462,6 +467,8 @@ enum devlink_attr {
 
 	DEVLINK_ATTR_PORT_EXTERNAL,		/* u8 */
 	DEVLINK_ATTR_PORT_CONTROLLER_NUMBER,	/* u32 */
+
+	DEVLINK_ATTR_PORT_PCI_SF_NUMBER,	/* u32 */
 	/* add new attributes above here, update the policy in devlink.c */
 
 	__DEVLINK_ATTR_MAX,
net/core/devlink.c +37 -0
index e5b71f3c2d4d..fada660fd515 100644
@@ -539,6 +539,15 @@ static int devlink_nl_port_attrs_put(struct sk_buff *msg,
 		if (nla_put_u8(msg, DEVLINK_ATTR_PORT_EXTERNAL, attrs->pci_vf.external))
 			return -EMSGSIZE;
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		if (nla_put_u32(msg, DEVLINK_ATTR_PORT_CONTROLLER_NUMBER,
+				attrs->pci_sf.controller) ||
+		    nla_put_u16(msg, DEVLINK_ATTR_PORT_PCI_PF_NUMBER, attrs->pci_sf.pf) ||
+		    nla_put_u32(msg, DEVLINK_ATTR_PORT_PCI_SF_NUMBER, attrs->pci_sf.sf))
+			return -EMSGSIZE;
+		if (nla_put_u8(msg, DEVLINK_ATTR_PORT_EXTERNAL, attrs->pci_vf.external))
+			return -EMSGSIZE;
+		break;
 	case DEVLINK_PORT_FLAVOUR_PHYSICAL:
 	case DEVLINK_PORT_FLAVOUR_CPU:
 	case DEVLINK_PORT_FLAVOUR_DSA:
@@ -7808,6 +7817,31 @@ void devlink_port_attrs_pci_vf_set(struct devlink_port *devlink_port, u32 contro
 }
 EXPORT_SYMBOL_GPL(devlink_port_attrs_pci_vf_set);
 
+/**
+ *	devlink_port_attrs_pci_sf_set - Set PCI SF port attributes
+ *
+ *	@devlink_port: devlink port
+ *	@controller: associated controller number for the devlink port instance
+ *	@pf: associated PF for the devlink port instance
+ *	@sf: associated SF of a PF for the devlink port instance
+ *	@external: indicates if the port is for an external controller
+ */
+void devlink_port_attrs_pci_sf_set(struct devlink_port *devlink_port, u32 controller,
+				   u16 pf, u32 sf, bool external)
+{
+	struct devlink_port_attrs *attrs = &devlink_port->attrs;
+	int ret;
+
+	ret = __devlink_port_attrs_set(devlink_port, DEVLINK_PORT_FLAVOUR_PCI_SF);
+	if (ret)
+		return;
+	attrs->pci_sf.controller = controller;
+	attrs->pci_sf.pf = pf;
+	attrs->pci_sf.sf = sf;
+	attrs->pci_sf.external = external;
+}
+EXPORT_SYMBOL_GPL(devlink_port_attrs_pci_sf_set);
+
 static int __devlink_port_phys_port_name_get(struct devlink_port *devlink_port,
 					     char *name, size_t len)
 {
@@ -7855,6 +7889,9 @@ static int __devlink_port_phys_port_name_get(struct devlink_port *devlink_port,
 		n = snprintf(name, len, "pf%uvf%u",
 			     attrs->pci_vf.pf, attrs->pci_vf.vf);
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		n = snprintf(name, len, "pf%usf%u", attrs->pci_sf.pf, attrs->pci_sf.sf);
+		break;
 	}
 
 	if (n >= len)
Signature · 1 line
2.26.2
#13 lore raw

Re: [PATCH net-next v2 1/8] devlink: Introduce PCI SF port flavour and port attribute

Quoted 9 lines · Parav Pandit
include/net/devlink.h +1 -0
index 48b1c1ef1ebd..1edb558125b0 100644
@@ -83,6 +83,20 @@ struct devlink_port_pci_vf_attrs {
 	u8 external:1;
 };
 
+/**

include/net/devlink.h struct devlink_port_pci_vf_attrs { · line 96

Quoted 10 lines
include/net/devlink.h +10 -0
+ * struct devlink_port_pci_sf_attrs - devlink port's PCI SF attributes
+ * @controller: Associated controller number
+ * @pf: Associated PCI PF number for this port.
+ * @sf: Associated PCI SF for of the PCI PF for this port.
+ * @external: when set, indicates if a port is for an external controller
+ */
+struct devlink_port_pci_sf_attrs {
+	u32 controller;
+	u16 pf;
+	u32 sf;

Why a u32? Do you expect to support that many SFs? Seems like even a u16 is more than you can adequately name within an IFNAMESZ buffer.

Quoted 14 lines
include/net/devlink.h +3 -0
+	u8 external:1;
+};
+
 /**
  * struct devlink_port_attrs - devlink port object
  * @flavour: flavour of the port
net/core/devlink.c +0 -0
index e5b71f3c2d4d..fada660fd515 100644

net/core/devlink.c static int __devlink_port_phys_port_name_get(struct devlink_port *devlink_port, · line 7897

Quoted 9 lines
net/core/devlink.c +3 -0
@@ -7855,6 +7889,9 @@ static int __devlink_port_phys_port_name_get(struct devlink_port *devlink_port,
 		n = snprintf(name, len, "pf%uvf%u",
 			     attrs->pci_vf.pf, attrs->pci_vf.vf);
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		n = snprintf(name, len, "pf%usf%u", attrs->pci_sf.pf, attrs->pci_sf.sf);
+		break;
 	}
 
 	if (n >= len)

And as I noted before, this function continues to grow device names and it is going to spill over the IFNAMESZ buffer and EINVAL is going to be confusing. It really needs better error handling back to users (not kernel buffers).

#14 lore raw

RE: [PATCH net-next v2 1/8] devlink: Introduce PCI SF port flavour and port attribute

Quoted 13 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 1:32 AM

Quoted 8 lines · Parav Pandit
include/net/devlink.h index renamed from include/net/devlink.h +0 -0

48b1c1ef1ebd..1edb558125b0 100644

include/net/devlink.h +0 -0
@@ -83,6 +83,20 @@ struct devlink_port_pci_vf_attrs {
 	u8 external:1;
 };
 
Quoted 15 lines

include/net/devlink.h struct devlink_port_pci_vf_attrs { · line 96

Quoted 11 lines
include/net/devlink.h +11 -0
+/**
+ * struct devlink_port_pci_sf_attrs - devlink port's PCI SF
+attributes
+ * @controller: Associated controller number
+ * @pf: Associated PCI PF number for this port.
+ * @sf: Associated PCI SF for of the PCI PF for this port.
+ * @external: when set, indicates if a port is for an external
+controller  */ struct devlink_port_pci_sf_attrs {
+	u32 controller;
+	u16 pf;
+	u32 sf;

Why a u32? Do you expect to support that many SFs? Seems like even a u16 is more than you can adequately name within an IFNAMESZ buffer.

I think u16 is likely enough, which let use creates 64K SF ports which is a lot. :-)
Was little concerned that it shouldn't fall short in few years. So picked u32.
Users will be able to make use of alternative names so just because IFNAMESZ is 16 characters, do not want to limit sfnum size. What do you think?

Quoted 24 lines

net/core/devlink.c static int

Quoted 14 lines
include/net/devlink.h +3 -0
+	u8 external:1;
+};
+
 /**
  * struct devlink_port_attrs - devlink port object
  * @flavour: flavour of the port
net/core/devlink.c index renamed from net/core/devlink.c +0 -0

e5b71f3c2d4d..fada660fd515 100644

net/core/devlink.c +0 -0
@@ -7855,6 +7889,9 @@ static int

__devlink_port_phys_port_name_get(struct devlink_port *devlink_port,

Quoted 9 lines
net/core/devlink.c +3 -0
 		n = snprintf(name, len, "pf%uvf%u",
 			     attrs->pci_vf.pf, attrs->pci_vf.vf);
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		n = snprintf(name, len, "pf%usf%u", attrs->pci_sf.pf, attrs-
 pci_sf.sf);
+		break;
 	}
 
Quoted 6 lines
Quoted 2 lines

if (n >= len)

And as I noted before, this function continues to grow device names and it is going to spill over the IFNAMESZ buffer and EINVAL is going to be confusing. It really needs better error handling back to users (not kernel buffers).

Alternative names [1] should help to overcome the limitation of IFNAMESZ. For error code EINVAL, should it be ENOSPC?
If so, should I insert a pre-patch in this series?

[1] ip link property add dev DEVICE [ altname NAME .. ]

#15 lore raw

Re: [PATCH net-next v2 1/8] devlink: Introduce PCI SF port flavour and port attribute

Quoted 63 lines · Parav Pandit
Quoted 13 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 1:32 AM

Quoted 8 lines · Parav Pandit
include/net/devlink.h index renamed from include/net/devlink.h +0 -0

48b1c1ef1ebd..1edb558125b0 100644

include/net/devlink.h +0 -0
@@ -83,6 +83,20 @@ struct devlink_port_pci_vf_attrs {
 	u8 external:1;
 };
 
Quoted 15 lines

include/net/devlink.h struct devlink_port_pci_vf_attrs { · line 96

Quoted 11 lines
include/net/devlink.h +11 -0
+/**
+ * struct devlink_port_pci_sf_attrs - devlink port's PCI SF
+attributes
+ * @controller: Associated controller number
+ * @pf: Associated PCI PF number for this port.
+ * @sf: Associated PCI SF for of the PCI PF for this port.
+ * @external: when set, indicates if a port is for an external
+controller  */ struct devlink_port_pci_sf_attrs {
+	u32 controller;
+	u16 pf;
+	u32 sf;

Why a u32? Do you expect to support that many SFs? Seems like even a u16 is more than you can adequately name within an IFNAMESZ buffer.

I think u16 is likely enough, which let use creates 64K SF ports which is a lot. :-)
Was little concerned that it shouldn't fall short in few years. So picked u32.
Users will be able to make use of alternative names so just because IFNAMESZ is 16 characters, do not want to limit sfnum size. What do you think?

Quoted 24 lines

net/core/devlink.c static int

Quoted 14 lines
include/net/devlink.h +3 -0
+	u8 external:1;
+};
+
 /**
  * struct devlink_port_attrs - devlink port object
  * @flavour: flavour of the port
net/core/devlink.c index renamed from net/core/devlink.c +0 -0

e5b71f3c2d4d..fada660fd515 100644

net/core/devlink.c +0 -0
@@ -7855,6 +7889,9 @@ static int

__devlink_port_phys_port_name_get(struct devlink_port *devlink_port,

Quoted 9 lines
net/core/devlink.c +3 -0
 		n = snprintf(name, len, "pf%uvf%u",
 			     attrs->pci_vf.pf, attrs->pci_vf.vf);
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		n = snprintf(name, len, "pf%usf%u", attrs->pci_sf.pf, attrs-
 pci_sf.sf);
+		break;
 	}
 
Quoted 6 lines
Quoted 2 lines

if (n >= len)

And as I noted before, this function continues to grow device names and it is going to spill over the IFNAMESZ buffer and EINVAL is going to be confusing. It really needs better error handling back to users (not kernel buffers).

Quoted 6 lines

Alternative names [1] should help to overcome the limitation of IFNAMESZ. For error code EINVAL, should it be ENOSPC?
If so, should I insert a pre-patch in this series?

[1] ip link property add dev DEVICE [ altname NAME .. ]

You keep adding patches that extend the template based names. Those are going to cause odd EINVAL failures (the absolute worst kind of configuration failure) with no way for a user to understand why the command is failing, and you need to handle that. Returning an extack message back to the user is preferred.

Yes, the altnames provides a solution after the the netdevice has been created, but I do not see how that works when the netdevice is created as part of devlink commands using the template names approach.

#16 lore raw

RE: [PATCH net-next v2 1/8] devlink: Introduce PCI SF port flavour and port attribute

Hi David,

Quoted 74 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 8:45 PM

Quoted 28 lines · Parav Pandit
Quoted 13 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 1:32 AM

Quoted 8 lines · Parav Pandit
include/net/devlink.h index renamed from include/net/devlink.h +0 -0

48b1c1ef1ebd..1edb558125b0 100644

include/net/devlink.h +0 -0
@@ -83,6 +83,20 @@ struct devlink_port_pci_vf_attrs {
 	u8 external:1;
 };
 
Quoted 15 lines

include/net/devlink.h struct devlink_port_pci_vf_attrs { · line 96

Quoted 11 lines
include/net/devlink.h +11 -0
+/**
+ * struct devlink_port_pci_sf_attrs - devlink port's PCI SF
+attributes
+ * @controller: Associated controller number
+ * @pf: Associated PCI PF number for this port.
+ * @sf: Associated PCI SF for of the PCI PF for this port.
+ * @external: when set, indicates if a port is for an external
+controller  */ struct devlink_port_pci_sf_attrs {
+	u32 controller;
+	u16 pf;
+	u32 sf;

Why a u32? Do you expect to support that many SFs? Seems like even a u16 is more than you can adequately name within an IFNAMESZ buffer.

Quoted 2 lines

I think u16 is likely enough, which let use creates 64K SF ports which is a lot. :-) Was little concerned that it shouldn't fall short in few years. So

picked u32.

Quoted 1 line

Users will be able to make use of alternative names so just because

IFNAMESZ is 16 characters, do not want to limit sfnum size.

Quoted 32 lines

What do you think?

Quoted 30 lines

net/core/devlink.c static int

Quoted 14 lines
include/net/devlink.h +3 -0
+	u8 external:1;
+};
+
 /**
  * struct devlink_port_attrs - devlink port object
  * @flavour: flavour of the port
net/core/devlink.c index renamed from net/core/devlink.c +0 -0

e5b71f3c2d4d..fada660fd515 100644

net/core/devlink.c +0 -0
@@ -7855,6 +7889,9 @@ static int

__devlink_port_phys_port_name_get(struct devlink_port *devlink_port,

Quoted 11 lines
net/core/devlink.c +3 -0
 		n = snprintf(name, len, "pf%uvf%u",
 			     attrs->pci_vf.pf, attrs->pci_vf.vf);
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		n = snprintf(name, len, "pf%usf%u", attrs->pci_sf.pf, attrs-
 pci_sf.sf);
+		break;
 	}
 

if (n >= len)

And as I noted before, this function continues to grow device names and it is going to spill over the IFNAMESZ buffer and EINVAL is going to be confusing. It really needs better error handling back to users (not

kernel buffers).

Quoted 4 lines

Alternative names [1] should help to overcome the limitation of IFNAMESZ. For error code EINVAL, should it be ENOSPC?
If so, should I insert a pre-patch in this series?

Quoted 8 lines
Quoted 2 lines

[1] ip link property add dev DEVICE [ altname NAME .. ]

You keep adding patches that extend the template based names. Those are going to cause odd EINVAL failures (the absolute worst kind of configuration failure) with no way for a user to understand why the command is failing, and you need to handle that. Returning an extack message back to the user is preferred.

Sure, make sense.
I will run one short series after this one, to return extack in below code flow and other where extack return is possible. In sysfs flow it is irrelevant anyway.
rtnl_getlink()
rtnl_phys_port_name_fill()
dev_get_phys_port_name()
ndo_phys_port_name()

is that ok?
This series is not really making phys_port_name any worse except that sfnum field width is bit larger than vfnum.

Now coming back to phys_port_name not fitting in 15 characters which can trigger -EINVAL error is very slim in most sane cases. Let's take an example.
A controller in valid range of 0 to 16,
PF in range of 0 to 7,
VF in range of 0 to 255,
SF in range of 0 to 65536,

Will generate VF phys_port_name=c16pf7vf255 (11 chars + 1 null) SF phys_port name = c16pf7sf65535 (13 chars + 1 null)
So yes, eth dev name won't fit in but phys_port_name failure is almost nil.

Quoted 4 lines

Yes, the altnames provides a solution after the the netdevice has been created, but I do not see how that works when the netdevice is created as part of devlink commands using the template names approach.

systemd/udev version v245 understands the parent PCI device of netdev and phys_port_name to construct a unique netdevice name.

This is the example from this patch series for PF port.

udevadm test-builtin net_id /sys/class/net/eth0
Load module index
Parsed configuration file /usr/lib/systemd/network/99-default.link Created link configuration context.
Using default interface naming scheme 'v245'. ID_NET_NAMING_SCHEME=v245
ID_NET_NAME_PATH=eni10npf0
Unload module index
Unloaded link configuration context.

And for SF port,

udevadm test-builtin net_id /sys/class/net/eth1
Load module index
Parsed configuration file /usr/lib/systemd/network/99-default.link Created link configuration context.
Using default interface naming scheme 'v245'. ID_NET_NAMING_SCHEME=v245 ID_NET_NAME_PATH=eni10npf0sf44
Unload module index
Unloaded link configuration context.

Similar VF naming in places for I think more than a year now.

#17 lore raw

Re: [PATCH net-next v2 1/8] devlink: Introduce PCI SF port flavour and port attribute

Quoted 8 lines · Parav Pandit
Quoted 5 lines

You keep adding patches that extend the template based names. Those are going to cause odd EINVAL failures (the absolute worst kind of configuration failure) with no way for a user to understand why the command is failing, and you need to handle that. Returning an extack message back to the user is preferred.

Sure, make sense.
I will run one short series after this one, to return extack in below code flow and other where extack return is possible. In sysfs flow it is irrelevant anyway.

yes, sysfs is a different problem.

Quoted 6 lines

rtnl_getlink()
rtnl_phys_port_name_fill()
dev_get_phys_port_name()
ndo_phys_port_name()

is that ok?

sure. The overflow is not going to happen today with 1-10 SFs; but you are pushing ever close to the limit hence the push.

Quoted 9 lines

This series is not really making phys_port_name any worse except that sfnum field width is bit larger than vfnum.

Now coming back to phys_port_name not fitting in 15 characters which can trigger -EINVAL error is very slim in most sane cases. Let's take an example.
A controller in valid range of 0 to 16,
PF in range of 0 to 7,
VF in range of 0 to 255,
SF in range of 0 to 65536,

Quoted 4 lines

Will generate VF phys_port_name=c16pf7vf255 (11 chars + 1 null) SF phys_port name = c16pf7sf65535 (13 chars + 1 null)
So yes, eth dev name won't fit in but phys_port_name failure is almost nil.

You lost me on that last sentence. Per your example for SF, adding 'eni' or 'eth' as a prefix (which you commonly use in examples and which are common prefixes) to that name and you overflow the IFNAMESZ limit.

(understand about the systemd integration and appreciate the forward thinking about that).

#18 lore raw

RE: [PATCH net-next v2 1/8] devlink: Introduce PCI SF port flavour and port attribute

Quoted 24 lines

From: David Ahern <[email protected]>
Sent: Saturday, September 19, 2020 10:19 AM

Quoted 8 lines · Parav Pandit
Quoted 6 lines

You keep adding patches that extend the template based names. Those are going to cause odd EINVAL failures (the absolute worst kind of configuration
failure) with no way for a user to understand why the command is failing, and you need to handle that. Returning an extack message back to the user is preferred.

Sure, make sense.
I will run one short series after this one, to return extack in below code flow

and other where extack return is possible.

Quoted 1 line

In sysfs flow it is irrelevant anyway.

yes, sysfs is a different problem.

Quoted 5 lines

rtnl_getlink()
rtnl_phys_port_name_fill()
dev_get_phys_port_name()
ndo_phys_port_name()

Quoted 5 lines
Quoted 1 line

is that ok?

sure. The overflow is not going to happen today with 1-10 SFs; but you are pushing ever close to the limit hence the push.

Ok. yeah. got it.

Quoted 10 lines
Quoted 2 lines

This series is not really making phys_port_name any worse except that sfnum

field width is bit larger than vfnum.

Quoted 2 lines

Now coming back to phys_port_name not fitting in 15 characters which can

trigger -EINVAL error is very slim in most sane cases.

Quoted 4 lines

Let's take an example.
A controller in valid range of 0 to 16, PF in range of 0 to 7, VF in range of 0 to 255, SF in range of 0 to 65536,

Quoted 9 lines
Quoted 4 lines

Will generate VF phys_port_name=c16pf7vf255 (11 chars + 1 null) SF phys_port name = c16pf7sf65535 (13 chars + 1 null) So yes, eth dev name won't fit in but phys_port_name failure is almost nil.

You lost me on that last sentence. Per your example for SF, adding 'eni' or 'eth' as a prefix (which you commonly use in examples and which are common prefixes) to that name and you overflow the IFNAMESZ limit.

Right. Got you.
Will run short series for returning extack.

Quoted 3 lines

(understand about the systemd integration and appreciate the forward thinking about that).

Thank you David.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#20 lore raw

RE: [PATCH net-next v2 1/8] devlink: Introduce PCI SF port flavour and port attribute

Quoted 15 lines

From: Jacob Keller <[email protected]>
Sent: Friday, September 18, 2020 12:00 AM

Quoted 8 lines · Parav Pandit

A PCI sub-function (SF) represents a portion of the device similar to PCI VF.

In an eswitch, PCI SF may have port which is normally represented using a representor netdevice.
To have better visibility of eswitch port, its association with SF, and its representor netdevice, introduce a PCI SF port flavour.

Quoted 9 lines
Quoted 6 lines

When devlink port flavour is PCI SF, fill up PCI SF attributes of the port.

Extend port name creation using PCI PF and SF number scheme on best effort basis, so that vendor drivers can skip defining their own scheme.

What does this mean? What's the scheme used?

Scheme used is equivalent as what is used for PCI VF ports. pfNvfM.
It is pfNsfM.
Below example shows the representor netdevice name as 'eni10npf0sf44' built by systemd/udev using phys_port_name.

Quoted 1 line

Do drivers still have the option to make their own scheme? If so, why?

Today we have two types of drivers (mlx5_core, netdevsim) which uses devlink core which creates the name.
Or other drivers (bnxt, nfp) which doesn't yet migrated to use devlink infra for PCI PF, VF ports.
Such drivers are phys_port_name and other ndos.
It is not the role of this patch to block those drivers, but any new implementation doesn't need to hand code switch_id and phys_port_name related ndos for SF. For example, bnxt_vf_rep_get_phys_port_name().

Quoted 3 lines

It's not obvious to me in this patch where the numbering scheme comes from. It looks like it's still up to the caller to set the numbers.

Naming scheme for PCI PF and PCI VF port flavours already exist. Scheme is equivalent for PCI SF flavour.

I thought example is good enough to show that, but I will update commit message to describe this scheme to make it clear. pfNsfM.

Quoted 43 lines
Quoted 5 lines

An example view of a PCI SF port.

$ devlink port show netdevsim/netdevsim10/2
netdevsim/netdevsim10/2: type eth netdev eni10npf0sf44 flavour pcisf

controller 0 pfnum 0 sfnum 44 external false splittable false

Quoted 31 lines

function:

    hw_addr 00:00:00:00:00:00
devlink port show netdevsim/netdevsim10/2 -jp {
    "port": {
        "netdevsim/netdevsim10/2": {
            "type": "eth",
            "netdev": "eni10npf0sf44",
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": false,
            "splittable": false,
            "function": {
                "hw_addr": "00:00:00:00:00:00"
            }
        }
    }
}
---
 include/net/devlink.h        | 17 +++++++++++++++++
 include/uapi/linux/devlink.h |  7 +++++++
 net/core/devlink.c           | 37 ++++++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+)
Quoted 1 line

static int __devlink_port_phys_port_name_get(struct devlink_port

*devlink_port,

net/core/devlink.c static int

Quoted 3 lines
net/core/devlink.c +0 -0
 					     char *name, size_t len)
 {
@@ -7855,6 +7889,9 @@ static int

__devlink_port_phys_port_name_get(struct devlink_port *devlink_port,

Quoted 9 lines

net/core/devlink.c static int __devlink_port_phys_port_name_get(struct devlink_port *devlink_port, · line 7896

Quoted 9 lines
net/core/devlink.c +3 -0
 		n = snprintf(name, len, "pf%uvf%u",
 			     attrs->pci_vf.pf, attrs->pci_vf.vf);
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		n = snprintf(name, len, "pf%usf%u", attrs->pci_sf.pf, attrs-
 pci_sf.sf);
+		break;
 	}
 

This is where the naming scheme is done, like pcipf and pcivf port flavours.

Quoted 2 lines
Quoted 2 lines

if (n >= len)

#21 lore raw

Re: [PATCH net-next v2 1/8] devlink: Introduce PCI SF port flavour and port attribute

Quoted 28 lines · Parav Pandit
Quoted 15 lines

From: Jacob Keller <[email protected]>
Sent: Friday, September 18, 2020 12:00 AM

Quoted 8 lines · Parav Pandit

A PCI sub-function (SF) represents a portion of the device similar to PCI VF.

In an eswitch, PCI SF may have port which is normally represented using a representor netdevice.
To have better visibility of eswitch port, its association with SF, and its representor netdevice, introduce a PCI SF port flavour.

Quoted 9 lines
Quoted 6 lines

When devlink port flavour is PCI SF, fill up PCI SF attributes of the port.

Extend port name creation using PCI PF and SF number scheme on best effort basis, so that vendor drivers can skip defining their own scheme.

What does this mean? What's the scheme used?

Scheme used is equivalent as what is used for PCI VF ports. pfNvfM.
It is pfNsfM.
Below example shows the representor netdevice name as 'eni10npf0sf44' built by systemd/udev using phys_port_name.

Quoted 7 lines
Quoted 1 line

Do drivers still have the option to make their own scheme? If so, why?

Today we have two types of drivers (mlx5_core, netdevsim) which uses devlink core which creates the name.
Or other drivers (bnxt, nfp) which doesn't yet migrated to use devlink infra for PCI PF, VF ports.
Such drivers are phys_port_name and other ndos.
It is not the role of this patch to block those drivers, but any new implementation doesn't need to hand code switch_id and phys_port_name related ndos for SF. For example, bnxt_vf_rep_get_phys_port_name().

Ok, thanks for the explanation.

Quoted 8 lines
Quoted 3 lines

It's not obvious to me in this patch where the numbering scheme comes from. It looks like it's still up to the caller to set the numbers.

Naming scheme for PCI PF and PCI VF port flavours already exist. Scheme is equivalent for PCI SF flavour.

I thought example is good enough to show that, but I will update commit message to describe this scheme to make it clear. pfNsfM.

I think I just hadn't quite moved from "sf number" to "name of the netdevice" and was thinking of scheme for how the sf number is selected, which isn't really what the statement was about.

Quoted 55 lines
Quoted 42 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

An example view of a PCI SF port.

$ devlink port show netdevsim/netdevsim10/2
netdevsim/netdevsim10/2: type eth netdev eni10npf0sf44 flavour pcisf

controller 0 pfnum 0 sfnum 44 external false splittable false

Quoted 31 lines

function:

    hw_addr 00:00:00:00:00:00
devlink port show netdevsim/netdevsim10/2 -jp {
    "port": {
        "netdevsim/netdevsim10/2": {
            "type": "eth",
            "netdev": "eni10npf0sf44",
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": false,
            "splittable": false,
            "function": {
                "hw_addr": "00:00:00:00:00:00"
            }
        }
    }
}
---
 include/net/devlink.h        | 17 +++++++++++++++++
 include/uapi/linux/devlink.h |  7 +++++++
 net/core/devlink.c           | 37 ++++++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+)
Quoted 1 line

static int __devlink_port_phys_port_name_get(struct devlink_port

*devlink_port,

net/core/devlink.c static int

Quoted 3 lines
net/core/devlink.c +0 -0
 					     char *name, size_t len)
 {
@@ -7855,6 +7889,9 @@ static int

__devlink_port_phys_port_name_get(struct devlink_port *devlink_port,

Quoted 9 lines

net/core/devlink.c static int __devlink_port_phys_port_name_get(struct devlink_port *devlink_port, · line 7896

Quoted 9 lines
net/core/devlink.c +3 -0
 		n = snprintf(name, len, "pf%uvf%u",
 			     attrs->pci_vf.pf, attrs->pci_vf.vf);
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		n = snprintf(name, len, "pf%usf%u", attrs->pci_sf.pf, attrs-
 pci_sf.sf);
+		break;
 	}
 

This is where the naming scheme is done, like pcipf and pcivf port flavours.

Quoted 2 lines
Quoted 2 lines

if (n >= len)

#22 lore raw

PATCH v2 2/8 devlink: Support add and delete devlink port

In reply to Parav Pandit

Extended devlink interface for the user to add and delete port. Extend devlink to connect user requests to driver to add/delete such port in the device.

When driver routines are invoked, devlink instance lock is not held. This enables driver to perform several devlink objects registration, unregistration such as (port, health reporter, resource etc) by using exising devlink APIs.
This also helps to uniformly used the code for port registration during driver unload and during port deletion initiated by user.

Examples of add, show and delete commands:
Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device
$ devlink port show netdevsim/netdevsim10/0
netdevsim/netdevsim10/0: type eth netdev eni10np1 flavour physical port 1 splittable false
$ devlink port add netdevsim/netdevsim10 flavour pcipf pfnum 0
$ devlink port show netdevsim/netdevsim10/1
netdevsim/netdevsim10/1: type eth netdev eni10npf0 flavour pcipf controller 0 pfnum 0 external false splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port show netdevsim/netdevsim10/1 -jp
{
    "port": {
        "netdevsim/netdevsim10/1": {
            "type": "eth",
            "netdev": "eni10npf0",
            "flavour": "pcipf",
            "controller": 0,
            "pfnum": 0,
            "external": false,
            "splittable": false,
            "function": {
                "hw_addr": "00:00:00:00:00:00",
                "state": "inactive"
            }
        }
    }
}
$ devlink port del netdevsim/netdevsim10/1
---
 include/net/devlink.h | 38 ++++++++++++++++++++++++
 net/core/devlink.c    | 67 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 105 insertions(+)
include/net/devlink.h +38 -0
index 1edb558125b0..ebab2c0360d0 100644
@@ -142,6 +142,17 @@ struct devlink_port {
 	struct mutex reporters_lock; /* Protects reporter_list */
 };
 
+struct devlink_port_new_attrs {
+	enum devlink_port_flavour flavour;
+	unsigned int port_index;
+	u32 controller;
+	u32 sfnum;
+	u16 pfnum;
+	u8 port_index_valid:1,
+	   controller_valid:1,
+	   sfnum_valid:1;
+};
+
 struct devlink_sb_pool_info {
 	enum devlink_sb_pool_type pool_type;
 	u32 size;
@@ -1189,6 +1200,33 @@ struct devlink_ops {
 	int (*port_function_hw_addr_set)(struct devlink *devlink, struct devlink_port *port,
 					 const u8 *hw_addr, int hw_addr_len,
 					 struct netlink_ext_ack *extack);
+	/**
+	 * @port_new: Port add function.
+	 *
+	 * Should be used by device driver to let caller add new port of a specified flavour
+	 * with optional attributes.
+	 * Driver should return -EOPNOTSUPP if it doesn't support port addition of a specified
+	 * flavour or specified attributes. Driver should set extack error message in case of fail
+	 * to add the port.
+	 * devlink core does not hold a devlink instance lock when this callback is invoked.
+	 * Driver must ensures synchronization when adding or deleting a port. Driver must
+	 * register a port with devlink core.
+	 */
+	int (*port_new)(struct devlink *devlink, const struct devlink_port_new_attrs *attrs,
+			struct netlink_ext_ack *extack);
+	/**
+	 * @port_del: Port delete function.
+	 *
+	 * Should be used by device driver to let caller delete port which was previously created
+	 * using port_new() callback.
+	 * Driver should return -EOPNOTSUPP if it doesn't support port deletion.
+	 * Driver should set extack error message in case of fail to delete the port.
+	 * devlink core does not hold a devlink instance lock when this callback is invoked.
+	 * Driver must ensures synchronization when adding or deleting a port. Driver must
+	 * register a port with devlink core.
+	 */
+	int (*port_del)(struct devlink *devlink, unsigned int port_index,
+			struct netlink_ext_ack *extack);
 };
 
 static inline void *devlink_priv(struct devlink *devlink)
net/core/devlink.c +67 -0

85 lines not on this page · raw message

Signature · 1 line
2.26.2

A message that is not in the archive (sent off-list or not yet received): [email protected]

#24 lore raw

RE: [PATCH net-next v2 2/8] devlink: Support add and delete devlink port

Quoted 17 lines

From: Jacob Keller <[email protected]>
Sent: Friday, September 18, 2020 12:13 AM

Quoted 11 lines · Parav Pandit

Extended devlink interface for the user to add and delete port. Extend devlink to connect user requests to driver to add/delete such port in the device.

When driver routines are invoked, devlink instance lock is not held. This enables driver to perform several devlink objects registration, unregistration such as (port, health reporter, resource etc) by using exising devlink APIs.
This also helps to uniformly used the code for port registration during driver unload and during port deletion initiated by user.

Quoted 3 lines

Ok. Seems like a good goal to be able to share code uniformly between driver load and new port creation.

Yes.

Quoted 10 lines

net/core/devlink.c static int devlink_nl_cmd_port_unsplit_doit(struct sk_buff *skb,

Quoted 9 lines
net/core/devlink.c +9 -0
+static int devlink_nl_cmd_port_new_doit(struct sk_buff *skb, struct
+genl_info *info) {
+	struct netlink_ext_ack *extack = info->extack;
+	struct devlink_port_new_attrs new_attrs = {};
+	struct devlink *devlink = info->user_ptr[0];
+
+	if (!info->attrs[DEVLINK_ATTR_PORT_FLAVOUR] ||
+	    !info->attrs[DEVLINK_ATTR_PORT_PCI_PF_NUMBER]) {
+		NL_SET_ERR_MSG_MOD(extack, "Port flavour or PCI PF are not

specified");

Quoted 11 lines

Review

Quoted 7 lines
net/core/devlink.c +3 -0
+		return -EINVAL;
+	}
+	new_attrs.flavour = nla_get_u16(info-

attrs[DEVLINK_ATTR_PORT_FLAVOUR]);

(quoted diff) +3 -0
+	new_attrs.pfnum =
+nla_get_u16(info->attrs[DEVLINK_ATTR_PORT_PCI_PF_NUMBER]);
+

Presuming that the device supports it, this could be used to allow creating other types of ports bsides subfunctions?

This series is creating PCI PF and subfunction ports.
Jiri's RFC [1] explained a possibility for VF representors to follow the similar scheme if device supports it.

I am not sure creating other port flavours are useful enough such as CPU, PHYSICAL etc. I do not have enough knowledge about use case for creating CPU ports, if at all it exists. Usually physical ports are linked to a card hardware on how many physical ports present on circuit. So I find it odd if a device support physical port creation, but again its my limited view at the moment.

Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +2 -0
+	if (info->attrs[DEVLINK_ATTR_PORT_INDEX]) {
+		new_attrs.port_index = nla_get_u32(info-

attrs[DEVLINK_ATTR_PORT_INDEX]);

(quoted diff) +2 -0
+		new_attrs.port_index_valid = true;
+	}

So if the userspace doesn't provide a port index, drivers are responsible for choosing one? Same for the other attributes I suppose?

Yes.

Quoted 18 lines
Quoted 18 lines
net/core/devlink.c +3 -0
+	if (info->attrs[DEVLINK_ATTR_PORT_CONTROLLER_NUMBER]) {
+		new_attrs.controller =
+			nla_get_u16(info-

attrs[DEVLINK_ATTR_PORT_CONTROLLER_NUMBER]);

net/core/devlink.c +4 -0
+		new_attrs.controller_valid = true;
+	}
+	if (info->attrs[DEVLINK_ATTR_PORT_PCI_SF_NUMBER]) {
+		new_attrs.sfnum = nla_get_u32(info-

attrs[DEVLINK_ATTR_PORT_PCI_SF_NUMBER]);

net/core/devlink.c +8 -0
+		new_attrs.sfnum_valid = true;
+	}
+
+	if (!devlink->ops->port_new)
+		return -EOPNOTSUPP;
+
+	return devlink->ops->port_new(devlink, &new_attrs, extack); }
+
#25 lore raw

Re: [PATCH net-next v2 2/8] devlink: Support add and delete devlink port

Quoted 43 lines · Parav Pandit
Quoted 17 lines

From: Jacob Keller <[email protected]>
Sent: Friday, September 18, 2020 12:13 AM

Quoted 11 lines · Parav Pandit

Extended devlink interface for the user to add and delete port. Extend devlink to connect user requests to driver to add/delete such port in the device.

When driver routines are invoked, devlink instance lock is not held. This enables driver to perform several devlink objects registration, unregistration such as (port, health reporter, resource etc) by using exising devlink APIs.
This also helps to uniformly used the code for port registration during driver unload and during port deletion initiated by user.

Quoted 3 lines

Ok. Seems like a good goal to be able to share code uniformly between driver load and new port creation.

Yes.

Quoted 10 lines

net/core/devlink.c static int devlink_nl_cmd_port_unsplit_doit(struct sk_buff *skb,

Quoted 9 lines
net/core/devlink.c +9 -0
+static int devlink_nl_cmd_port_new_doit(struct sk_buff *skb, struct
+genl_info *info) {
+	struct netlink_ext_ack *extack = info->extack;
+	struct devlink_port_new_attrs new_attrs = {};
+	struct devlink *devlink = info->user_ptr[0];
+
+	if (!info->attrs[DEVLINK_ATTR_PORT_FLAVOUR] ||
+	    !info->attrs[DEVLINK_ATTR_PORT_PCI_PF_NUMBER]) {
+		NL_SET_ERR_MSG_MOD(extack, "Port flavour or PCI PF are not

specified");

Quoted 11 lines

Review

Quoted 7 lines
net/core/devlink.c +3 -0
+		return -EINVAL;
+	}
+	new_attrs.flavour = nla_get_u16(info-

attrs[DEVLINK_ATTR_PORT_FLAVOUR]);

(quoted diff) +3 -0
+	new_attrs.pfnum =
+nla_get_u16(info->attrs[DEVLINK_ATTR_PORT_PCI_PF_NUMBER]);
+

Presuming that the device supports it, this could be used to allow creating other types of ports bsides subfunctions?

Quoted 3 lines

This series is creating PCI PF and subfunction ports.
Jiri's RFC [1] explained a possibility for VF representors to follow the similar scheme if device supports it.

Right, VFs was the most obvious point. The ability to create VFs without needing to destroy all VFs and re-create them seems quite useful.

Quoted 5 lines

I am not sure creating other port flavours are useful enough such as CPU, PHYSICAL etc. I do not have enough knowledge about use case for creating CPU ports, if at all it exists. Usually physical ports are linked to a card hardware on how many physical ports present on circuit. So I find it odd if a device support physical port creation, but again its my limited view at the moment.

Yea, I agree here too. I find that somewhat odd, but I suppose for everything but PHYSICAL types it's not impossible.

#26 lore raw

RE: [PATCH net-next v2 2/8] devlink: Support add and delete devlink port

Quoted 50 lines

From: Jacob Keller <[email protected]>
Sent: Saturday, September 19, 2020 4:37 AM

Quoted 43 lines · Parav Pandit
Quoted 17 lines

From: Jacob Keller <[email protected]>
Sent: Friday, September 18, 2020 12:13 AM

Quoted 11 lines · Parav Pandit

Extended devlink interface for the user to add and delete port. Extend devlink to connect user requests to driver to add/delete such port in the device.

When driver routines are invoked, devlink instance lock is not held. This enables driver to perform several devlink objects registration, unregistration such as (port, health reporter, resource etc) by using exising devlink APIs.
This also helps to uniformly used the code for port registration during driver unload and during port deletion initiated by user.

Quoted 3 lines

Ok. Seems like a good goal to be able to share code uniformly between driver load and new port creation.

Yes.

Quoted 21 lines

net/core/devlink.c static int devlink_nl_cmd_port_unsplit_doit(struct sk_buff *skb,

Quoted 9 lines
net/core/devlink.c +9 -0
+static int devlink_nl_cmd_port_new_doit(struct sk_buff *skb, struct
+genl_info *info) {
+	struct netlink_ext_ack *extack = info->extack;
+	struct devlink_port_new_attrs new_attrs = {};
+	struct devlink *devlink = info->user_ptr[0];
+
+	if (!info->attrs[DEVLINK_ATTR_PORT_FLAVOUR] ||
+	    !info->attrs[DEVLINK_ATTR_PORT_PCI_PF_NUMBER]) {
+		NL_SET_ERR_MSG_MOD(extack, "Port flavour or PCI PF are not

specified");

Review

Quoted 7 lines
net/core/devlink.c +3 -0
+		return -EINVAL;
+	}
+	new_attrs.flavour = nla_get_u16(info-

attrs[DEVLINK_ATTR_PORT_FLAVOUR]);

(quoted diff) +3 -0
+	new_attrs.pfnum =
+nla_get_u16(info->attrs[DEVLINK_ATTR_PORT_PCI_PF_NUMBER]);
+

Presuming that the device supports it, this could be used to allow creating other types of ports bsides subfunctions?

Quoted 8 lines
Quoted 2 lines

This series is creating PCI PF and subfunction ports.
Jiri's RFC [1] explained a possibility for VF representors to follow the similar

scheme if device supports it.

Quoted 1 line

Right, VFs was the most obvious point. The ability to create VFs without needing to destroy all VFs and re-create them seems quite useful.

Yes.

Quoted 11 lines
Quoted 1 line

I am not sure creating other port flavours are useful enough such as CPU,

PHYSICAL etc.

Quoted 1 line

I do not have enough knowledge about use case for creating CPU ports, if at

all it exists.

Quoted 1 line

Usually physical ports are linked to a card hardware on how many physical

ports present on circuit.

Quoted 1 line

So I find it odd if a device support physical port creation, but again its my

limited view at the moment.

Quoted 1 line

Yea, I agree here too. I find that somewhat odd, but I suppose for everything but PHYSICAL types it's not impossible.

Ok.

#27 lore raw

PATCH v2 3/8 devlink: Prepare code to fill multiple port function attributes

In reply to Parav Pandit

Prepare code to fill zero or more port function optional attributes. Subsequent patch makes use of this to fill more port function attributes.

---
 net/core/devlink.c | 53 +++++++++++++++++++++++++---------------------
 1 file changed, 29 insertions(+), 24 deletions(-)
net/core/devlink.c +29 -24
index e93730065c57..d152489e48da 100644
@@ -570,6 +570,31 @@ static int devlink_nl_port_attrs_put(struct sk_buff *msg,
 	return 0;
 }
 
+static int
+devlink_port_function_hw_addr_fill(struct devlink *devlink, const struct devlink_ops *ops,
+				   struct devlink_port *port, struct sk_buff *msg,
+				   struct netlink_ext_ack *extack, bool *msg_updated)
+{
+	u8 hw_addr[MAX_ADDR_LEN];
+	int hw_addr_len;
+	int err;
+
+	if (!ops->port_function_hw_addr_get)
+		return 0;
+
+	err = ops->port_function_hw_addr_get(devlink, port, hw_addr, &hw_addr_len, extack);
+	if (err) {
+		if (err == -EOPNOTSUPP)
+			return 0;
+		return err;
+	}
+	err = nla_put(msg, DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR, hw_addr_len, hw_addr);
+	if (err)
+		return err;
+	*msg_updated = true;
+	return 0;
+}
+
 static int
 devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *port,
 				   struct netlink_ext_ack *extack)
@@ -577,36 +602,16 @@ devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *por
 	struct devlink *devlink = port->devlink;
 	const struct devlink_ops *ops;
 	struct nlattr *function_attr;
-	bool empty_nest = true;
-	int err = 0;
+	bool msg_updated = false;
+	int err;
 
 	function_attr = nla_nest_start_noflag(msg, DEVLINK_ATTR_PORT_FUNCTION);
 	if (!function_attr)
 		return -EMSGSIZE;
 
 	ops = devlink->ops;
-	if (ops->port_function_hw_addr_get) {
-		int hw_addr_len;
-		u8 hw_addr[MAX_ADDR_LEN];
-
-		err = ops->port_function_hw_addr_get(devlink, port, hw_addr, &hw_addr_len, extack);
-		if (err == -EOPNOTSUPP) {
-			/* Port function attributes are optional for a port. If port doesn't
-			 * support function attribute, returning -EOPNOTSUPP is not an error.
-			 */
-			err = 0;
-			goto out;
-		} else if (err) {
-			goto out;
-		}
-		err = nla_put(msg, DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR, hw_addr_len, hw_addr);
-		if (err)
-			goto out;
-		empty_nest = false;
-	}
-
-out:
-	if (err || empty_nest)
+	err = devlink_port_function_hw_addr_fill(devlink, ops, port, msg, extack, &msg_updated);
+	if (err || !msg_updated)
 		nla_nest_cancel(msg, function_attr);
 	else
 		nla_nest_end(msg, function_attr);
Signature · 1 line
2.26.2

A message that is not in the archive (sent off-list or not yet received): [email protected]

#29 lore raw

RE: [PATCH net-next v2 3/8] devlink: Prepare code to fill multiple port function attributes

Hi Jacob,

Quoted 84 lines

From: Jacob Keller <[email protected]>
Sent: Friday, September 18, 2020 12:29 AM

net/core/devlink.c static int devlink_nl_port_attrs_put(struct sk_buff

Quoted 16 lines · Parav Pandit

Prepare code to fill zero or more port function optional attributes. Subsequent patch makes use of this to fill more port function attributes.

---
 net/core/devlink.c | 53
+++++++++++++++++++++++++---------------------
 1 file changed, 29 insertions(+), 24 deletions(-)
net/core/devlink.c index renamed from net/core/devlink.c +0 -0

e93730065c57..d152489e48da 100644

net/core/devlink.c +0 -0
@@ -570,6 +570,31 @@ static int devlink_nl_port_attrs_put(struct sk_buff

*msg,

net/core/devlink.c static int devlink_nl_port_attrs_put(struct sk_buff *msg,

Quoted 5 lines
net/core/devlink.c +2 -0
 	return 0;
 }
 
+static int
+devlink_port_function_hw_addr_fill(struct devlink *devlink, const struct

devlink_ops *ops,

net/core/devlink.c static int devlink_nl_port_attrs_put(struct sk_buff · line 575

Quoted 1 line
net/core/devlink.c +1 -0
+				   struct devlink_port *port, struct sk_buff

*msg,

net/core/devlink.c static int devlink_nl_port_attrs_put(struct sk_buff · line 576

Quoted 1 line
net/core/devlink.c +1 -0
+				   struct netlink_ext_ack *extack, bool

*msg_updated) {

net/core/devlink.c static int devlink_nl_port_attrs_put(struct sk_buff *msg,

Quoted 8 lines
net/core/devlink.c +8 -0
+	u8 hw_addr[MAX_ADDR_LEN];
+	int hw_addr_len;
+	int err;
+
+	if (!ops->port_function_hw_addr_get)
+		return 0;
+
+	err = ops->port_function_hw_addr_get(devlink, port, hw_addr,

&hw_addr_len, extack);

net/core/devlink.c static int devlink_nl_port_attrs_put(struct sk_buff *msg,

Quoted 6 lines
net/core/devlink.c +6 -0
+	if (err) {
+		if (err == -EOPNOTSUPP)
+			return 0;
+		return err;
+	}
+	err = nla_put(msg, DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR,

hw_addr_len, hw_addr);

net/core/devlink.c static int devlink_nl_port_attrs_put(struct sk_buff *msg,

Quoted 8 lines
net/core/devlink.c +6 -0
+	if (err)
+		return err;
+	*msg_updated = true;
+	return 0;
+}
+
 static int
 devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port

*port,

net/core/devlink.c static int devlink_nl_port_attrs_put(struct sk_buff · line 599

Quoted 1 line
net/core/devlink.c +0 -0
 				   struct netlink_ext_ack *extack) @@ -577,36

+602,16 @@

Quoted 1 line

devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port

*por

net/core/devlink.c devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *por

Quoted 9 lines
net/core/devlink.c +2 -2
 	struct devlink *devlink = port->devlink;
 	const struct devlink_ops *ops;
 	struct nlattr *function_attr;
-	bool empty_nest = true;
-	int err = 0;
+	bool msg_updated = false;
+	int err;
 
 	function_attr = nla_nest_start_noflag(msg,

DEVLINK_ATTR_PORT_FUNCTION);

net/core/devlink.c devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *por

Quoted 9 lines
net/core/devlink.c +0 -5
 	if (!function_attr)
 		return -EMSGSIZE;
 
 	ops = devlink->ops;
-	if (ops->port_function_hw_addr_get) {
-		int hw_addr_len;
-		u8 hw_addr[MAX_ADDR_LEN];
-
-		err = ops->port_function_hw_addr_get(devlink, port, hw_addr,

&hw_addr_len, extack);

net/core/devlink.c

Quoted 2 lines
net/core/devlink.c +0 -2
-		if (err == -EOPNOTSUPP) {
-			/* Port function attributes are optional for a port. If

port doesn't

Quoted 6 lines

net/core/devlink.c

Quoted 1 line
net/core/devlink.c +0 -1
-			 * support function attribute, returning -EOPNOTSUPP is

not an error.

net/core/devlink.c devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *por · line 596

Quoted 1 line
net/core/devlink.c +0 -1
-			 */

We lost this comment in the move it looks like. I think it's still useful to keep for clarity of why we're converting -EOPNOTSUPP in the return.

You are right. It is a useful comment.
However, it is already covered in include/net/devlink.h in the standard comment of the callback function. For new driver implementation, looking there will be more useful.
So I guess its ok to drop from here.

Quoted 22 lines

net/core/devlink.c devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *por

Quoted 7 lines
net/core/devlink.c +0 -6
-			err = 0;
-			goto out;
-		} else if (err) {
-			goto out;
-		}
-		err = nla_put(msg,

DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR, hw_addr_len, hw_addr);

net/core/devlink.c devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *por

Quoted 8 lines
net/core/devlink.c +1 -7
-		if (err)
-			goto out;
-		empty_nest = false;
-	}
-
-out:
-	if (err || empty_nest)
+	err = devlink_port_function_hw_addr_fill(devlink, ops, port, msg,

extack, &msg_updated);

Quoted 5 lines
net/core/devlink.c +1 -0
+	if (err || !msg_updated)
 		nla_nest_cancel(msg, function_attr);
 	else
 		nla_nest_end(msg, function_attr);
#30 lore raw

Re: [PATCH net-next v2 3/8] devlink: Prepare code to fill multiple port function attributes

Quoted 6 lines · Parav Pandit

Hi Jacob,

Quoted 4 lines

From: Jacob Keller <[email protected]>
Sent: Friday, September 18, 2020 12:29 AM

Quoted 7 lines
Quoted 2 lines

We lost this comment in the move it looks like. I think it's still useful to keep for clarity of why we're converting -EOPNOTSUPP in the return.

You are right. It is a useful comment.
However, it is already covered in include/net/devlink.h in the standard comment of the callback function. For new driver implementation, looking there will be more useful.
So I guess its ok to drop from here.

Yea if it's still in the header I don't think it's too important to keep here.

Thanks!
-Jake

#31 lore raw

RE: [PATCH net-next v2 3/8] devlink: Prepare code to fill multiple port function attributes

Quoted 16 lines

From: Jacob Keller <[email protected]>
Sent: Saturday, September 19, 2020 4:24 AM

Quoted 8 lines · Parav Pandit

Hi Jacob,

Quoted 6 lines

From: Jacob Keller <[email protected]>
Sent: Friday, September 18, 2020 12:29 AM

We lost this comment in the move it looks like. I think it's still useful to keep for clarity of why we're converting -EOPNOTSUPP in the

return.

Quoted 9 lines
Quoted 2 lines

You are right. It is a useful comment.
However, it is already covered in include/net/devlink.h in the standard

comment of the callback function.

Quoted 3 lines

For new driver implementation, looking there will be more useful. So I guess its ok to drop from here.

Yea if it's still in the header I don't think it's too important to keep here.

Alright. Will keep in the header.
Thanks.

Quoted 2 lines

Thanks!
-Jake

#32 lore raw

PATCH v2 4/8 devlink: Support get and set state of port function

In reply to Parav Pandit

devlink port function can be in active or inactive state. Allow users to get and set port function's state.

Example of a PCI SF port which supports a port function:
Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device
$ devlink port show
netdevsim/netdevsim10/0: type eth netdev eth0 flavour physical port 1 splittable false
$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0 sfnum 44
$ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour pcisf controller 0 pfnum 0 sfnum 44 external false splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr 00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/11 -jp
{
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0sf44",
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": false,
            "splittable": false,
            "function": {
                "hw_addr": "00:11:22:33:44:55",
                "state": "active"
            }
        }
    }
}
---
 include/net/devlink.h        | 20 ++++++++++
 include/uapi/linux/devlink.h |  6 +++
 net/core/devlink.c           | 77 +++++++++++++++++++++++++++++++++++-
 3 files changed, 101 insertions(+), 2 deletions(-)
include/net/devlink.h +20 -0
index ebab2c0360d0..500c22835686 100644
@@ -1200,6 +1200,26 @@ struct devlink_ops {
 	int (*port_function_hw_addr_set)(struct devlink *devlink, struct devlink_port *port,
 					 const u8 *hw_addr, int hw_addr_len,
 					 struct netlink_ext_ack *extack);
+	/**
+	 * @port_function_state_get: Port function's state get function.
+	 *
+	 * Should be used by device drivers to report the state of a function managed
+	 * by the devlink port. Driver should return -EOPNOTSUPP if it doesn't support port
+	 * function handling for a particular port.
+	 */
+	int (*port_function_state_get)(struct devlink *devlink, struct devlink_port *port,
+				       enum devlink_port_function_state *state,
+				       struct netlink_ext_ack *extack);
+	/**
+	 * @port_function_state_set: Port function's state set function.
+	 *
+	 * Should be used by device drivers to set the state of a function managed
+	 * by the devlink port. Driver should return -EOPNOTSUPP if it doesn't support port
+	 * function handling for a particular port.
+	 */
+	int (*port_function_state_set)(struct devlink *devlink, struct devlink_port *port,
+				       enum devlink_port_function_state state,
+				       struct netlink_ext_ack *extack);
 	/**
 	 * @port_new: Port add function.
 	 *
include/uapi/linux/devlink.h +6 -0
index 09c41b9ce407..8e513f1cd638 100644
@@ -518,9 +518,15 @@ enum devlink_resource_unit {
 enum devlink_port_function_attr {
 	DEVLINK_PORT_FUNCTION_ATTR_UNSPEC,
 	DEVLINK_PORT_FUNCTION_ATTR_HW_ADDR,	/* binary */
+	DEVLINK_PORT_FUNCTION_ATTR_STATE,	/* u8 */
 
 	__DEVLINK_PORT_FUNCTION_ATTR_MAX,
 	DEVLINK_PORT_FUNCTION_ATTR_MAX = __DEVLINK_PORT_FUNCTION_ATTR_MAX - 1
 };
 
+enum devlink_port_function_state {
+	DEVLINK_PORT_FUNCTION_STATE_INACTIVE,
+	DEVLINK_PORT_FUNCTION_STATE_ACTIVE,
+};
+
 #endif /* _UAPI_LINUX_DEVLINK_H_ */
net/core/devlink.c +75 -2

108 lines not on this page · raw message

Signature · 1 line
2.26.2
#33 lore raw

Re: [PATCH net-next v2 4/8] devlink: Support get and set state of port function

net/core/devlink.c devlink_port_function_hw_addr_fill(struct devlink *devlink, const struct devlink · line 601

Quoted 9 lines · Parav Pandit
net/core/devlink.c +1 -0
index d152489e48da..c82098cb75da 100644
@@ -595,6 +598,40 @@ devlink_port_function_hw_addr_fill(struct devlink *devlink, const struct devlink
 	return 0;
 }
 
+static bool devlink_port_function_state_valid(u8 state)

you have a named enum so why not 'enum devlink_port_function_state state'?

Quoted 13 lines
net/core/devlink.c +13 -0
+{
+	return state == DEVLINK_PORT_FUNCTION_STATE_INACTIVE ||
+	       state == DEVLINK_PORT_FUNCTION_STATE_ACTIVE;
+}
+
+static int devlink_port_function_state_fill(struct devlink *devlink, const struct devlink_ops *ops,
+					    struct devlink_port *port, struct sk_buff *msg,
+					    struct netlink_ext_ack *extack, bool *msg_updated)
+{
+	enum devlink_port_function_state state;
+	int err;
+
+	if (!ops->port_function_state_get)

net/core/devlink.c devlink_port_function_hw_addr_fill(struct devlink *devlink, const struct devlink · line 624

Quoted 10 lines
net/core/devlink.c +10 -0
+		return 0;
+
+	err = ops->port_function_state_get(devlink, port, &state, extack);
+	if (err) {
+		if (err == -EOPNOTSUPP)
+			return 0;
+		return err;
+	}
+	if (!devlink_port_function_state_valid(state)) {
+		WARN_ON(1);

WARN_ON_ONCE at most.

Quoted 13 lines
net/core/devlink.c +10 -0
+		NL_SET_ERR_MSG_MOD(extack, "Invalid state value read from driver");
+		return -EINVAL;
+	}
+	err = nla_put_u8(msg, DEVLINK_PORT_FUNCTION_ATTR_STATE, state);
+	if (err)
+		return err;
+	*msg_updated = true;
+	return 0;
+}
+
 static int
 devlink_nl_port_function_attrs_put(struct sk_buff *msg, struct devlink_port *port,
 				   struct netlink_ext_ack *extack)
#34 lore raw

RE: [PATCH net-next v2 4/8] devlink: Support get and set state of port function

Quoted 12 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 1:54 AM

net/core/devlink.c devlink_port_function_hw_addr_fill(struct devlink

Quoted 5 lines · Parav Pandit
net/core/devlink.c index renamed from net/core/devlink.c +0 -0

d152489e48da..c82098cb75da 100644

net/core/devlink.c +0 -0
@@ -595,6 +598,40 @@ devlink_port_function_hw_addr_fill(struct devlink

*devlink, const struct devlink

Quoted 7 lines

net/core/devlink.c devlink_port_function_hw_addr_fill(struct devlink *devlink, const struct devlink · line 601

Quoted 4 lines
net/core/devlink.c +1 -0
 	return 0;
 }
 
+static bool devlink_port_function_state_valid(u8 state)

you have a named enum so why not 'enum devlink_port_function_state state'?

Right. I should. I missed it.
Will do.

Quoted 12 lines

net/core/devlink.c devlink_port_function_hw_addr_fill(struct devlink *devlink, const struct devlink

Quoted 7 lines
net/core/devlink.c +6 -0
+{
+	return state == DEVLINK_PORT_FUNCTION_STATE_INACTIVE ||
+	       state == DEVLINK_PORT_FUNCTION_STATE_ACTIVE;
+}
+
+static int devlink_port_function_state_fill(struct devlink *devlink, const struct

devlink_ops *ops,

net/core/devlink.c devlink_port_function_hw_addr_fill(struct devlink · line 608

Quoted 1 line
net/core/devlink.c +1 -0
+					    struct devlink_port *port, struct

sk_buff *msg,

net/core/devlink.c devlink_port_function_hw_addr_fill(struct devlink · line 609

Quoted 1 line
net/core/devlink.c +1 -0
+					    struct netlink_ext_ack *extack, bool

*msg_updated) {

Quoted 17 lines
Quoted 4 lines
net/core/devlink.c +4 -0
+	enum devlink_port_function_state state;
+	int err;
+
+	if (!ops->port_function_state_get)

net/core/devlink.c devlink_port_function_hw_addr_fill(struct devlink *devlink, const struct devlink · line 624

Quoted 10 lines
net/core/devlink.c +10 -0
+		return 0;
+
+	err = ops->port_function_state_get(devlink, port, &state, extack);
+	if (err) {
+		if (err == -EOPNOTSUPP)
+			return 0;
+		return err;
+	}
+	if (!devlink_port_function_state_valid(state)) {
+		WARN_ON(1);

WARN_ON_ONCE at most.

Yep. I will change to WARN_ON_ONCE.

#35 lore raw

PATCH v2 5/8 netdevsim: Add support for add and delete of a PCI PF port

In reply to Parav Pandit

Simulate PCI PF ports. Allow user to create one or more PCI PF ports.

Examples:

Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device

Add and show devlink port of flavour 'pcipf' for PF number 0.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port show netdevsim/netdevsim10/10
netdevsim/netdevsim10/10: type eth netdev eni10npf0 flavour pcipf controller 0 pfnum 0 external false splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive

Delete newly added devlink port

$ devlink port add netdevsim/netdevsim10/10

---
Changelog:
v1->v2:
- Fixed extra semicolon at end of switch case reportec by coccinelle

---
 drivers/net/netdevsim/Makefile        |   3 +-
 drivers/net/netdevsim/dev.c           |  10 +
 drivers/net/netdevsim/netdevsim.h     |  19 ++
 drivers/net/netdevsim/port_function.c | 337 ++++++++++++++++++++++++++
 4 files changed, 368 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/netdevsim/port_function.c
drivers/net/netdevsim/Makefile +2 -1
index ade086eed955..e69e895af62c 100644
@@ -3,7 +3,8 @@
 obj-$(CONFIG_NETDEVSIM) += netdevsim.o
 
 netdevsim-objs := \
-	netdev.o dev.o ethtool.o fib.o bus.o health.o udp_tunnels.o
+	netdev.o dev.o ethtool.o fib.o bus.o health.o udp_tunnels.o \
+	port_function.o
 
 ifeq ($(CONFIG_BPF_SYSCALL),y)
 netdevsim-objs += \
drivers/net/netdevsim/dev.c +10 -0
index 32f339fedb21..e3b81c8b5125 100644
@@ -884,6 +884,8 @@ static const struct devlink_ops nsim_dev_devlink_ops = {
 	.trap_group_set = nsim_dev_devlink_trap_group_set,
 	.trap_policer_set = nsim_dev_devlink_trap_policer_set,
 	.trap_policer_counter_get = nsim_dev_devlink_trap_policer_counter_get,
+	.port_new = nsim_dev_devlink_port_new,
+	.port_del = nsim_dev_devlink_port_del,
 };
 
 #define NSIM_DEV_MAX_MACS_DEFAULT 32
@@ -1017,6 +1019,8 @@ static int nsim_dev_reload_create(struct nsim_dev *nsim_dev,
 						      nsim_dev->ddir,
 						      nsim_dev,
 						&nsim_dev_take_snapshot_fops);
+
+	nsim_dev_port_function_enable(nsim_dev);
 	return 0;
 
 err_health_exit:
@@ -1050,6 +1054,7 @@ int nsim_dev_probe(struct nsim_bus_dev *nsim_bus_dev)
 	nsim_dev->max_macs = NSIM_DEV_MAX_MACS_DEFAULT;
 	nsim_dev->test1 = NSIM_DEV_TEST1_DEFAULT;
 	spin_lock_init(&nsim_dev->fa_cookie_lock);
+	nsim_dev_port_function_init(nsim_dev);
 
 	dev_set_drvdata(&nsim_bus_dev->dev, nsim_dev);
 
@@ -1097,6 +1102,7 @@ int nsim_dev_probe(struct nsim_bus_dev *nsim_bus_dev)
 	if (err)
 		goto err_bpf_dev_exit;
 
+	nsim_dev_port_function_enable(nsim_dev);
 	devlink_params_publish(devlink);
 	devlink_reload_enable(devlink);
 	return 0;
@@ -1131,6 +1137,9 @@ static void nsim_dev_reload_destroy(struct nsim_dev *nsim_dev)
 
 	if (devlink_is_reload_failed(devlink))
 		return;
+
+	/* Disable and destroy any user created devlink ports */
+	nsim_dev_port_function_disable(nsim_dev);
 	debugfs_remove(nsim_dev->take_snapshot);
 	nsim_dev_port_del_all(nsim_dev);
 	nsim_dev_health_exit(nsim_dev);
@@ -1155,6 +1164,7 @@ void nsim_dev_remove(struct nsim_bus_dev *nsim_bus_dev)
 				  ARRAY_SIZE(nsim_devlink_params));
 	devlink_unregister(devlink);
 	devlink_resources_unregister(devlink, NULL);
+	nsim_dev_port_function_exit(nsim_dev);
 	devlink_free(devlink);
 }
 
drivers/net/netdevsim/netdevsim.h +19 -0
index 0c86561e6d8d..aec3c4d5fda7 100644
@@ -213,6 +213,16 @@ struct nsim_dev {
 		bool ipv4_only;
 		u32 sleep;
 	} udp_ports;
+	struct {
+		refcount_t refcount; /* refcount along with disable_complete serializes
+				      * port operations with port function disablement
+				      * during driver unload.
+				      */
+		struct completion disable_complete;
+		struct list_head head;
+		struct ida ida;
+		struct ida pfnum_ida;
+	} port_functions;
 };
 
 static inline struct net *nsim_dev_net(struct nsim_dev *nsim_dev)
@@ -283,3 +293,12 @@ struct nsim_bus_dev {
 
 int nsim_bus_init(void);
 void nsim_bus_exit(void);
+
+void nsim_dev_port_function_init(struct nsim_dev *nsim_dev);
+void nsim_dev_port_function_exit(struct nsim_dev *nsim_dev);
+void nsim_dev_port_function_enable(struct nsim_dev *nsim_dev);
+void nsim_dev_port_function_disable(struct nsim_dev *nsim_dev);
+int nsim_dev_devlink_port_new(struct devlink *devlink, const struct devlink_port_new_attrs *attrs,
+			      struct netlink_ext_ack *extack);
+int nsim_dev_devlink_port_del(struct devlink *devlink, unsigned int port_index,
+			      struct netlink_ext_ack *extack);
drivers/net/netdevsim/port_function.c new file +337 -0

337 lines not on this page · raw message

Signature · 1 line
2.26.2
#36 lore raw

PATCH v2 6/8 netdevsim: Simulate get/set hardware address of a PCI port

In reply to Parav Pandit

Allow users to get/set hardware address for the PCI port.

Below example creates one devlink port, queries a port, sets a hardware address.

Example of a PCI SF port which supports a port function hw_addr set:
Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device
$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port show netdevsim/netdevsim10/10
netdevsim/netdevsim10/10: type eth netdev eni10npf0 flavour pcipf controller 0 pfnum 0 external false splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/10 hw_addr 00:11:22:33:44:55
$ devlink port show netdevsim/netdevsim10/10 -jp
{
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0",
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": true,
            "splittable": false,
            "function": {
                "hw_addr": "00:11:22:33:44:55"
            }
        }
    }
}
---
 drivers/net/netdevsim/dev.c           |  2 ++
 drivers/net/netdevsim/netdevsim.h     |  6 ++++
 drivers/net/netdevsim/port_function.c | 44 +++++++++++++++++++++++++++
 3 files changed, 52 insertions(+)
drivers/net/netdevsim/dev.c +2 -0
index e3b81c8b5125..ef2e293f358b 100644
@@ -886,6 +886,8 @@ static const struct devlink_ops nsim_dev_devlink_ops = {
 	.trap_policer_counter_get = nsim_dev_devlink_trap_policer_counter_get,
 	.port_new = nsim_dev_devlink_port_new,
 	.port_del = nsim_dev_devlink_port_del,
+	.port_function_hw_addr_get = nsim_dev_port_function_hw_addr_get,
+	.port_function_hw_addr_set = nsim_dev_port_function_hw_addr_set,
 };
 
 #define NSIM_DEV_MAX_MACS_DEFAULT 32
drivers/net/netdevsim/netdevsim.h +6 -0
index aec3c4d5fda7..8dc8f4e5dcd8 100644
@@ -302,3 +302,9 @@ int nsim_dev_devlink_port_new(struct devlink *devlink, const struct devlink_port
 			      struct netlink_ext_ack *extack);
 int nsim_dev_devlink_port_del(struct devlink *devlink, unsigned int port_index,
 			      struct netlink_ext_ack *extack);
+int nsim_dev_port_function_hw_addr_get(struct devlink *devlink, struct devlink_port *port,
+				       u8 *hw_addr, int *hw_addr_len,
+				       struct netlink_ext_ack *extack);
+int nsim_dev_port_function_hw_addr_set(struct devlink *devlink, struct devlink_port *port,
+				       const u8 *hw_addr, int hw_addr_len,
+				       struct netlink_ext_ack *extack);
drivers/net/netdevsim/port_function.c +44 -0

53 lines not on this page · raw message

Signature · 1 line
2.26.2
#37 lore raw

PATCH v2 7/8 netdevsim: Simulate port function state for a PCI port

In reply to Parav Pandit

Simulate port function state of a PCI port.
This enables users to get and set the state of the PCI port function.

Example of a PCI SF port which supports a port function:
Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device
$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port function set netdevsim/netdevsim10/10 hw_addr 00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/10
netdevsim/netdevsim10/10: type eth netdev eni10npf0 flavour pcipf controller 0 pfnum 0 external true splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/10 hw_addr 00:11:22:33:44:55 state active
$ devlink port show  netdevsim/netdevsim10/10 -jp
{
    "port": {
        "netdevsim/netdevsim10/10": {
            "type": "eth",
            "netdev": "eni10npf0",
            "flavour": "pcipf",
            "controller": 0,
            "pfnum": 0,
            "external": false,
            "splittable": false,
            "function": {
                "hw_addr": "00:11:22:33:44:55",
                "state": "active"
            }
        }
    }
}
---
 drivers/net/netdevsim/dev.c           |  2 ++
 drivers/net/netdevsim/netdevsim.h     |  6 ++++++
 drivers/net/netdevsim/port_function.c | 30 +++++++++++++++++++++++++++
 3 files changed, 38 insertions(+)
drivers/net/netdevsim/dev.c +2 -0
index ef2e293f358b..ec1e5dc74be1 100644
@@ -888,6 +888,8 @@ static const struct devlink_ops nsim_dev_devlink_ops = {
 	.port_del = nsim_dev_devlink_port_del,
 	.port_function_hw_addr_get = nsim_dev_port_function_hw_addr_get,
 	.port_function_hw_addr_set = nsim_dev_port_function_hw_addr_set,
+	.port_function_state_get = nsim_dev_port_function_state_get,
+	.port_function_state_set = nsim_dev_port_function_state_set,
 };
 
 #define NSIM_DEV_MAX_MACS_DEFAULT 32
drivers/net/netdevsim/netdevsim.h +6 -0
index 8dc8f4e5dcd8..0ea9705eda38 100644
@@ -308,3 +308,9 @@ int nsim_dev_port_function_hw_addr_get(struct devlink *devlink, struct devlink_p
 int nsim_dev_port_function_hw_addr_set(struct devlink *devlink, struct devlink_port *port,
 				       const u8 *hw_addr, int hw_addr_len,
 				       struct netlink_ext_ack *extack);
+int nsim_dev_port_function_state_get(struct devlink *devlink, struct devlink_port *port,
+				     enum devlink_port_function_state *state,
+				     struct netlink_ext_ack *extack);
+int nsim_dev_port_function_state_set(struct devlink *devlink, struct devlink_port *port,
+				     enum devlink_port_function_state state,
+				     struct netlink_ext_ack *extack);
drivers/net/netdevsim/port_function.c +30 -0
index 6feeeaf19ce8..99581d3d15fe 100644
@@ -16,6 +16,7 @@ struct nsim_port_function {
 	u16 pfnum;
 	struct nsim_port_function *pf_port; /* Valid only for SF port */
 	u8 hw_addr[ETH_ALEN];
+	u8 state; /* enum devlink_port_function_state */
 };
 
 void nsim_dev_port_function_init(struct nsim_dev *nsim_dev)
@@ -196,6 +197,7 @@ static int nsim_devlink_port_function_add(struct devlink *devlink, struct nsim_d
 
 	list_add(&port->list, &nsim_dev->port_functions.head);
 
+	port->state = DEVLINK_PORT_FUNCTION_STATE_INACTIVE;
 	err = devlink_port_register(devlink, &port->dl_port, port->port_index);
 	if (err)
 		goto reg_err;
@@ -379,3 +381,31 @@ int nsim_dev_port_function_hw_addr_set(struct devlink *devlink, struct devlink_p
 	memcpy(port->hw_addr, hw_addr, ETH_ALEN);
 	return 0;
 }
+
+int nsim_dev_port_function_state_get(struct devlink *devlink, struct devlink_port *dl_port,
+				     enum devlink_port_function_state *state,
+				     struct netlink_ext_ack *extack)
+{
+	struct nsim_dev *nsim_dev = devlink_priv(devlink);
+	struct nsim_port_function *port;
+
+	port = nsim_dev_to_port_function(nsim_dev, dl_port);
+	if (IS_ERR(port))
+		return PTR_ERR(port);
+	*state = port->state;
+	return 0;
+}
+
+int nsim_dev_port_function_state_set(struct devlink *devlink, struct devlink_port *dl_port,
+				     enum devlink_port_function_state state,
+				     struct netlink_ext_ack *extack)
+{
+	struct nsim_dev *nsim_dev = devlink_priv(devlink);
+	struct nsim_port_function *port;
+
+	port = nsim_dev_to_port_function(nsim_dev, dl_port);
+	if (IS_ERR(port))
+		return PTR_ERR(port);
+	port->state = state;
+	return 0;
+}
Signature · 1 line
2.26.2
#38 lore raw

PATCH v2 8/8 netdevsim: Add support for add and delete PCI SF port

In reply to Parav Pandit

Simulate PCI SF ports. Allow user to create one or more PCI SF ports.

Examples:

Create a PCI PF and PCI SF port.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0 sfnum 44
$ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour pcisf controller 0 pfnum 0 sfnum 44 external true splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr 00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/11 -jp
{
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0sf44",
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": true,
            "splittable": false,
            "function": {
                "hw_addr": "00:11:22:33:44:55",
                "state": "active"
            }
        }
    }
}

Delete newly added devlink port

$ devlink port add netdevsim/netdevsim10/11

Add devlink port of flavour 'pcisf' where port index and sfnum are auto assigned by driver.

$ devlink port add netdevsim/netdevsim10 flavour pcisf controller 0 pfnum 0
---
 drivers/net/netdevsim/netdevsim.h     |  1 +
 drivers/net/netdevsim/port_function.c | 95 +++++++++++++++++++++++++--
 2 files changed, 92 insertions(+), 4 deletions(-)
drivers/net/netdevsim/netdevsim.h +1 -0
index 0ea9705eda38..c70782e444d5 100644
@@ -222,6 +222,7 @@ struct nsim_dev {
 		struct list_head head;
 		struct ida ida;
 		struct ida pfnum_ida;
+		struct ida sfnum_ida;
 	} port_functions;
 };
 
drivers/net/netdevsim/port_function.c +91 -4

184 lines not on this page · raw message

Signature · 1 line
2.26.2
#39 lore raw

Re: [PATCH net-next v2 8/8] netdevsim: Add support for add and delete PCI SF port

Quoted 12 lines · Parav Pandit

Simulate PCI SF ports. Allow user to create one or more PCI SF ports.

Examples:

Create a PCI PF and PCI SF port.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0 sfnum 44
$ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour pcisf controller 0 pfnum 0 sfnum 44 external true splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
Quoted 8 lines
$ devlink port function set netdevsim/netdevsim10/11 hw_addr 00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/11 -jp
{
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0sf44",

I could be missing something, but it does not seem like this patch creates the netdevice for the subfunction.

Quoted 239 lines
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": true,
            "splittable": false,
            "function": {
                "hw_addr": "00:11:22:33:44:55",
                "state": "active"
            }
        }
    }
}

Delete newly added devlink port

$ devlink port add netdevsim/netdevsim10/11

Add devlink port of flavour 'pcisf' where port index and sfnum are auto assigned by driver.

$ devlink port add netdevsim/netdevsim10 flavour pcisf controller 0 pfnum 0
---
 drivers/net/netdevsim/netdevsim.h     |  1 +
 drivers/net/netdevsim/port_function.c | 95 +++++++++++++++++++++++++--
 2 files changed, 92 insertions(+), 4 deletions(-)
drivers/net/netdevsim/netdevsim.h +1 -0
index 0ea9705eda38..c70782e444d5 100644
@@ -222,6 +222,7 @@ struct nsim_dev {
 		struct list_head head;
 		struct ida ida;
 		struct ida pfnum_ida;
+		struct ida sfnum_ida;
 	} port_functions;
 };
 
drivers/net/netdevsim/port_function.c +91 -4
index 99581d3d15fe..e1812acd55b4 100644
@@ -13,10 +13,12 @@ struct nsim_port_function {
 	unsigned int port_index;
 	enum devlink_port_flavour flavour;
 	u32 controller;
+	u32 sfnum;
 	u16 pfnum;
 	struct nsim_port_function *pf_port; /* Valid only for SF port */
 	u8 hw_addr[ETH_ALEN];
 	u8 state; /* enum devlink_port_function_state */
+	int refcount; /* Counts how many sf ports are bound attached to this pf port. */
 };
 
 void nsim_dev_port_function_init(struct nsim_dev *nsim_dev)
@@ -25,10 +27,13 @@ void nsim_dev_port_function_init(struct nsim_dev *nsim_dev)
 	INIT_LIST_HEAD(&nsim_dev->port_functions.head);
 	ida_init(&nsim_dev->port_functions.ida);
 	ida_init(&nsim_dev->port_functions.pfnum_ida);
+	ida_init(&nsim_dev->port_functions.sfnum_ida);
 }
 
 void nsim_dev_port_function_exit(struct nsim_dev *nsim_dev)
 {
+	WARN_ON(!ida_is_empty(&nsim_dev->port_functions.sfnum_ida));
+	ida_destroy(&nsim_dev->port_functions.sfnum_ida);
 	WARN_ON(!ida_is_empty(&nsim_dev->port_functions.pfnum_ida));
 	ida_destroy(&nsim_dev->port_functions.pfnum_ida);
 	WARN_ON(!ida_is_empty(&nsim_dev->port_functions.ida));
@@ -119,9 +124,24 @@ nsim_devlink_port_function_alloc(struct nsim_dev *dev, const struct devlink_port
 			goto fn_ida_err;
 		port->pfnum = ret;
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		if (attrs->sfnum_valid)
+			ret = ida_alloc_range(&dev->port_functions.sfnum_ida, attrs->sfnum,
+					      attrs->sfnum, GFP_KERNEL);
+		else
+			ret = ida_alloc(&dev->port_functions.sfnum_ida, GFP_KERNEL);
+		if (ret < 0)
+			goto fn_ida_err;
+		port->sfnum = ret;
+		port->pfnum = attrs->pfnum;
+		break;
 	default:
 		break;
 	}
+	/* refcount_t is not needed as port is protected by port_functions.mutex.
+	 * This count is to keep track of how many SF ports are attached a PF port.
+	 */
+	port->refcount = 1;
 	return port;
 
 fn_ida_err:
@@ -137,6 +157,9 @@ static void nsim_devlink_port_function_free(struct nsim_dev *dev, struct nsim_po
 	case DEVLINK_PORT_FLAVOUR_PCI_PF:
 		ida_simple_remove(&dev->port_functions.pfnum_ida, port->pfnum);
 		break;
+	case DEVLINK_PORT_FLAVOUR_PCI_SF:
+		ida_simple_remove(&dev->port_functions.sfnum_ida, port->sfnum);
+		break;
 	default:
 		break;
 	}
@@ -170,6 +193,11 @@ nsim_dev_port_port_exists(struct nsim_dev *nsim_dev, const struct devlink_port_n
 		if (attrs->flavour == DEVLINK_PORT_FLAVOUR_PCI_PF &&
 		    tmp->flavour == DEVLINK_PORT_FLAVOUR_PCI_PF && tmp->pfnum == attrs->pfnum)
 			return true;
+
+		if (attrs->flavour == DEVLINK_PORT_FLAVOUR_PCI_SF &&
+		    tmp->flavour == DEVLINK_PORT_FLAVOUR_PCI_SF &&
+		    tmp->sfnum == attrs->sfnum && tmp->pfnum == attrs->pfnum)
+			return true;
 	}
 	return false;
 }
@@ -183,21 +211,71 @@ nsim_dev_devlink_port_index_lookup(const struct nsim_dev *nsim_dev, unsigned int
 	list_for_each_entry(port, &nsim_dev->port_functions.head, list) {
 		if (port->port_index != port_index)
 			continue;
+		if (port->refcount > 1) {
+			NL_SET_ERR_MSG_MOD(extack, "Port is in use");
+			return ERR_PTR(-EBUSY);
+		}
 		return port;
 	}
 	NL_SET_ERR_MSG_MOD(extack, "User created port not found");
 	return ERR_PTR(-ENOENT);
 }
 
+static struct nsim_port_function *
+pf_port_get(struct nsim_dev *nsim_dev, struct nsim_port_function *port)
+{
+	struct nsim_port_function *tmp;
+
+	/* PF port addition doesn't need a parent. */
+	if (port->flavour == DEVLINK_PORT_FLAVOUR_PCI_PF)
+		return NULL;
+
+	list_for_each_entry(tmp, &nsim_dev->port_functions.head, list) {
+		if (tmp->flavour != DEVLINK_PORT_FLAVOUR_PCI_PF || tmp->pfnum != port->pfnum)
+			continue;
+
+		if (tmp->refcount + 1 == INT_MAX)
+			return ERR_PTR(-ENOSPC);
+
+		port->pf_port = tmp;
+		tmp->refcount++;
+		return tmp;
+	}
+	return ERR_PTR(-ENOENT);
+}
+
+static void pf_port_put(struct nsim_port_function *port)
+{
+	if (port->pf_port) {
+		port->pf_port->refcount--;
+		WARN_ON(port->pf_port->refcount < 0);
+	}
+	port->refcount--;
+	WARN_ON(port->refcount != 0);
+}
+
 static int nsim_devlink_port_function_add(struct devlink *devlink, struct nsim_dev *nsim_dev,
 					  struct nsim_port_function *port,
 					  struct netlink_ext_ack *extack)
 {
+	struct nsim_port_function *pf_port;
 	int err;
 
-	list_add(&port->list, &nsim_dev->port_functions.head);
+	/* Keep all PF ports at the start, so that when driver is unloaded
+	 * All SF ports from the end of the list can be removed first.
+	 */
+	if (port->flavour == DEVLINK_PORT_FLAVOUR_PCI_PF)
+		list_add(&port->list, &nsim_dev->port_functions.head);
+	else
+		list_add_tail(&port->list, &nsim_dev->port_functions.head);
+
+	pf_port = pf_port_get(nsim_dev, port);
+	if (IS_ERR(pf_port)) {
+		NL_SET_ERR_MSG_MOD(extack, "Fail to get pf port");
+		err = PTR_ERR(pf_port);
+		goto pf_err;
+	}
 
-	port->state = DEVLINK_PORT_FUNCTION_STATE_INACTIVE;
 	err = devlink_port_register(devlink, &port->dl_port, port->port_index);
 	if (err)
 		goto reg_err;
@@ -213,6 +291,8 @@ static int nsim_devlink_port_function_add(struct devlink *devlink, struct nsim_d
 	devlink_port_type_clear(&port->dl_port);
 	devlink_port_unregister(&port->dl_port);
 reg_err:
+	pf_port_put(port);
+pf_err:
 	list_del(&port->list);
 	return err;
 }
@@ -224,12 +304,14 @@ static void nsim_devlink_port_function_del(struct nsim_dev *nsim_dev,
 	unregister_netdev(port->netdev);
 	devlink_port_unregister(&port->dl_port);
 	list_del(&port->list);
+	pf_port_put(port);
 }
 
 static bool nsim_dev_port_flavour_supported(const struct nsim_dev *nsim_dev,
 					    const struct devlink_port_new_attrs *attrs)
 {
-	return attrs->flavour == DEVLINK_PORT_FLAVOUR_PCI_PF;
+	return attrs->flavour == DEVLINK_PORT_FLAVOUR_PCI_PF ||
+	       attrs->flavour == DEVLINK_PORT_FLAVOUR_PCI_SF;
 }
 
 int nsim_dev_devlink_port_new(struct devlink *devlink, const struct devlink_port_new_attrs *attrs,
@@ -266,7 +348,11 @@ int nsim_dev_devlink_port_new(struct devlink *devlink, const struct devlink_port
 	       nsim_dev->switch_id.id_len);
 	port->dl_port.attrs.switch_id.id_len = nsim_dev->switch_id.id_len;
 
-	devlink_port_attrs_pci_pf_set(&port->dl_port, port->controller, port->pfnum, false);
+	if (attrs->flavour == DEVLINK_PORT_FLAVOUR_PCI_PF)
+		devlink_port_attrs_pci_pf_set(&port->dl_port, port->controller, port->pfnum, false);
+	else
+		devlink_port_attrs_pci_sf_set(&port->dl_port, port->controller, port->pfnum,
+					      port->sfnum, false);
 
 	err = nsim_devlink_port_function_add(devlink, nsim_dev, port, extack);
 	if (err)
@@ -333,6 +419,7 @@ void nsim_dev_port_function_disable(struct nsim_dev *nsim_dev)
 	 * ports.
 	 */
 
+	/* Remove SF ports first, followed by PF ports. */
 	list_for_each_entry_safe_reverse(port, tmp, &nsim_dev->port_functions.head, list) {
 		nsim_devlink_port_function_del(nsim_dev, port);
 		nsim_devlink_port_function_free(nsim_dev, port);
#40 lore raw

RE: [PATCH net-next v2 8/8] netdevsim: Add support for add and delete PCI SF port

Quoted 21 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 2:01 AM

Quoted 9 lines · Parav Pandit

Simulate PCI SF ports. Allow user to create one or more PCI SF ports.

Examples:

Create a PCI PF and PCI SF port.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0 $
devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0 sfnum
44 $ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour pcisf

controller 0 pfnum 0 sfnum 44 external true splittable false

Quoted 6 lines

function:

    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr
00:11:22:33:44:55 state active
Quoted 9 lines
Quoted 5 lines
$ devlink port show netdevsim/netdevsim10/11 -jp {
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0sf44",

I could be missing something, but it does not seem like this patch creates the netdevice for the subfunction.

The sf port created here is the eswitch port with a valid switch id similar to PF and physical port. So the netdev created is the representor netdevice.
It is created uniformly for subfunction and pf port flavours.

This series enables user to add PCI PF and subfunction ports.
PF port addition (and its representor netdev) is done in patch 5 [1]. This patch for SF utilizes the same ' struct nsim_port_function' for PF and SF. Only difference between them is flavour.
[1] https://lore.kernel.org/netdev/[email protected]/

Quoted 22 lines
Quoted 22 lines
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": true,
            "splittable": false,
            "function": {
                "hw_addr": "00:11:22:33:44:55",
                "state": "active"
            }
        }
    }
}

Delete newly added devlink port

$ devlink port add netdevsim/netdevsim10/11

Add devlink port of flavour 'pcisf' where port index and sfnum are auto assigned by driver.

$ devlink port add netdevsim/netdevsim10 flavour pcisf controller 0
pfnum 0
#41 lore raw

Re: [PATCH net-next v2 8/8] netdevsim: Add support for add and delete PCI SF port

Quoted 20 lines · Parav Pandit
Quoted 20 lines
Quoted 7 lines

Examples:

Create a PCI PF and PCI SF port.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0 $
devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0 sfnum
44 $ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour pcisf

controller 0 pfnum 0 sfnum 44 external true splittable false

Quoted 11 lines

function:

    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr
00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/11 -jp {
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0sf44",
Quoted 6 lines
Quoted 3 lines

I could be missing something, but it does not seem like this patch creates the netdevice for the subfunction.

The sf port created here is the eswitch port with a valid switch id similar to PF and physical port. So the netdev created is the representor netdevice.
It is created uniformly for subfunction and pf port flavours.

To be clear: If I run the devlink commands to create a sub-function, ip link show should list a net_device that corresponds to the sub-function?

#42 lore raw

RE: [PATCH net-next v2 8/8] netdevsim: Add support for add and delete PCI SF port

Hi David,

Quoted 29 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 9:08 AM

Quoted 25 lines · Parav Pandit
Quoted 25 lines
Quoted 9 lines

Examples:

Create a PCI PF and PCI SF port.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0 $
devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0
sfnum
44 $ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour
pcisf

controller 0 pfnum 0 sfnum 44 external true splittable false

Quoted 11 lines

function:

    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr
00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/11 -jp {
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0sf44",

I could be missing something, but it does not seem like this patch creates the netdevice for the subfunction.

Quoted 7 lines
Quoted 1 line

The sf port created here is the eswitch port with a valid switch id similar to PF

and physical port.

Quoted 2 lines

So the netdev created is the representor netdevice. It is created uniformly for subfunction and pf port flavours.

To be clear: If I run the devlink commands to create a sub-function, ip link show should list a net_device that corresponds to the sub-function?

In this series only representor netdevice corresponds to sub-function will be visible in ip link show, i.e. eni10npf0sf44.

Netdevsim is only simulating the eswitch side or control path at present for pf/vf/sf ports. So other end of this port (netdevice/rdma device/vdpa device) are not yet created.

Subfunction will be anchored on virtbus described in RFC [1], which is not yet in-kernel yet. Grep for "every SF a device is created on virtbus" to jump to this part of the long RFC.

[1] https://lore.kernel.org/netdev/20200519092258.GF4655@nanopsycho/

#43 lore raw

Re: [PATCH net-next v2 8/8] netdevsim: Add support for add and delete PCI SF port

Quoted 32 lines · Parav Pandit

Hi David,

Quoted 30 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 9:08 AM

Quoted 26 lines · Parav Pandit
Quoted 16 lines
Quoted 9 lines

Examples:

Create a PCI PF and PCI SF port.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0 $
devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0
sfnum
44 $ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour
pcisf

controller 0 pfnum 0 sfnum 44 external true splittable false

Quoted 6 lines
   function:
     hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr
00:11:22:33:44:55 state active
Quoted 9 lines
Quoted 5 lines
$ devlink port show netdevsim/netdevsim10/11 -jp {
     "port": {
         "netdevsim/netdevsim10/11": {
             "type": "eth",
             "netdev": "eni10npf0sf44",

I could be missing something, but it does not seem like this patch creates the netdevice for the subfunction.

The sf port created here is the eswitch port with a valid switch id similar to PF

Quoted 8 lines
Quoted 6 lines

and physical port.

Quoted 2 lines

So the netdev created is the representor netdevice. It is created uniformly for subfunction and pf port flavours.

To be clear: If I run the devlink commands to create a sub-function, ip link show should list a net_device that corresponds to the sub-function?

In this series only representor netdevice corresponds to sub-function will be visible in ip link show, i.e. eni10npf0sf44.

This should be OK if the eSwitch mode is changed to switchdev.
But i think it should be possible to create a subfunction even in legacy mode where representor netdev is not created.

Quoted 9 lines

Netdevsim is only simulating the eswitch side or control path at present for pf/vf/sf ports. So other end of this port (netdevice/rdma device/vdpa device) are not yet created.

Subfunction will be anchored on virtbus described in RFC [1], which is not yet in-kernel yet. Grep for "every SF a device is created on virtbus" to jump to this part of the long RFC.

[1] https://lore.kernel.org/netdev/20200519092258.GF4655@nanopsycho/

#44 lore raw

RE: [PATCH net-next v2 8/8] netdevsim: Add support for add and delete PCI SF port

Quoted 47 lines

From: Samudrala, Sridhar <[email protected]>
Sent: Friday, September 18, 2020 10:23 AM

Quoted 40 lines · Parav Pandit

Hi David,

Quoted 37 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 9:08 AM

Quoted 22 lines · Parav Pandit
Quoted 22 lines
Quoted 9 lines

Examples:

Create a PCI PF and PCI SF port.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0
sfnum
44 $ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour
pcisf

controller 0 pfnum 0 sfnum 44 external true splittable false

Quoted 11 lines
   function:
     hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr
00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/11 -jp {
     "port": {
         "netdevsim/netdevsim10/11": {
             "type": "eth",
             "netdev": "eni10npf0sf44",
Quoted 5 lines
Quoted 3 lines

I could be missing something, but it does not seem like this patch creates the netdevice for the subfunction.

The sf port created here is the eswitch port with a valid switch id similar to PF

and physical port.

Quoted 2 lines

So the netdev created is the representor netdevice. It is created uniformly for subfunction and pf port flavours.

To be clear: If I run the devlink commands to create a sub-function, ip link show should list a net_device that corresponds to the sub-function?

Quoted 6 lines
Quoted 1 line

In this series only representor netdevice corresponds to sub-function will be

visible in ip link show, i.e. eni10npf0sf44.

This should be OK if the eSwitch mode is changed to switchdev. But i think it should be possible to create a subfunction even in legacy mode where representor netdev is not created.

switch_id is optional attribute of the devlink port. It is applicable only when it is eswitch port in switchdev mode.

Quoted 14 lines
Quoted 3 lines

Netdevsim is only simulating the eswitch side or control path at present for

pf/vf/sf ports.

Quoted 1 line

So other end of this port (netdevice/rdma device/vdpa device) are not yet

created.

Quoted 2 lines

Subfunction will be anchored on virtbus described in RFC [1], which is not yet

in-kernel yet.

Quoted 1 line

Grep for "every SF a device is created on virtbus" to jump to this part of the

long RFC.

Quoted 3 lines

[1] https://lore.kernel.org/netdev/20200519092258.GF4655@nanopsycho/

#45 lore raw

Re: [PATCH net-next v2 8/8] netdevsim: Add support for add and delete PCI SF port

In reply to Parav Pandit
Quoted 41 lines · Parav Pandit

Hi David,

Quoted 29 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 9:08 AM

Quoted 25 lines · Parav Pandit
Quoted 25 lines
Quoted 9 lines

Examples:

Create a PCI PF and PCI SF port.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0 $
devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0
sfnum
44 $ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour
pcisf

controller 0 pfnum 0 sfnum 44 external true splittable false

Quoted 11 lines

function:

    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr
00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/11 -jp {
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0sf44",

I could be missing something, but it does not seem like this patch creates the netdevice for the subfunction.

Quoted 7 lines
Quoted 1 line

The sf port created here is the eswitch port with a valid switch id similar to PF

and physical port.

Quoted 2 lines

So the netdev created is the representor netdevice. It is created uniformly for subfunction and pf port flavours.

To be clear: If I run the devlink commands to create a sub-function, ip link show should list a net_device that corresponds to the sub-function?

In this series only representor netdevice corresponds to sub-function will be visible in ip link show, i.e. eni10npf0sf44.

Quoted 8 lines

Netdevsim is only simulating the eswitch side or control path at present for pf/vf/sf ports. So other end of this port (netdevice/rdma device/vdpa device) are not yet created.

Subfunction will be anchored on virtbus described in RFC [1], which is not yet in-kernel yet. Grep for "every SF a device is created on virtbus" to jump to this part of the long RFC.

[1] https://lore.kernel.org/netdev/20200519092258.GF4655@nanopsycho/

Thanks for the reference. I have seen that. I am interested in this idea of creating netdevs for 'slices' of an asic or nic, but it is not clear to me how it connects end to end. Will you be able to create a sub-function based netdevice, assign it limited resources from the nic and then assign that netdevice to a container for example?

#46 lore raw

RE: [PATCH net-next v2 8/8] netdevsim: Add support for add and delete PCI SF port

Quoted 60 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 8:54 PM

Quoted 39 lines · Parav Pandit

Hi David,

Quoted 37 lines

From: David Ahern <[email protected]>
Sent: Friday, September 18, 2020 9:08 AM

Quoted 22 lines · Parav Pandit
Quoted 22 lines
Quoted 9 lines

Examples:

Create a PCI PF and PCI SF port.

$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0
sfnum
44 $ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour
pcisf

controller 0 pfnum 0 sfnum 44 external true splittable false

Quoted 11 lines

function:

    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr
00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/11 -jp {
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0sf44",
Quoted 5 lines
Quoted 3 lines

I could be missing something, but it does not seem like this patch creates the netdevice for the subfunction.

The sf port created here is the eswitch port with a valid switch id similar to PF

and physical port.

Quoted 2 lines

So the netdev created is the representor netdevice. It is created uniformly for subfunction and pf port flavours.

To be clear: If I run the devlink commands to create a sub-function, ip link show should list a net_device that corresponds to the sub-

function?

Quoted 2 lines

In this series only representor netdevice corresponds to sub-function will

be visible in ip link show, i.e. eni10npf0sf44.

Quoted 2 lines

Netdevsim is only simulating the eswitch side or control path at present for

pf/vf/sf ports.

Quoted 1 line

So other end of this port (netdevice/rdma device/vdpa device) are not yet

created.

Quoted 2 lines

Subfunction will be anchored on virtbus described in RFC [1], which is not

yet in-kernel yet.

Quoted 1 line

Grep for "every SF a device is created on virtbus" to jump to this part of the

long RFC.

Quoted 1 line
Quoted 8 lines
Quoted 2 lines

[1] https://lore.kernel.org/netdev/20200519092258.GF4655@nanopsycho/

Thanks for the reference. I have seen that. I am interested in this idea of creating netdevs for 'slices' of an asic or nic, but it is not clear to me how it connects end to end. Will you be able to create a sub-function based netdevice, assign it limited resources from the nic and then assign that netdevice to a container for example?

Yep. You described is precisely well.
This short series creates eswitch representor for the moment.
Once virtbus is in_kernel, will extend to create actual netdevice of the subfunction too.

maintainer
#47 lore raw
In reply to Parav Pandit
Quoted 5 lines · On Thu, 17 Sep 2020 20:20:12 +0300 Parav Pandit wrote:

Hi Dave, Jakub,

Similar to PCI VF, PCI SF represents portion of the device. PCI SF is represented using a new devlink port flavour.

Quoted 7 lines

This short series implements small part of the RFC described in detail at [1] and [2].

It extends
(a) devlink core to expose new devlink port flavour 'pcisf'.
(b) Expose new user interface to add/delete devlink port.
(c) Extends netdevsim driver to simulate PCI PF and SF ports
(d) Add port function state attribute

Is this an RFC? It doesn't add any in-tree users.

#48 lore raw
Quoted 13 lines

From: Jakub Kicinski <[email protected]>
Sent: Friday, September 18, 2020 10:22 PM

Quoted 6 lines · On Thu, 17 Sep 2020 20:20:12 +0300 Parav Pandit wrote:

Hi Dave, Jakub,

Similar to PCI VF, PCI SF represents portion of the device. PCI SF is represented using a new devlink port flavour.

This short series implements small part of the RFC described in detail at [1]

and [2].

Quoted 1 line
Quoted 7 lines
Quoted 5 lines

It extends
(a) devlink core to expose new devlink port flavour 'pcisf'.
(b) Expose new user interface to add/delete devlink port.
(c) Extends netdevsim driver to simulate PCI PF and SF ports
(d) Add port function state attribute

Is this an RFC? It doesn't add any in-tree users.

It is not an RFC.
devlink + mlx5 + netdevsim is crossing 25+ patches on eswitch side.
So splitting it to logical piece as devlink + netdevsim.
After which mlx5 eswitch side come close to 15 + 4 patches which can run as two separate patchset.

What do you suggest?

maintainer
#49 lore raw
Quoted 18 lines · On Fri, 18 Sep 2020 17:08:15 +0000 Parav Pandit wrote:
Quoted 18 lines

From: Jakub Kicinski <[email protected]>
Sent: Friday, September 18, 2020 10:22 PM

Quoted 6 lines · On Thu, 17 Sep 2020 20:20:12 +0300 Parav Pandit wrote:

Hi Dave, Jakub,

Similar to PCI VF, PCI SF represents portion of the device. PCI SF is represented using a new devlink port flavour.

This short series implements small part of the RFC described in detail at [1]

and [2].

Quoted 6 lines

It extends
(a) devlink core to expose new devlink port flavour 'pcisf'.
(b) Expose new user interface to add/delete devlink port.
(c) Extends netdevsim driver to simulate PCI PF and SF ports
(d) Add port function state attribute

Quoted 8 lines
Quoted 1 line

Is this an RFC? It doesn't add any in-tree users.

It is not an RFC.
devlink + mlx5 + netdevsim is crossing 25+ patches on eswitch side. So splitting it to logical piece as devlink + netdevsim. After which mlx5 eswitch side come close to 15 + 4 patches which can run as two separate patchset.

What do you suggest?

Start with real patches, not netdevsim.

#50 lore raw
Quoted 32 lines

From: Jakub Kicinski <[email protected]>
Sent: Friday, September 18, 2020 11:07 PM

Quoted 26 lines · On Fri, 18 Sep 2020 17:08:15 +0000 Parav Pandit wrote:
Quoted 13 lines

From: Jakub Kicinski <[email protected]>
Sent: Friday, September 18, 2020 10:22 PM

Quoted 7 lines · On Thu, 17 Sep 2020 20:20:12 +0300 Parav Pandit wrote:

Hi Dave, Jakub,

Similar to PCI VF, PCI SF represents portion of the device. PCI SF is represented using a new devlink port flavour.

This short series implements small part of the RFC described in detail at [1]

and [2].

Quoted 1 line
Quoted 7 lines
Quoted 5 lines

It extends
(a) devlink core to expose new devlink port flavour 'pcisf'.
(b) Expose new user interface to add/delete devlink port.
(c) Extends netdevsim driver to simulate PCI PF and SF ports
(d) Add port function state attribute

Is this an RFC? It doesn't add any in-tree users.

It is not an RFC.
devlink + mlx5 + netdevsim is crossing 25+ patches on eswitch side. So splitting it to logical piece as devlink + netdevsim. After which mlx5 eswitch side come close to 15 + 4 patches which can run as two separate patchset.

Quoted 3 lines
Quoted 1 line

What do you suggest?

Start with real patches, not netdevsim.

Hmm. Shall I split the series below, would that be ok?

First patchset,
(a) devlink piece to add/delete port
(b) mlx5 counter part

Second patchset,
(a) devlink piece to set the state
(b) mlx5 counter part

Follow on patchset to create/delete sf's netdev on virtbus in mlx5 + devlink plumbing. Netdevsim after that.

maintainer
#51 lore raw
Quoted 10 lines · On Fri, 18 Sep 2020 17:47:24 +0000 Parav Pandit wrote:
Quoted 3 lines
Quoted 1 line

What do you suggest?

Start with real patches, not netdevsim.

Hmm. Shall I split the series below, would that be ok?

First patchset,
(a) devlink piece to add/delete port
(b) mlx5 counter part

Quoted 6 lines

Second patchset,
(a) devlink piece to set the state
(b) mlx5 counter part

Follow on patchset to create/delete sf's netdev on virtbus in mlx5 + devlink plumbing. Netdevsim after that.

I'd start from the virtbus part so we can see what's being created.

#52 lore raw
Quoted 19 lines

From: Jakub Kicinski <[email protected]>
Sent: Friday, September 18, 2020 11:58 PM

Quoted 14 lines · On Fri, 18 Sep 2020 17:47:24 +0000 Parav Pandit wrote:
Quoted 3 lines
Quoted 1 line

What do you suggest?

Start with real patches, not netdevsim.

Hmm. Shall I split the series below, would that be ok?

First patchset,
(a) devlink piece to add/delete port
(b) mlx5 counter part

Second patchset,
(a) devlink piece to set the state
(b) mlx5 counter part

Quoted 5 lines
Quoted 1 line

Follow on patchset to create/delete sf's netdev on virtbus in mlx5 + devlink

plumbing.

Quoted 1 line

Netdevsim after that.

I'd start from the virtbus part so we can see what's being created.

How do you reach there without a user interface?

maintainer
#53 lore raw
Quoted 18 lines · On Fri, 18 Sep 2020 20:09:24 +0000 Parav Pandit wrote:
Quoted 18 lines

From: Jakub Kicinski <[email protected]>
Sent: Friday, September 18, 2020 11:58 PM

Quoted 14 lines · On Fri, 18 Sep 2020 17:47:24 +0000 Parav Pandit wrote:
Quoted 3 lines
Quoted 1 line

What do you suggest?

Start with real patches, not netdevsim.

Hmm. Shall I split the series below, would that be ok?

First patchset,
(a) devlink piece to add/delete port
(b) mlx5 counter part

Second patchset,
(a) devlink piece to set the state
(b) mlx5 counter part

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Follow on patchset to create/delete sf's netdev on virtbus in mlx5 + devlink

plumbing.

Quoted 1 line

Netdevsim after that.

I'd start from the virtbus part so we can see what's being created.

How do you reach there without a user interface?

Well.. why do you have a user interface which doesn't cause anything to happen (devices won't get created)? You're splitting the submission, it's obvious the implementation won't be complete after the first one.

My expectation is that your implementation of the devlink commands will just hand them off to FW, so there won't be anything interesting there to review.

Start with the hard / risky parts - I consider the virtbus to be that, since the conversation there includes multiple vendors, use cases and subsystems.

#54 lore raw
Quoted 31 lines

From: Jakub Kicinski <[email protected]>
Sent: Tuesday, September 22, 2020 3:32 AM

Quoted 25 lines · On Fri, 18 Sep 2020 20:09:24 +0000 Parav Pandit wrote:
Quoted 24 lines

From: Jakub Kicinski <[email protected]>
Sent: Friday, September 18, 2020 11:58 PM

Quoted 10 lines · On Fri, 18 Sep 2020 17:47:24 +0000 Parav Pandit wrote:
Quoted 3 lines
Quoted 1 line

What do you suggest?

Start with real patches, not netdevsim.

Hmm. Shall I split the series below, would that be ok?

First patchset,
(a) devlink piece to add/delete port
(b) mlx5 counter part

Quoted 6 lines

Second patchset,
(a) devlink piece to set the state
(b) mlx5 counter part

Follow on patchset to create/delete sf's netdev on virtbus in mlx5
+ devlink

plumbing.

Quoted 1 line

Netdevsim after that.

I'd start from the virtbus part so we can see what's being created.

Quoted 8 lines
Quoted 1 line

How do you reach there without a user interface?

Well.. why do you have a user interface which doesn't cause anything to happen (devices won't get created)? You're splitting the submission, it's obvious the implementation won't be complete after the first one.

My expectation is that your implementation of the devlink commands will just hand them off to FW, so there won't be anything interesting there to review.

Correct. Since handing off to firmware and processing event from firmware is creating fairly more patches, In first series I will just go inline where devlink command would create/remove virtbus device on state active/inactive.

This way it should be possible to have minimal working series under 20 patches. This way in one patchset, I prefer to cover
(a) devlink plumbing for port add/del, port state
(b) mlx5 eswitch refactor and devlink callback handling
(c) devlink plumbing around
(d) virtbus device and their netdev creation

This also gives complete view from user interface to netdev device creation.
Post this series, will improve the internals of mlx5 via events etc which doesn't need multi-vendor, virtbus and devlink involvement. Will differ netdevsim to later date.

Quoted 3 lines

Start with the hard / risky parts - I consider the virtbus to be that, since the conversation there includes multiple vendors, use cases and subsystems.

#55 lore raw

PATCH 8/8 netdevsim: Add support for add and delete PCI SF port

In reply to Parav Pandit

Simulate PCI SF ports. Allow user to create one or more PCI SF ports.

Examples:

Create a PCI PF and PCI SF port.
Create a device with ID=10 and one physical port.

$ echo "10 1" > /sys/bus/netdevsim/new_device
$ devlink port show
netdevsim/netdevsim10/0: type eth netdev eni10np1 flavour physical port 1 splittable false
$ devlink port add netdevsim/netdevsim10/10 flavour pcipf pfnum 0
$ devlink port add netdevsim/netdevsim10/11 flavour pcisf pfnum 0 sfnum 44
$ devlink port show netdevsim/netdevsim10/11
netdevsim/netdevsim10/11: type eth netdev eni10npf0sf44 flavour pcisf controller 0 pfnum 0 sfnum 44 external true splittable false
  function:
    hw_addr 00:00:00:00:00:00 state inactive
$ devlink port function set netdevsim/netdevsim10/11 hw_addr 00:11:22:33:44:55 state active
$ devlink port show netdevsim/netdevsim10/11 -jp
{
    "port": {
        "netdevsim/netdevsim10/11": {
            "type": "eth",
            "netdev": "eni10npf0sf44",
            "flavour": "pcisf",
            "controller": 0,
            "pfnum": 0,
            "sfnum": 44,
            "external": true,
            "splittable": false,
            "function": {
                "hw_addr": "00:11:22:33:44:55",
                "state": "active"
            }
        }
    }
}

Delete newly added devlink port

$ devlink port add netdevsim/netdevsim10/11

Add devlink port of flavour 'pcisf' where port index and sfnum are auto assigned by driver.

$ devlink port add netdevsim/netdevsim10 flavour pcisf controller 0 pfnum 0
---
 drivers/net/netdevsim/netdevsim.h     |  1 +
 drivers/net/netdevsim/port_function.c | 95 +++++++++++++++++++++++++--
 2 files changed, 92 insertions(+), 4 deletions(-)
drivers/net/netdevsim/netdevsim.h +1 -0
index 0ea9705eda38..c70782e444d5 100644
@@ -222,6 +222,7 @@ struct nsim_dev {
 		struct list_head head;
 		struct ida ida;
 		struct ida pfnum_ida;
+		struct ida sfnum_ida;
 	} port_functions;
 };
 
drivers/net/netdevsim/port_function.c +91 -4

184 lines not on this page · raw message

Signature · 1 line
2.26.2