Skip to contentSkip to conversation

extend set/get netlink for embedded

PATCH In review John Fastabend 4 participants 13 messages Oct 24, 2012 – Nov 13, 2012

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v2 cover extend set/get netlink for embedded

This extends the PF_BRIDGE setlink and getlink so that they can be used generically outside of the Linux bridge module. Doing this allows embedded devices to use the same netlink interface that the software bridge is currently using.

In this patchset I opted to create two new ndo ops ndo_bridge_setlink and ndo_bridge_getlink. These ops pass the nlmsghdr to the device for handling. The netlink message is extended to support nested IFLA_AF_SPEC attributes. A IFLA_BRIDGE_FLAGS attribute is used to determine the target of the message either a master netdev is the target or the target is "self" indicating the target is the netdev. In this way we can send netlink msg to an embedded device or the linux sw bridge or both. If the set completes sucessfully the flag is cleared in this way we can learn what failed in the both case. This scheme is similar to how FDB updates are handled. If no flag attribute is present the message is sent to the master device to support the existing messages.

An initial IFLA_BRIDGE_MODE attribute is added to indicate if the bridge is in a VEPA mode or VEB mode. This is most useful for SR-IOV device and can be used to indicate support for VF to VF or VF to PF traffic. Without this we have no way of knowing this other then trial and error because not all hardware supports this.

In the future additional attributes can be added to handle other attributes. We could for example move some of the linux bridge sysfs attributes here if we want a netlink interface for them. This should also allow DSA switches to use the bridging tools. See RFC patch from Lennert

http://patchwork.ozlabs.org/patch/16578/

I could have simply added the VEPA attribute handling to the rtnl_setlink() routine but it seemed like a good feature to have all the bridging events and configuration on the same type. Also this should allow more powerful offloaded switches like the hardware supported via DSA access to the bridge control interface.

Also I think this interface should allow Vlad to add his port based VLAN interface here as well.

Any feedback/criticism appreciated thanks!

---

John Fastabend (3):
ixgbe: add setlink, getlink support to ixgbe and ixgbevf
net: set and query VEB/VEPA bridge mode via PF_BRIDGE
net: create generic bridge ops

 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c     |   59 ++++++
 drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c    |    3 
 drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c |   10 +
 include/linux/netdevice.h                         |   10 +
 include/linux/rtnetlink.h                         |    3 
 include/uapi/linux/if_bridge.h                    |   18 ++
 net/bridge/br_device.c                            |    2 
 net/bridge/br_netlink.c                           |   75 +-------
 net/bridge/br_private.h                           |    7 +
 net/core/rtnetlink.c                              |  207 +++++++++++++++++++++
 10 files changed, 328 insertions(+), 66 deletions(-)
Signature · 1 line
Signature
#1 lore raw

PATCH v2 1/3 net: create generic bridge ops

The PF_BRIDGE:RTM_{GET|SET}LINK nlmsg family and type are currently embedded in the ./net/bridge module. This prohibits them from being used by other bridging devices. One example of this being hardware that has embedded bridging components.

In order to use these nlmsg types more generically this patch adds two net_device_ops hooks. One to set link bridge attributes and another to dump the current bride attributes.

ndo_bridge_setlink()
ndo_bridge_getlink()

CC Lennert Buytenhek <[email protected]>
CC Stephen Hemminger <[email protected]>

---

 include/linux/netdevice.h |   10 ++++++
 net/bridge/br_device.c    |    2 +
 net/bridge/br_netlink.c   |   73 +++++++----------------------------------
 net/bridge/br_private.h   |    3 ++
 net/core/rtnetlink.c      |   80 +++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 108 insertions(+), 60 deletions(-)
include/linux/netdevice.h +10 -0
index f8eda02..7bf867c 100644
@@ -887,6 +887,10 @@ struct netdev_fcoe_hbainfo {
  *		       struct net_device *dev, int idx)
  *	Used to add FDB entries to dump requests. Implementers should add
  *	entries to skb and update idx with the number of entries.
+ *
+ * int (*ndo_bridge_setlink)(struct net_device *dev, struct nlmsghdr *nlh)
+ * int (*ndo_bridge_getlink)(struct sk_buff *skb, u32 pid, u32 seq,
+ *			     struct net_device *dev)
  */
 struct net_device_ops {
 	int			(*ndo_init)(struct net_device *dev);
@@ -998,6 +1002,12 @@ struct net_device_ops {
 						struct netlink_callback *cb,
 						struct net_device *dev,
 						int idx);
+
+	int			(*ndo_bridge_setlink)(struct net_device *dev,
+						      struct nlmsghdr *nlh);
+	int			(*ndo_bridge_getlink)(struct sk_buff *skb,
+						      u32 pid, u32 seq,
+						      struct net_device *dev);
 };
 
 /*
net/bridge/br_device.c +2 -0
index 070e8a6..63b5b08 100644
@@ -313,6 +313,8 @@ static const struct net_device_ops br_netdev_ops = {
 	.ndo_fdb_add		 = br_fdb_add,
 	.ndo_fdb_del		 = br_fdb_delete,
 	.ndo_fdb_dump		 = br_fdb_dump,
+	.ndo_bridge_getlink	 = br_getlink,
+	.ndo_bridge_setlink	 = br_setlink,
 };
 
 static void br_dev_free(struct net_device *dev)
net/bridge/br_netlink.c +13 -60

106 lines not on this page · raw message

net/bridge/br_private.h +3 -0
index 9b278c4..fdcd5f6 100644
@@ -553,6 +553,9 @@ extern struct rtnl_link_ops br_link_ops;
 extern int br_netlink_init(void);
 extern void br_netlink_fini(void);
 extern void br_ifinfo_notify(int event, struct net_bridge_port *port);
+extern int br_setlink(struct net_device *dev, struct nlmsghdr *nlmsg);
+extern int br_getlink(struct sk_buff *skb, u32 pid, u32 seq,
+		      struct net_device *dev);
 
 #ifdef CONFIG_SYSFS
 /* br_sysfs_if.c */
net/core/rtnetlink.c +80 -0

91 lines not on this page · raw message

#4 lore raw

PATCH v2 2/3 net: set and query VEB/VEPA bridge mode via PF_BRIDGE

In reply to John Fastabend

Hardware switches may support enabling and disabling the loopback switch which puts the device in a VEPA mode defined in the IEEE 802.1Qbg specification. In this mode frames are not switched in the hardware but sent directly to the switch. SR-IOV capable NICs will likely support this mode I am aware of at least two such devices. Also I am told (but don't have any of this hardware available) that there are devices that only support VEPA modes. In these cases it is important at a minimum to be able to query these attributes.

This patch adds an additional IFLA_BRIDGE_MODE attribute that can be set and dumped via the PF_BRIDGE:{SET|GET}LINK operations. Also anticipating bridge attributes that may be common for both embedded bridges and software bridges this adds a flags attribute IFLA_BRIDGE_FLAGS currently used to determine if the command or event is being generated to/from an embedded bridge or software bridge. Finally, the event generation is pulled out of the bridge module and into rtnetlink proper.

For example using the macvlan driver in VEPA mode on top of an embedded switch requires putting the embedded switch into a VEPA mode to get the expected results.

	--------  --------
        | VEPA |  | VEPA |       <-- macvlan vepa edge relays
        --------  --------
           |        |
           |        |
        ------------------
        |      VEPA      |       <-- embedded switch in NIC
        ------------------
                |
                |
        -------------------
        | external switch |      <-- shiny new physical
	-------------------          switch with VEPA support

A packet sent from the macvlan VEPA at the top could be loopbacked on the embedded switch and never seen by the external switch. So in order for this to work the embedded switch needs to be set in the VEPA state via the above described commands.

By making these attributes nested in IFLA_AF_SPEC we allow future extensions to be made as needed.

CC Lennert Buytenhek <[email protected]>
CC Stephen Hemminger <[email protected]>

---

 include/uapi/linux/if_bridge.h |   18 ++++++++
 net/bridge/br_netlink.c        |    2 -
 net/bridge/br_private.h        |    4 +-
 net/core/rtnetlink.c           |   85 ++++++++++++++++++++++++++++++++++++++--
 4 files changed, 102 insertions(+), 7 deletions(-)
include/uapi/linux/if_bridge.h +18 -0
index a8fe954..b388579 100644
@@ -97,5 +97,23 @@ struct __fdb_entry {
 	__u16 unused;
 };
 
+/* Bridge Flags */
+#define BRIDGE_FLAGS_MASTER	1	/* Bridge command to/from master */
+#define BRIDGE_FLAGS_SELF	2	/* Bridge command to/from lowerdev */
 
+#define BRIDGE_MODE_VEB		0	/* Default loopback mode */
+#define BRIDGE_MODE_VEPA	1	/* 802.1Qbg defined VEPA mode */
+
+/* Bridge management nested attributes
+ * [IFLA_AF_SPEC] = {
+ *     [IFLA_BRIDGE_FLAGS]
+ *     [IFLA_BRIDGE_MODE]
+ * }
+ */
+enum {
+	IFLA_BRIDGE_FLAGS,
+	IFLA_BRIDGE_MODE,
+	__IFLA_BRIDGE_MAX,
+};
+#define IFLA_BRIDGE_MAX (__IFLA_BRIDGE_MAX - 1)
 #endif /* _UAPI_LINUX_IF_BRIDGE_H */
net/bridge/br_netlink.c +0 -2
index 743511b..14b065c 100644
@@ -166,8 +166,6 @@ int br_setlink(struct net_device *dev, struct nlmsghdr *nlh)
 	br_port_state_selection(p->br);
 	spin_unlock_bh(&p->br->lock);
 
-	br_ifinfo_notify(RTM_NEWLINK, p);
-
 	return 0;
 }
 
net/bridge/br_private.h +3 -1
index fdcd5f6..6f40c14 100644
@@ -158,7 +158,9 @@ struct net_bridge_port
 
 static inline struct net_bridge_port *br_port_get_rcu(const struct net_device *dev)
 {
-	struct net_bridge_port *port = rcu_dereference(dev->rx_handler_data);
+	struct net_bridge_port *port =
+			rcu_dereference_rtnl(dev->rx_handler_data);
+
 	return br_port_exists(dev) ? port : NULL;
 }
 
net/core/rtnetlink.c +81 -4

109 lines not on this page · raw message

#9 lore raw

PATCH v2 3/3 ixgbe: add setlink, getlink support to ixgbe and ixgbevf

In reply to John Fastabend

This adds support for the net device ops to manage the embedded hardware bridge on ixgbe devices. With this patch the bridge mode can be toggled between VEB and VEPA to support stacking macvlan devices or using the embedded switch without any SW component in 802.1Qbg/br environments.

Additionally, this adds source address pruning to the ixgbevf driver to prune any frames sent back from a reflective relay on the switch. This is required because the existing hardware does not support this. Without it frames get pushed into the stack with its own src mac which is invalid per 802.1Qbg VEPA definition.

---

 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c     |   59 ++++++++++++++++++++-
 drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c    |    3 +
 drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c |   10 ++++
 include/linux/rtnetlink.h                         |    3 +
 net/core/rtnetlink.c                              |   50 ++++++++++++++++++
 5 files changed, 122 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +56 -3
index 88d636a..9a88e01 100644
@@ -44,6 +44,7 @@
 #include <linux/ethtool.h>
 #include <linux/if.h>
 #include <linux/if_vlan.h>
+#include <linux/if_bridge.h>
 #include <linux/prefetch.h>
 #include <scsi/fc/fc_fcoe.h>
 
@@ -3224,7 +3225,6 @@ static void ixgbe_configure_virtualization(struct ixgbe_adapter *adapter)
 	IXGBE_WRITE_REG(hw, IXGBE_VFRE(reg_offset ^ 1), reg_offset - 1);
 	IXGBE_WRITE_REG(hw, IXGBE_VFTE(reg_offset), (~0) << vf_shift);
 	IXGBE_WRITE_REG(hw, IXGBE_VFTE(reg_offset ^ 1), reg_offset - 1);
-	IXGBE_WRITE_REG(hw, IXGBE_PFDTXGSWC, IXGBE_PFDTXGSWC_VT_LBEN);
 
 	/* Map PF MAC address in RAR Entry 0 to first pool following VFs */
 	hw->mac.ops.set_vmdq(hw, 0, VMDQ_P(0));
@@ -3247,8 +3247,6 @@ static void ixgbe_configure_virtualization(struct ixgbe_adapter *adapter)
 
 	IXGBE_WRITE_REG(hw, IXGBE_GCR_EXT, gcr_ext);
 
-	/* enable Tx loopback for VF/PF communication */
-	IXGBE_WRITE_REG(hw, IXGBE_PFDTXGSWC, IXGBE_PFDTXGSWC_VT_LBEN);
 
 	/* Enable MAC Anti-Spoofing */
 	hw->mac.ops.set_mac_anti_spoofing(hw, (adapter->num_vfs != 0),
@@ -7025,6 +7023,59 @@ static int ixgbe_ndo_fdb_dump(struct sk_buff *skb,
 	return idx;
 }
 
+static int ixgbe_ndo_bridge_setlink(struct net_device *dev,
+				    struct nlmsghdr *nlh)
+{
+	struct ixgbe_adapter *adapter = netdev_priv(dev);
+	struct nlattr *attr, *br_spec;
+	int rem;
+
+	if (!(adapter->flags & IXGBE_FLAG_SRIOV_ENABLED))
+		return -EOPNOTSUPP;
+
+	br_spec = nlmsg_find_attr(nlh, sizeof(struct ifinfomsg), IFLA_AF_SPEC);
+
+	nla_for_each_nested(attr, br_spec, rem) {
+		__u16 mode;
+		u32 reg = 0;
+
+		if (nla_type(attr) != IFLA_BRIDGE_MODE)
+			continue;
+
+		mode = nla_get_u16(attr);
+		if (mode == BRIDGE_MODE_VEPA)
+			reg = 0;
+		else if (mode == BRIDGE_MODE_VEB)
+			reg = IXGBE_PFDTXGSWC_VT_LBEN;
+		else
+			return -EINVAL;
+
+		IXGBE_WRITE_REG(&adapter->hw, IXGBE_PFDTXGSWC, reg);
+
+		e_info(drv, "enabling bridge mode: %s\n",
+			mode == BRIDGE_MODE_VEPA ? "VEPA" : "VEB");
+	}
+
+	return 0;
+}
+
+static int ixgbe_ndo_bridge_getlink(struct sk_buff *skb, u32 pid, u32 seq,
+				    struct net_device *dev)
+{
+	struct ixgbe_adapter *adapter = netdev_priv(dev);
+	u16 mode;
+
+	if (!(adapter->flags & IXGBE_FLAG_SRIOV_ENABLED))
+		return 0;
+
+	if (IXGBE_READ_REG(&adapter->hw, IXGBE_PFDTXGSWC) & 1)
+		mode = BRIDGE_MODE_VEB;
+	else
+		mode = BRIDGE_MODE_VEPA;
+
+	return ndo_dflt_bridge_getlink(skb, pid, seq, dev, mode);
+}
+
 static const struct net_device_ops ixgbe_netdev_ops = {
 	.ndo_open		= ixgbe_open,
 	.ndo_stop		= ixgbe_close,
@@ -7064,6 +7115,8 @@ static const struct net_device_ops ixgbe_netdev_ops = {
 	.ndo_fdb_add		= ixgbe_ndo_fdb_add,
 	.ndo_fdb_del		= ixgbe_ndo_fdb_del,
 	.ndo_fdb_dump		= ixgbe_ndo_fdb_dump,
+	.ndo_bridge_setlink	= ixgbe_ndo_bridge_setlink,
+	.ndo_bridge_getlink	= ixgbe_ndo_bridge_getlink,
 };
 
 /**
drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c +3 -0
index 96876b7..7e3ac28 100644
@@ -117,6 +117,9 @@ void ixgbe_enable_sriov(struct ixgbe_adapter *adapter,
 		}
 	}
 
+	/* Initialize default switching mode VEB */
+	IXGBE_WRITE_REG(hw, IXGBE_PFDTXGSWC, IXGBE_PFDTXGSWC_VT_LBEN);
+
 	/* If call to enable VFs succeeded then allocate memory
 	 * for per VF control structures.
 	 */
drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c +10 -0
index 07d7eab..ac6a76d 100644
@@ -478,6 +478,16 @@ static bool ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
 		}
 		skb->protocol = eth_type_trans(skb, rx_ring->netdev);
 
+		/* Workaround hardware that can't do proper VEPA multicast
+		 * source pruning.
+		 */
+		if ((skb->pkt_type & (PACKET_BROADCAST | PACKET_MULTICAST)) &&
+		    !(compare_ether_addr(adapter->netdev->dev_addr,
+					eth_hdr(skb)->h_source))) {
+			dev_kfree_skb_irq(skb);
+			goto next_desc;
+		}
+
 		ixgbevf_receive_skb(q_vector, skb, staterr, rx_desc);
 
 next_desc:
include/linux/rtnetlink.h +3 -0
index 7002bbf..489dd7bb 100644
@@ -69,4 +69,7 @@ extern int ndo_dflt_fdb_dump(struct sk_buff *skb,
 			     struct netlink_callback *cb,
 			     struct net_device *dev,
 			     int idx);
+
+extern int ndo_dflt_bridge_getlink(struct sk_buff *skb, u32 pid, u32 seq,
+				   struct net_device *dev, u16 mode);
 #endif	/* __LINUX_RTNETLINK_H */
net/core/rtnetlink.c +50 -0

56 lines not on this page · raw message

#10 lore raw
In reply to John Fastabend
Quoted 52 lines

-----Original Message-----
From: [email protected] [mailto:netdev- [email protected]] On Behalf Of John Fastabend
Sent: Wednesday, October 24, 2012 8:13 PM
To: [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Eilon Greenstein
Subject: [net-next PATCH v2 0/3] extend set/get netlink for embedded

This extends the PF_BRIDGE setlink and getlink so that they can be used generically outside of the Linux bridge module. Doing this allows embedded devices to use the same netlink interface that the software bridge is currently using.

In this patchset I opted to create two new ndo ops ndo_bridge_setlink and ndo_bridge_getlink. These ops pass the nlmsghdr to the device for handling. The netlink message is extended to support nested IFLA_AF_SPEC attributes. A IFLA_BRIDGE_FLAGS attribute is used to determine the target of the message either a master netdev is the target or the target is "self" indicating the target is the netdev.
In this way we can send netlink msg to an embedded device or the linux sw bridge or both. If the set completes sucessfully the flag is cleared in this way we can learn what failed in the both case.
This scheme is similar to how FDB updates are handled. If no flag attribute is present the message is sent to the master device to support the existing messages.

An initial IFLA_BRIDGE_MODE attribute is added to indicate if the bridge is in a VEPA mode or VEB mode. This is most useful for SR-IOV device and can be used to indicate support for VF to VF or VF to PF traffic. Without this we have no way of knowing this other then trial and error because not all hardware supports this.

In the future additional attributes can be added to handle other attributes. We could for example move some of the linux bridge sysfs attributes here if we want a netlink interface for them.
This should also allow DSA switches to use the bridging tools. See RFC patch from Lennert

http://patchwork.ozlabs.org/patch/16578/

I could have simply added the VEPA attribute handling to the rtnl_setlink() routine but it seemed like a good feature to have all the bridging events and configuration on the same type. Also this should allow more powerful offloaded switches like the hardware supported via DSA access to the bridge control interface.

Quoted 5 lines

Also I think this interface should allow Vlad to add his port based VLAN interface here as well.

Any feedback/criticism appreciated thanks!

This series looks fine from bnx2x point of view.
The dynamic change from VEB to VEPA will require a firmware change, so we might arrive a little late for the party.
Ariel

#2 lore raw

Re: [net-next PATCH v2 1/3] net: create generic bridge ops

In reply to John Fastabend
Quoted 11 lines · John Fastabend

The PF_BRIDGE:RTM_{GET|SET}LINK nlmsg family and type are currently embedded in the ./net/bridge module. This prohibits them from being used by other bridging devices. One example of this being hardware that has embedded bridging components.

In order to use these nlmsg types more generically this patch adds two net_device_ops hooks. One to set link bridge attributes and another to dump the current bride attributes.

ndo_bridge_setlink()
ndo_bridge_getlink()

[...]

Do you have any userland code for this?

[...]

Quoted 11 lines
net/core/rtnetlink.c +5 -0
@@ -2252,6 +2252,83 @@ static int rtnl_fdb_dump(struct sk_buff *skb, struct netlink_callback *cb)
 	return skb->len;
 }
 
+static int rtnl_bridge_getlink(struct sk_buff *skb, struct netlink_callback *cb)
+{
+	struct net *net = sock_net(skb->sk);
+	struct net_device *dev;
+	int idx = 0;

net/core/rtnetlink.c static int rtnl_fdb_dump(struct sk_buff *skb, struct netlink_callback *cb) · line 2269

Quoted 10 lines
net/core/rtnetlink.c +10 -0
+	u32 portid = NETLINK_CB(cb->skb).portid;
+	u32 seq = cb->nlh->nlmsg_seq;
+
+	rcu_read_lock();
+	for_each_netdev_rcu(net, dev) {
+		const struct net_device_ops *ops = dev->netdev_ops;
+		struct net_device *master = dev->master;
+
+		if (idx < cb->args[0])
+			continue;

[...]

This is wrong, as idx will not be incremented and continued iteration won't work at all. Since this has already been applied, I'll send a patch for this.

Ben.

Signature · 3 lines
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
#5 lore raw

Re: [net-next PATCH v2 2/3] net: set and query VEB/VEPA bridge mode via PF_BRIDGE

In reply to John Fastabend
Quoted 18 lines · John Fastabend

Hardware switches may support enabling and disabling the loopback switch which puts the device in a VEPA mode defined in the IEEE 802.1Qbg specification. In this mode frames are not switched in the hardware but sent directly to the switch. SR-IOV capable NICs will likely support this mode I am aware of at least two such devices. Also I am told (but don't have any of this hardware available) that there are devices that only support VEPA modes. In these cases it is important at a minimum to be able to query these attributes.

This patch adds an additional IFLA_BRIDGE_MODE attribute that can be set and dumped via the PF_BRIDGE:{SET|GET}LINK operations. Also anticipating bridge attributes that may be common for both embedded bridges and software bridges this adds a flags attribute IFLA_BRIDGE_FLAGS currently used to determine if the command or event is being generated to/from an embedded bridge or software bridge. Finally, the event generation is pulled out of the bridge module and into rtnetlink proper.

[...]

Quoted 2 lines
net/core/rtnetlink.c +0 -0

[...]

Quoted 7 lines
net/core/rtnetlink.c +7 -0
+static int rtnl_bridge_notify(struct net_device *dev, u16 flags)
+{
+	struct net *net = dev_net(dev);
+	struct net_device *master = dev->master;
+	struct sk_buff *skb;
+	int err = -EOPNOTSUPP;
+

net/core/rtnetlink.c static int rtnl_bridge_getlink(struct sk_buff *skb, struct netlink_callback *cb) · line 2329

Quoted 10 lines
net/core/rtnetlink.c +10 -0
+	skb = nlmsg_new(bridge_nlmsg_size(), GFP_ATOMIC);
+	if (!skb) {
+		err = -ENOMEM;
+		goto errout;
+	}
+
+	if (!flags && master && master->netdev_ops->ndo_bridge_getlink)
+		err = master->netdev_ops->ndo_bridge_getlink(skb, 0, 0, dev);
+	else if (dev->netdev_ops->ndo_bridge_getlink)
+		err = dev->netdev_ops->ndo_bridge_getlink(skb, 0, 0, dev);

This doesn't make sense. If flags == BRIDGE_FLAGS_MASTER then we only send the 'self' information. If flags == BRIDGE_FLAGS_MASTER | BRIDGE_FLAGS_SELF then we only send the master information.

[...]

Quoted 45 lines

static int rtnl_bridge_setlink(struct sk_buff *skb, struct nlmsghdr *nlh,

net/core/rtnetlink.c +23 -4
 			       void *arg)
 {
 	struct net *net = sock_net(skb->sk);
 	struct ifinfomsg *ifm;
 	struct net_device *dev;
-	int err = -EINVAL;
+	struct nlattr *br_spec, *attr = NULL;
+	int rem, err = -EOPNOTSUPP;
+	u16 flags = 0;
 
 	if (nlmsg_len(nlh) < sizeof(*ifm))
 		return -EINVAL;
@@ -2316,15 +2363,45 @@ static int rtnl_bridge_setlink(struct sk_buff *skb, struct nlmsghdr *nlh,
 		return -ENODEV;
 	}
 
-	if (dev->master && dev->master->netdev_ops->ndo_bridge_setlink) {
+	br_spec = nlmsg_find_attr(nlh, sizeof(struct ifinfomsg), IFLA_AF_SPEC);
+	if (br_spec) {
+		nla_for_each_nested(attr, br_spec, rem) {
+			if (nla_type(attr) == IFLA_BRIDGE_FLAGS) {
+				flags = nla_get_u16(attr);
+				break;
+			}
+		}
+	}
+
+	if (!flags || (flags & BRIDGE_FLAGS_MASTER)) {
+		if (!dev->master ||
+		    !dev->master->netdev_ops->ndo_bridge_setlink) {
+			err = -EOPNOTSUPP;
+			goto out;
+		}
+
 		err = dev->master->netdev_ops->ndo_bridge_setlink(dev, nlh);
 		if (err)
 			goto out;
+
+		flags &= ~BRIDGE_FLAGS_MASTER;
 	}
 
-	if (dev->netdev_ops->ndo_bridge_setlink)
-		err = dev->netdev_ops->ndo_bridge_setlink(dev, nlh);
+	if ((flags & BRIDGE_FLAGS_SELF)) {

net/core/rtnetlink.c static int rtnl_bridge_setlink(struct sk_buff *skb, struct nlmsghdr *nlh, · line 2400

Quoted 10 lines
net/core/rtnetlink.c +9 -0
+		if (!dev->netdev_ops->ndo_bridge_setlink)
+			err = -EOPNOTSUPP;
+		else
+			err = dev->netdev_ops->ndo_bridge_setlink(dev, nlh);
+
+		if (!err)
+			flags &= ~BRIDGE_FLAGS_SELF;
+	}
 
+	if (attr && nla_type(attr) == IFLA_BRIDGE_FLAGS)

This condition is wrong; attr will *not* be NULL if the nla_for_each_nested() loop terminates without finding an IFLA_BRIDGE_FLAGS attribute.

net/core/rtnetlink.c static int rtnl_bridge_setlink(struct sk_buff *skb, struct nlmsghdr *nlh, · line 2404

Quoted 4 lines
net/core/rtnetlink.c +4 -0
+		memcpy(nla_data(attr), &flags, sizeof(flags));
+	/* Generate event to notify upper layer of bridge change */
+	if (!err)
+		err = rtnl_bridge_notify(dev, flags);

flags has been modified above; surely we want to use the original value here?

Ben.

Quoted 4 lines
net/core/rtnetlink.c +0 -0
 out:
 	return err;
 }
Signature · 3 lines
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
#6 lore raw

Re: [net-next PATCH v2 2/3] net: set and query VEB/VEPA bridge mode via PF_BRIDGE

Quoted 36 lines · Ben Hutchings
Quoted 18 lines · John Fastabend

Hardware switches may support enabling and disabling the loopback switch which puts the device in a VEPA mode defined in the IEEE 802.1Qbg specification. In this mode frames are not switched in the hardware but sent directly to the switch. SR-IOV capable NICs will likely support this mode I am aware of at least two such devices. Also I am told (but don't have any of this hardware available) that there are devices that only support VEPA modes. In these cases it is important at a minimum to be able to query these attributes.

This patch adds an additional IFLA_BRIDGE_MODE attribute that can be set and dumped via the PF_BRIDGE:{SET|GET}LINK operations. Also anticipating bridge attributes that may be common for both embedded bridges and software bridges this adds a flags attribute IFLA_BRIDGE_FLAGS currently used to determine if the command or event is being generated to/from an embedded bridge or software bridge. Finally, the event generation is pulled out of the bridge module and into rtnetlink proper.

[...]

Quoted 2 lines
net/core/rtnetlink.c +0 -0

[...]

Quoted 13 lines
net/core/rtnetlink.c +13 -0
+static int rtnl_bridge_notify(struct net_device *dev, u16 flags)
+{
+	struct net *net = dev_net(dev);
+	struct net_device *master = dev->master;
+	struct sk_buff *skb;
+	int err = -EOPNOTSUPP;
+
+	skb = nlmsg_new(bridge_nlmsg_size(), GFP_ATOMIC);
+	if (!skb) {
+		err = -ENOMEM;
+		goto errout;
+	}
+
Quoted 9 lines

net/core/rtnetlink.c static int rtnl_bridge_getlink(struct sk_buff *skb, struct netlink_callback *cb) · line 2329

Quoted 4 lines
net/core/rtnetlink.c +4 -0
+	if (!flags && master && master->netdev_ops->ndo_bridge_getlink)
+		err = master->netdev_ops->ndo_bridge_getlink(skb, 0, 0, dev);
+	else if (dev->netdev_ops->ndo_bridge_getlink)
+		err = dev->netdev_ops->ndo_bridge_getlink(skb, 0, 0, dev);

This doesn't make sense. If flags == BRIDGE_FLAGS_MASTER then we only send the 'self' information. If flags == BRIDGE_FLAGS_MASTER | BRIDGE_FLAGS_SELF then we only send the master information.

agh. I screwed this up when I made this handle setting both master and self flags. I'll fix it. Thanks.

Quoted 55 lines

[...]

Quoted 54 lines
  static int rtnl_bridge_setlink(struct sk_buff *skb, struct nlmsghdr *nlh,
  			       void *arg)
  {
  	struct net *net = sock_net(skb->sk);
  	struct ifinfomsg *ifm;
  	struct net_device *dev;
net/core/rtnetlink.c +31 -4
-	int err = -EINVAL;
+	struct nlattr *br_spec, *attr = NULL;
+	int rem, err = -EOPNOTSUPP;
+	u16 flags = 0;
 
  	if (nlmsg_len(nlh) < sizeof(*ifm))
  		return -EINVAL;
@@ -2316,15 +2363,45 @@ static int rtnl_bridge_setlink(struct sk_buff *skb, struct nlmsghdr *nlh,
  		return -ENODEV;
  	}
 
-	if (dev->master && dev->master->netdev_ops->ndo_bridge_setlink) {
+	br_spec = nlmsg_find_attr(nlh, sizeof(struct ifinfomsg), IFLA_AF_SPEC);
+	if (br_spec) {
+		nla_for_each_nested(attr, br_spec, rem) {
+			if (nla_type(attr) == IFLA_BRIDGE_FLAGS) {
+				flags = nla_get_u16(attr);
+				break;
+			}
+		}
+	}
+
+	if (!flags || (flags & BRIDGE_FLAGS_MASTER)) {
+		if (!dev->master ||
+		    !dev->master->netdev_ops->ndo_bridge_setlink) {
+			err = -EOPNOTSUPP;
+			goto out;
+		}
+
  		err = dev->master->netdev_ops->ndo_bridge_setlink(dev, nlh);
  		if (err)
  			goto out;
+
+		flags &= ~BRIDGE_FLAGS_MASTER;
  	}
 
-	if (dev->netdev_ops->ndo_bridge_setlink)
-		err = dev->netdev_ops->ndo_bridge_setlink(dev, nlh);
+	if ((flags & BRIDGE_FLAGS_SELF)) {
+		if (!dev->netdev_ops->ndo_bridge_setlink)
+			err = -EOPNOTSUPP;
+		else
+			err = dev->netdev_ops->ndo_bridge_setlink(dev, nlh);
+
+		if (!err)
+			flags &= ~BRIDGE_FLAGS_SELF;
+	}
 
Quoted 5 lines

net/core/rtnetlink.c static int rtnl_bridge_setlink(struct sk_buff *skb, struct nlmsghdr *nlh, · line 2400

Quoted 1 line
net/core/rtnetlink.c +1 -0
+	if (attr && nla_type(attr) == IFLA_BRIDGE_FLAGS)

This condition is wrong; attr will *not* be NULL if the nla_for_each_nested() loop terminates without finding an IFLA_BRIDGE_FLAGS attribute.

It might be NULL if the nlmsg has no IFLA_AF_SPEC attr. In this case we still need to send the PROTINFO attribute to the master which could be the linux bridge.

Quoted 8 lines

net/core/rtnetlink.c static int rtnl_bridge_setlink(struct sk_buff *skb, struct nlmsghdr *nlh, · line 2404

Quoted 5 lines
net/core/rtnetlink.c +4 -0
+		memcpy(nla_data(attr), &flags, sizeof(flags));
+	/* Generate event to notify upper layer of bridge change */
+	if (!err)
+		err = rtnl_bridge_notify(dev, flags);

flags has been modified above; surely we want to use the original value here?

Yes. Thanks Ben. I'll have a patch shortly.

Quoted 8 lines

Ben.

Quoted 4 lines
net/core/rtnetlink.c +0 -0
  out:
  	return err;
  }
#7 lore raw

Re: [net-next PATCH v2 2/3] net: set and query VEB/VEPA bridge mode via PF_BRIDGE

Quoted 20 lines · John Fastabend
Quoted 19 lines · Ben Hutchings
Quoted 18 lines · John Fastabend

Hardware switches may support enabling and disabling the loopback switch which puts the device in a VEPA mode defined in the IEEE 802.1Qbg specification. In this mode frames are not switched in the hardware but sent directly to the switch. SR-IOV capable NICs will likely support this mode I am aware of at least two such devices. Also I am told (but don't have any of this hardware available) that there are devices that only support VEPA modes. In these cases it is important at a minimum to be able to query these attributes.

This patch adds an additional IFLA_BRIDGE_MODE attribute that can be set and dumped via the PF_BRIDGE:{SET|GET}LINK operations. Also anticipating bridge attributes that may be common for both embedded bridges and software bridges this adds a flags attribute IFLA_BRIDGE_FLAGS currently used to determine if the command or event is being generated to/from an embedded bridge or software bridge. Finally, the event generation is pulled out of the bridge module and into rtnetlink proper.

[...]

Quoted 9 lines
Quoted 5 lines

net/core/rtnetlink.c static int rtnl_bridge_setlink(struct sk_buff *skb, struct nlmsghdr *nlh, · line 2400

Quoted 1 line
net/core/rtnetlink.c +1 -0
+	if (attr && nla_type(attr) == IFLA_BRIDGE_FLAGS)

This condition is wrong; attr will *not* be NULL if the nla_for_each_nested() loop terminates without finding an IFLA_BRIDGE_FLAGS attribute.

It might be NULL if the nlmsg has no IFLA_AF_SPEC attr. In this case we still need to send the PROTINFO attribute to the master which could be the linux bridge.

[...]

I think nla_for_each_nested() can leave attr non-null but also not valid for use with nla_type(). And that's a problem. I think it would be better to use an explicit flag for whether we found that attribute, rather than trying to re-test here.

Ben.

Signature · 3 lines
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
#8 lore raw

Re: [net-next PATCH v2 2/3] net: set and query VEB/VEPA bridge mode via PF_BRIDGE

Quoted 31 lines · Ben Hutchings
Quoted 20 lines · John Fastabend
Quoted 19 lines · Ben Hutchings
Quoted 18 lines · John Fastabend

Hardware switches may support enabling and disabling the loopback switch which puts the device in a VEPA mode defined in the IEEE 802.1Qbg specification. In this mode frames are not switched in the hardware but sent directly to the switch. SR-IOV capable NICs will likely support this mode I am aware of at least two such devices. Also I am told (but don't have any of this hardware available) that there are devices that only support VEPA modes. In these cases it is important at a minimum to be able to query these attributes.

This patch adds an additional IFLA_BRIDGE_MODE attribute that can be set and dumped via the PF_BRIDGE:{SET|GET}LINK operations. Also anticipating bridge attributes that may be common for both embedded bridges and software bridges this adds a flags attribute IFLA_BRIDGE_FLAGS currently used to determine if the command or event is being generated to/from an embedded bridge or software bridge. Finally, the event generation is pulled out of the bridge module and into rtnetlink proper.

[...]

Quoted 9 lines
Quoted 5 lines

net/core/rtnetlink.c static int rtnl_bridge_setlink(struct sk_buff *skb, struct nlmsghdr *nlh, · line 2400

Quoted 1 line
net/core/rtnetlink.c +1 -0
+	if (attr && nla_type(attr) == IFLA_BRIDGE_FLAGS)

This condition is wrong; attr will *not* be NULL if the nla_for_each_nested() loop terminates without finding an IFLA_BRIDGE_FLAGS attribute.

It might be NULL if the nlmsg has no IFLA_AF_SPEC attr. In this case we still need to send the PROTINFO attribute to the master which could be the linux bridge.

Quoted 9 lines

[...]

I think nla_for_each_nested() can leave attr non-null but also not valid for use with nla_type(). And that's a problem. I think it would be better to use an explicit flag for whether we found that attribute, rather than trying to re-test here.

Ben.

OK. I'll add an explicit flag for this. Thanks.

#3 lore raw

Re: [net-next PATCH v2 1/3] net: create generic bridge ops

In reply to Ben Hutchings
Quoted 10 lines · Ben Hutchings
Quoted 9 lines · John Fastabend

The PF_BRIDGE:RTM_{GET|SET}LINK nlmsg family and type are currently embedded in the ./net/bridge module. This prohibits them from being used by other bridging devices. One example of this being hardware that has embedded bridging components.

In order to use these nlmsg types more generically this patch adds two net_device_ops hooks. One to set link bridge attributes and another to dump the current bride attributes.

Quoted 5 lines
Quoted 2 lines

ndo_bridge_setlink()
ndo_bridge_getlink()

[...]

Do you have any userland code for this?

Just sent this to netdev. Or you can pull it here,

git://github.com/jrfastab/net-next-iproute2-jf.git

I'll test your idx fix now.

#11 lore raw
In reply to Ariel Elior
Quoted 6 lines

This series looks fine from bnx2x point of view.
The dynamic change from VEB to VEPA will require a firmware change, so we might arrive a little late for the party.
Ariel

In the meantime you could enable just the get routines so the upper layer could learn how your SR-IOV switching is being done or not done.

My basic expectation is hardware defaults to VEB but not sure if that is true in all cases.

.John