Skip to contentSkip to conversation

veth and GSO maximums

PATCH In review Stephen Hemminger 4 participants 8 messages Dec 7, 2017 – Dec 9, 2017

Subsystems (2)
Networking Network drivers
lore
Share & tools
lore

Stephen Hemminger

author
#0 lore raw

v3 cover veth and GSO maximums

This is the more general way to solving the issue of GSO limits not being set correctly for containers on Azure. If a GSO packet is sent to host that exceeds the limit (reported by NDIS), then the host is forced to do segmentation in software which has noticeable performance impact.

The core rtnetlink infrastructure already has the messages and infrastructure to allow changing gso limits. With an updated iproute2 the following already works:

  # ip li set dev dummy0 gso_max_size 30000

These patches are about making it easier with veth.

Stephen Hemminger (2):
rtnetlink: allow GSO maximums to be set on device creation
veth: set peer GSO values

 drivers/net/veth.c   |  3 +++
 net/core/rtnetlink.c | 34 ++++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+)
Signature · 1 line
2.11.0
author
#1 lore raw

PATCH v3 1/2 rtnetlink: allow GSO maximums to be set on device creation

From: Stephen Hemminger <[email protected]>

Netlink device already allows changing GSO sizes with ip set command. The part that is missing is allowing overriding GSO settings on device creation.

Signed-off-by Stephen Hemminger <[email protected]>

allow set gso on create

---
 net/core/rtnetlink.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
net/core/rtnetlink.c +34 -0
index a4faefd65006..412ebf0b09c6 100644
@@ -1637,6 +1637,8 @@ static const struct nla_policy ifla_policy[IFLA_MAX+1] = {
 	[IFLA_PROMISCUITY]	= { .type = NLA_U32 },
 	[IFLA_NUM_TX_QUEUES]	= { .type = NLA_U32 },
 	[IFLA_NUM_RX_QUEUES]	= { .type = NLA_U32 },
+	[IFLA_GSO_MAX_SEGS]	= { .type = NLA_U32 },
+	[IFLA_GSO_MAX_SIZE]	= { .type = NLA_U32 },
 	[IFLA_PHYS_PORT_ID]	= { .type = NLA_BINARY, .len = MAX_PHYS_ITEM_ID_LEN },
 	[IFLA_CARRIER_CHANGES]	= { .type = NLA_U32 },  /* ignored */
 	[IFLA_PHYS_SWITCH_ID]	= { .type = NLA_BINARY, .len = MAX_PHYS_ITEM_ID_LEN },

@@ -2287,6 +2289,34 @@ static int do_setlink(const struct sk_buff *skb,
 		}
 	}
 
+	if (tb[IFLA_GSO_MAX_SIZE]) {
+		u32 max_size = nla_get_u32(tb[IFLA_GSO_MAX_SIZE]);
+
+		if (max_size > GSO_MAX_SIZE) {
+			err = -EINVAL;
+			goto errout;
+		}
+
+		if (dev->gso_max_size ^ max_size) {
+			netif_set_gso_max_size(dev, max_size);
+			status |= DO_SETLINK_MODIFIED;
+		}
+	}
+
+	if (tb[IFLA_GSO_MAX_SEGS]) {
+		u32 max_segs = nla_get_u32(tb[IFLA_GSO_MAX_SEGS]);
+
+		if (max_segs > GSO_MAX_SEGS) {
+			err = -EINVAL;
+			goto errout;
+		}
+
+		if (dev->gso_max_segs ^ max_segs) {
+			dev->gso_max_segs = max_segs;
+			status |= DO_SETLINK_MODIFIED;
+		}
+	}
+
 	if (tb[IFLA_OPERSTATE])
 		set_operstate(dev, nla_get_u8(tb[IFLA_OPERSTATE]));
 

@@ -2651,6 +2681,10 @@ struct net_device *rtnl_create_link(struct net *net,
 		dev->link_mode = nla_get_u8(tb[IFLA_LINKMODE]);
 	if (tb[IFLA_GROUP])
 		dev_set_group(dev, nla_get_u32(tb[IFLA_GROUP]));
+	if (tb[IFLA_GSO_MAX_SIZE])
+		netif_set_gso_max_size(dev, nla_get_u32(tb[IFLA_GSO_MAX_SIZE]));
+	if (tb[IFLA_GSO_MAX_SEGS])
+		dev->gso_max_size = nla_get_u32(tb[IFLA_GSO_MAX_SEGS]);
 
 	return dev;
 }
Signature · 1 line
2.11.0
author
#2 lore raw

PATCH v3 2/2 veth: set peer GSO values

In reply to Stephen Hemminger

When new veth is created, and GSO values have been configured on one device, clone those values to the peer.

For example:

   # ip link add dev vm1 gso_max_size 65530 type veth peer name vm2

This should create vm1 <--> vm2 with both having GSO maximum size set to 65530.

Signed-off-by Stephen Hemminger <[email protected]>
---
 drivers/net/veth.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/net/veth.c +3 -0
index f5438d0978ca..a69ad39ee57e 100644
@@ -410,6 +410,9 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
 	if (ifmp && (dev->ifindex != 0))
 		peer->ifindex = ifmp->ifi_index;
 
+	peer->gso_max_size = dev->gso_max_size;
+	peer->gso_max_segs = dev->gso_max_segs;
+
 	err = register_netdevice(peer);
 	put_net(net);
 	net = NULL;
Signature · 1 line
2.11.0
#3 lore raw

Re: [PATCH net-next 2/2] veth: set peer GSO values

Quoted 6 lines · Stephen Hemminger
drivers/net/veth.c +0 -0
index f5438d0978ca..a69ad39ee57e 100644

drivers/net/veth.c static int veth_newlink(struct net *src_net, struct net_device *dev, · line 418

Quoted 9 lines
drivers/net/veth.c +3 -0
@@ -410,6 +410,9 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
 	if (ifmp && (dev->ifindex != 0))
 		peer->ifindex = ifmp->ifi_index;
 
+	peer->gso_max_size = dev->gso_max_size;
+	peer->gso_max_segs = dev->gso_max_segs;
+
 	err = register_netdevice(peer);
 	put_net(net);
 	net = NULL;

What if gso changes are made after device create? They are not propagated to the peer device like they are on link create.

#4 lore raw

Re: [PATCH net-next 2/2] veth: set peer GSO values

Quoted 9 lines · David Ahern
Quoted 8 lines · Stephen Hemminger
drivers/net/veth.c +0 -0
index f5438d0978ca..a69ad39ee57e 100644
@@ -410,6 +410,9 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
 	if (ifmp && (dev->ifindex != 0))
 		peer->ifindex = ifmp->ifi_index;
Quoted 10 lines

drivers/net/veth.c static int veth_newlink(struct net *src_net, struct net_device *dev, · line 418

Quoted 7 lines
drivers/net/veth.c +3 -0
+	peer->gso_max_size = dev->gso_max_size;
+	peer->gso_max_segs = dev->gso_max_segs;
+
 	err = register_netdevice(peer);
 	put_net(net);
 	net = NULL;

What if gso changes are made after device create? They are not propagated to the peer device like they are on link create.

This would be a nice addition after ongoing patches are merged, since veth usually lives in another netns. For docker, it requires a couple of extra commands to expose the peer's netns.

maintainer
#5 lore raw
In reply to Stephen Hemminger

From: Stephen Hemminger <[email protected]>
Date: Thu, 7 Dec 2017 15:40:18 -0800

Quoted 6 lines

This is the more general way to solving the issue of GSO limits not being set correctly for containers on Azure. If a GSO packet is sent to host that exceeds the limit (reported by NDIS), then the host is forced to do segmentation in software which has noticeable performance impact.

Quoted 6 lines

The core rtnetlink infrastructure already has the messages and infrastructure to allow changing gso limits. With an updated iproute2 the following already works:

  # ip li set dev dummy0 gso_max_size 30000

These patches are about making it easier with veth.

Ok, this is definitely a step forward.

Series applied, thanks Stephen.

#6 lore raw
Quoted 14 lines · David Miller

From: Stephen Hemminger <[email protected]>
Date: Thu, 7 Dec 2017 15:40:18 -0800

Quoted 11 lines

This is the more general way to solving the issue of GSO limits not being set correctly for containers on Azure. If a GSO packet is sent to host that exceeds the limit (reported by NDIS), then the host is forced to do segmentation in software which has noticeable performance impact.

The core rtnetlink infrastructure already has the messages and infrastructure to allow changing gso limits. With an updated iproute2 the following already works:

  # ip li set dev dummy0 gso_max_size 30000
Quoted 5 lines
Quoted 1 line

These patches are about making it easier with veth.

Ok, this is definitely a step forward.

Series applied, thanks Stephen.

Thanks.
Still not seeing the iproute2 patch though, either master or net-next.

author
#7 lore raw
Quoted 15 lines · On Fri, 8 Dec 2017 15:50:25 -0800 Solio Sarabia <[email protected]> wrote:
Quoted 14 lines · David Miller

From: Stephen Hemminger <[email protected]>
Date: Thu, 7 Dec 2017 15:40:18 -0800

Quoted 11 lines

This is the more general way to solving the issue of GSO limits not being set correctly for containers on Azure. If a GSO packet is sent to host that exceeds the limit (reported by NDIS), then the host is forced to do segmentation in software which has noticeable performance impact.

The core rtnetlink infrastructure already has the messages and infrastructure to allow changing gso limits. With an updated iproute2 the following already works:

  # ip li set dev dummy0 gso_max_size 30000
Quoted 8 lines
Quoted 5 lines
Quoted 1 line

These patches are about making it easier with veth.

Ok, this is definitely a step forward.

Series applied, thanks Stephen.

Thanks.
Still not seeing the iproute2 patch though, either master or net-next.

I held off on iproute2 until kernel change is accepted upstream.