Skip to contentSkip to conversation

sctp: Add GSO support

PATCH Superseded Marcelo Ricardo Leitner 2 participants 8 messages Apr 29, 2016 – May 3, 2016

Subsystems (3)
Networking Network drivers SCTP protocol

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Marcelo Ricardo Leitner

author maintainer
#0 lore raw

cover sctp: Add GSO support

This patchset adds sctp GSO support.

Performance tests indicates that increases throughput by 10% if using bigger chunk sizes, specially if bigger than MTU. For small chunks, it doesn't help much if not using heavy firewall rules.

For small chunks it will probably be of more use once we get something like MSG_MORE as David Laight had suggested.

I believe I could address all comments from the RFC attempt.

Marcelo Ricardo Leitner (2):
skbuff: export skb_gro_receive
sctp: Add GSO support

 include/linux/netdev_features.h |   7 +-
 include/linux/netdevice.h       |   1 +
 include/linux/skbuff.h          |   7 +
 include/net/sctp/sctp.h         |   4 +
 include/net/sctp/structs.h      |   2 +
 net/core/skbuff.c               |  11 +-
 net/ipv4/af_inet.c              |   1 +
 net/sctp/Makefile               |   3 +-
 net/sctp/offload.c              |  98 +++++++++++
 net/sctp/output.c               | 348 +++++++++++++++++++++++++++-------------
 net/sctp/protocol.c             |   3 +
 net/sctp/socket.c               |   2 +
 12 files changed, 366 insertions(+), 121 deletions(-)
 create mode 100644 net/sctp/offload.c
Signature · 1 line
2.5.0
author maintainer
#1 lore raw

PATCH 1/2 skbuff: export skb_gro_receive

sctp GSO requires it and sctp can be compiled as a module, so we need to export this function.

Signed-off-by Marcelo Ricardo Leitner <[email protected]>
---
 net/core/skbuff.c | 1 +
 1 file changed, 1 insertion(+)
net/core/skbuff.c +1 -0
index 7a1d48983f81e68b42e8beb664db9aef00440f13..1b05114baeb5e3aa4f333dccd24a97aff86892ec 100644
@@ -3435,6 +3435,7 @@ done:
 	NAPI_GRO_CB(skb)->same_flow = 1;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(skb_gro_receive);
 
 void __init skb_init(void)
 {
Signature · 1 line
2.5.0
author maintainer
#2 lore raw

PATCH 2/2 sctp: Add GSO support

In reply to Marcelo Ricardo Leitner

SCTP has this pecualiarity that its packets cannot be just segmented to (P)MTU. Its chunks must be contained in IP segments, padding respected. So we can't just generate a big skb, set gso_size to the fragmentation point and deliver it to IP layer.

This patch takes a different approach. SCTP will now build a skb as it would be if it was received using GRO. That is, there will be a cover skb with protocol headers and children ones containing the actual segments, already segmented to a way that respects SCTP RFCs.

With that, we can tell skb_segment() to just split based on frag_list, trusting its sizes are already in accordance.

This way SCTP can benefit from GSO and instead of passing several packets through the stack, it can pass a single large packet.

Tested-by Xin Long <[email protected]>
Signed-off-by Marcelo Ricardo Leitner <[email protected]>
---
 include/linux/netdev_features.h |   7 +-
 include/linux/netdevice.h       |   1 +
 include/linux/skbuff.h          |   7 +
 include/net/sctp/sctp.h         |   4 +
 include/net/sctp/structs.h      |   2 +
 net/core/skbuff.c               |  10 +-
 net/ipv4/af_inet.c              |   1 +
 net/sctp/Makefile               |   3 +-
 net/sctp/offload.c              |  98 +++++++++++
 net/sctp/output.c               | 348 +++++++++++++++++++++++++++-------------
 net/sctp/protocol.c             |   3 +
 net/sctp/socket.c               |   2 +
 12 files changed, 365 insertions(+), 121 deletions(-)
 create mode 100644 net/sctp/offload.c
include/linux/netdev_features.h +5 -2
index bc87362667497fd845a2fcc5ad0eddbf031d1eaf..838aa14fec16cdc3814066351e4c533f64d0e340 100644
@@ -53,8 +53,9 @@ enum {
 					 *     headers in software.
 					 */
 	NETIF_F_GSO_TUNNEL_REMCSUM_BIT, /* ... TUNNEL with TSO & REMCSUM */
+	NETIF_F_GSO_SCTP_BIT,		/* ... SCTP fragmentation */
 	/**/NETIF_F_GSO_LAST =		/* last bit, see GSO_MASK */
-		NETIF_F_GSO_TUNNEL_REMCSUM_BIT,
+		NETIF_F_GSO_SCTP_BIT,
 
 	NETIF_F_FCOE_CRC_BIT,		/* FCoE CRC32 */
 	NETIF_F_SCTP_CRC_BIT,		/* SCTP checksum offload */

@@ -128,6 +129,7 @@ enum {
 #define NETIF_F_TSO_MANGLEID	__NETIF_F(TSO_MANGLEID)
 #define NETIF_F_GSO_PARTIAL	 __NETIF_F(GSO_PARTIAL)
 #define NETIF_F_GSO_TUNNEL_REMCSUM __NETIF_F(GSO_TUNNEL_REMCSUM)
+#define NETIF_F_GSO_SCTP	__NETIF_F(GSO_SCTP)
 #define NETIF_F_HW_VLAN_STAG_FILTER __NETIF_F(HW_VLAN_STAG_FILTER)
 #define NETIF_F_HW_VLAN_STAG_RX	__NETIF_F(HW_VLAN_STAG_RX)
 #define NETIF_F_HW_VLAN_STAG_TX	__NETIF_F(HW_VLAN_STAG_TX)

@@ -166,7 +168,8 @@ enum {
 				 NETIF_F_FSO)
 
 /* List of features with software fallbacks. */
-#define NETIF_F_GSO_SOFTWARE	(NETIF_F_ALL_TSO | NETIF_F_UFO)
+#define NETIF_F_GSO_SOFTWARE	(NETIF_F_ALL_TSO | NETIF_F_UFO | \
+				 NETIF_F_GSO_SCTP)
 
 /*
  * If one device supports one of these features, then enable them
include/linux/netdevice.h +1 -0
index 934ca866562d9ba9a9d2fb2e34f627660bd0c994..91f084d2d7e066c398cd98d3fd70eec8ec063d5d 100644
@@ -4010,6 +4010,7 @@ static inline bool net_gso_ok(netdev_features_t features, int gso_type)
 	BUILD_BUG_ON(SKB_GSO_UDP_TUNNEL_CSUM != (NETIF_F_GSO_UDP_TUNNEL_CSUM >> NETIF_F_GSO_SHIFT));
 	BUILD_BUG_ON(SKB_GSO_PARTIAL != (NETIF_F_GSO_PARTIAL >> NETIF_F_GSO_SHIFT));
 	BUILD_BUG_ON(SKB_GSO_TUNNEL_REMCSUM != (NETIF_F_GSO_TUNNEL_REMCSUM >> NETIF_F_GSO_SHIFT));
+	BUILD_BUG_ON(SKB_GSO_SCTP    != (NETIF_F_GSO_SCTP >> NETIF_F_GSO_SHIFT));
 
 	return (features & feature) = feature;
 }
include/linux/skbuff.h +7 -0
index c84a5a1078c5528bf6fc84573f63f3c6f470ce8f..df557f77732e0d74a896522e0c9f864f49d6438a 100644
@@ -301,6 +301,11 @@ struct sk_buff;
 #endif
 extern int sysctl_max_skb_frags;
 
+/* Set skb_shinfo(skb)->gso_size to this in case you want skb_segment to
+ * segment using its current segmentation instead.
+ */
+#define GSO_BY_FRAGS	0xFFFF
+
 typedef struct skb_frag_struct skb_frag_t;
 
 struct skb_frag_struct {

@@ -482,6 +487,8 @@ enum {
 	SKB_GSO_PARTIAL = 1 << 13,
 
 	SKB_GSO_TUNNEL_REMCSUM = 1 << 14,
+
+	SKB_GSO_SCTP = 1 << 15,
 };
 
 #if BITS_PER_LONG > 32
include/net/sctp/sctp.h +4 -0
index b392ac8382f2bf0be118f797a4444cc0eb4ddeb5..632e205ca54bfe85124753e09445251056e19aa7 100644
@@ -186,6 +186,10 @@ void sctp_assocs_proc_exit(struct net *net);
 int sctp_remaddr_proc_init(struct net *net);
 void sctp_remaddr_proc_exit(struct net *net);
 
+/*
+ * sctp/offload.c
+ */
+int sctp_offload_init(void);
 
 /*
  * Module global variables
include/net/sctp/structs.h +2 -0
index 558bae3cbe0d5107d52c8cb31b324cfd5479def0..692805bff54cdab723605e100a9bdbee8db6b563 100644
@@ -696,6 +696,8 @@ struct sctp_packet {
 	size_t overhead;
 	/* This is the total size of all chunks INCLUDING padding.  */
 	size_t size;
+	/* This is the maximum size this packet may have */
+	size_t max_size;
 
 	/* The packet is destined for this transport address.
 	 * The function we finally use to pass down to the next lower
net/core/skbuff.c +7 -3
index 1b05114baeb5e3aa4f333dccd24a97aff86892ec..537a20d2fe27c63cf452d3c81b4ccb7878a21619 100644
@@ -3113,9 +3113,13 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb,
 		int hsize;
 		int size;
 
-		len = head_skb->len - offset;
-		if (len > mss)
-			len = mss;
+		if (unlikely(mss = GSO_BY_FRAGS)) {
+			len = list_skb->len;
+		} else {
+			len = head_skb->len - offset;
+			if (len > mss)
+				len = mss;
+		}
 
 		hsize = skb_headlen(head_skb) - offset;
 		if (hsize < 0)
net/ipv4/af_inet.c +1 -0
index 2e6e65fc4d203b91a06075e02d2dd1ac8141f3db..0415e4be6962e4a6c590f92497ba62aa698f235c 100644
@@ -1220,6 +1220,7 @@ static struct sk_buff *inet_gso_segment(struct sk_buff *skb,
 		       SKB_GSO_TCP_FIXEDID |
 		       SKB_GSO_TUNNEL_REMCSUM |
 		       SKB_GSO_PARTIAL |
+		       SKB_GSO_SCTP |
 		       0)))
 		goto out;
net/sctp/Makefile +2 -1
index 0fca5824ad0e93c905e2cbd59ff2ff7e2077ca7c..6c4f7496cec612b52e1e69664a209b4d58763be5 100644
@@ -11,7 +11,8 @@ sctp-y := sm_statetable.o sm_statefuns.o sm_sideeffect.o \
 	  transport.o chunk.o sm_make_chunk.o ulpevent.o \
 	  inqueue.o outqueue.o ulpqueue.o \
 	  tsnmap.o bind_addr.o socket.o primitive.o \
-	  output.o input.o debug.o ssnmap.o auth.o
+	  output.o input.o debug.o ssnmap.o auth.o \
+	  offload.o
 
 sctp_probe-y := probe.o
net/sctp/offload.c new file +98 -0

98 lines not on this page · raw message

net/sctp/output.c +232 -115

467 lines not on this page · raw message

net/sctp/protocol.c +3 -0
index d3d50daa248b06d7a4306d903b2dad89e9d2acbd..40022ee885d7e8d9fbce3c7d9df43f57f0bcfa0e 100644
@@ -1516,6 +1516,9 @@ static __init int sctp_init(void)
 	if (status)
 		goto err_v6_add_protocol;
 
+	if (sctp_offload_init() < 0)
+		pr_crit("%s: Cannot add SCTP protocol offload\n", __func__);
+
 out:
 	return status;
 err_v6_add_protocol:
net/sctp/socket.c +2 -0
index 777d0324594a33a407e9ec157a7634334b1292e2..c53f08eb61b3e0516685a94093b638979521dcb9 100644
@@ -4003,6 +4003,8 @@ static int sctp_init_sock(struct sock *sk)
 		return -ESOCKTNOSUPPORT;
 	}
 
+	sk->sk_gso_type = SKB_GSO_SCTP;
+
 	/* Initialize default send parameters. These parameters can be
 	 * modified with the SCTP_DEFAULT_SEND_PARAM socket option.
 	 */
Signature · 1 line
2.5.0
author maintainer
#3 lore raw
In reply to Marcelo Ricardo Leitner

Cc'ing Xin. Sorry Xin.

Quoted 31 lines · Marcelo Ricardo Leitner

This patchset adds sctp GSO support.

Performance tests indicates that increases throughput by 10% if using bigger chunk sizes, specially if bigger than MTU. For small chunks, it doesn't help much if not using heavy firewall rules.

For small chunks it will probably be of more use once we get something like MSG_MORE as David Laight had suggested.

I believe I could address all comments from the RFC attempt.

Marcelo Ricardo Leitner (2):
skbuff: export skb_gro_receive
sctp: Add GSO support

 include/linux/netdev_features.h |   7 +-
 include/linux/netdevice.h       |   1 +
 include/linux/skbuff.h          |   7 +
 include/net/sctp/sctp.h         |   4 +
 include/net/sctp/structs.h      |   2 +
 net/core/skbuff.c               |  11 +-
 net/ipv4/af_inet.c              |   1 +
 net/sctp/Makefile               |   3 +-
 net/sctp/offload.c              |  98 +++++++++++
 net/sctp/output.c               | 348 +++++++++++++++++++++++++++-------------
 net/sctp/protocol.c             |   3 +
 net/sctp/socket.c               |   2 +
 12 files changed, 366 insertions(+), 121 deletions(-)
 create mode 100644 net/sctp/offload.c
maintainer
#4 lore raw
In reply to Marcelo Ricardo Leitner

From: Marcelo Ricardo Leitner <[email protected]>
Date: Fri, 29 Apr 2016 18:33:31 -0300

Quoted 10 lines

This patchset adds sctp GSO support.

Performance tests indicates that increases throughput by 10% if using bigger chunk sizes, specially if bigger than MTU. For small chunks, it doesn't help much if not using heavy firewall rules.

For small chunks it will probably be of more use once we get something like MSG_MORE as David Laight had suggested.

I believe I could address all comments from the RFC attempt.

Are these packets idempotent?

Ie. if we GRO a bunch of SCTP frames on receive and that GRO frame is forwarded rather than received locally, is the same exact packet stream emitted on transmit?

author maintainer
#5 lore raw
Quoted 12 lines · David Miller

From: Marcelo Ricardo Leitner <[email protected]>
Date: Fri, 29 Apr 2016 18:33:31 -0300

Quoted 9 lines

This patchset adds sctp GSO support.

Performance tests indicates that increases throughput by 10% if using bigger chunk sizes, specially if bigger than MTU. For small chunks, it doesn't help much if not using heavy firewall rules.

For small chunks it will probably be of more use once we get something like MSG_MORE as David Laight had suggested.

Quoted 7 lines
Quoted 1 line

I believe I could address all comments from the RFC attempt.

Are these packets idempotent?

Ie. if we GRO a bunch of SCTP frames on receive and that GRO frame is forwarded rather than received locally, is the same exact packet stream emitted on transmit?

Forward path is not going to happen because we can't do GRO for SCTP, unfortunatelly. We would have to somehow maintain frame boundaries (as I did for GSO here) (so that AUTH chunks have a delimited scope, for example) and that's not feasible with the current way we do GRO. Well, at least I couldn't see how.

So this is just for pure tx path, no forwarding involved.

Marcelo

maintainer
#6 lore raw

From: Marcelo Ricardo Leitner <[email protected]>
Date: Tue, 3 May 2016 08:49:18 -0300

Quoted 1 line

So this is just for pure tx path, no forwarding involved.

And if that GSO segment is looped back into the stack via the loopback interface, the packet classifier mirror action, or netfilter?

You cannot just GSO things and then not handle such packets properly on receive.

author maintainer
#7 lore raw
Quoted 11 lines · David Miller

From: Marcelo Ricardo Leitner <[email protected]>
Date: Tue, 3 May 2016 08:49:18 -0300

Quoted 1 line

So this is just for pure tx path, no forwarding involved.

And if that GSO segment is looped back into the stack via the loopback interface, the packet classifier mirror action, or netfilter?

You cannot just GSO things and then not handle such packets properly on receive.

Good point. It will fail if one uses AUTH due to the reason I mentioned for GRO. I have to split the GSO packet upon rx too and that should do it. Will post a v2 when I have it.

Thanks,
Marcelo