net: enable GRO for IPIP and SIT
Subsystems (3)
cover net: enable GRO for IPIP and SIT
This patch sets populates the IPIP and SIT offload structures with gro_receive and gro_complete functions. This enables use of GRO for these. Also, fixed a problem in IPv6 where we were not properly initializing flush_id.
Peformance results are below. Note that these tests were done on bnx2x which doesn't provide RX checksum offload of IPIP or SIT (i.e. does not give CHEKCSUM_COMPLETE). Also, we don't get 4-tuple hash for RSS only 2-tuple in this case so all the packets between two hosts are winding up on the same queue. Net result is the interrupting CPU is the bottleneck in GRO (checksumming every packet there).
Testing:
netperf TCP_STREAM between two hosts using bnx2x.
* Before fix
IPIP
1 connection
6.53% CPU utilization
6544.71 Mbps
20 connections
13.79% CPU utilization
9284.54 MbpsSIT
1 connection
6.68% CPU utilization
5653.36 Mbps
20 connections
18.88% CPU utilization
9154.61 Mbps* After fix
IPIP
1 connection
5.73% CPU utilization
9279.53 Mbps
20 connections
7.14% CPU utilization
7279.35 Mbps
SIT
1 connection
2.95% CPU utilization
9143.36 Mbps
20 connections
7.09% CPU utilization
6255.3 Mbps
Tom Herbert (3):
ipv6: Clear flush_id to make GRO work
ipip: Add gro callbacks to ipip offload
sit: Add gro callbacks to sit_offload
net/ipv4/af_inet.c | 2 ++ net/ipv6/ip6_offload.c | 5 +++++ 2 files changed, 7 insertions(+)
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH 1/3 ipv6: Clear flush_id to make GRO work
In TCP gro we check flush_id which is derived from the IP identifier. In IPv4 gro path the flush_id is set with the expectation that every matched packet increments IP identifier. In IPv6, the flush_id is never set and thus is uinitialized. What's worse is that in IPv6 over IPv4 encapsulation, the IP identifier is taken from the outer header which is currently not incremented on every packet for Linux stack, so GRO in this case never matches packets (identifier is not increasing).
This patch clears flush_id for every time for a matched packet in IPv6 gro_receive. We need to do this each time to overwrite the setting that would be done in IPv4 gro_receive per the outer header in IPv6 over Ipv4 encapsulation.
--- net/ipv6/ip6_offload.c | 3 +++ 1 file changed, 3 insertions(+)
net/ipv6/ip6_offload.c +3 -0
@@ -261,6 +261,9 @@ static struct sk_buff **ipv6_gro_receive(struct sk_buff **head,/* flush if Traffic Class fields are different */ NAPI_GRO_CB (p)->flush |= !!(first_word &htonl (0x0FF00000 ));NAPI_GRO_CB (p)->flush |= flush; + +/* Clear flush_id, there's really no concept of ID in IPv6. */ +NAPI_GRO_CB (p)->flush_id =0 ; }NAPI_GRO_CB (skb)->flush |= flush;
Signature · 1 line
2.1.0.rc2.206.gedb03e5
Re: [PATCH net-next 1/3] ipv6: Clear flush_id to make GRO work
Quoted 24 lines · Tom Herbert
In TCP gro we check
flush_idwhich is derived from the IP identifier. In IPv4 gro path theflush_idis set with the expectation that every matched packet increments IP identifier. In IPv6, theflush_idis never set and thus is uinitialized. What's worse is that in IPv6 over IPv4 encapsulation, the IP identifier is taken from the outer header which is currently not incremented on every packet for Linux stack, so GRO in this case never matches packets (identifier is not increasing).This patch clears
flush_idfor every time for a matched packet in IPv6gro_receive. We need to do this each time to overwrite the setting that would be done in IPv4gro_receiveper the outer header in IPv6 over Ipv4 encapsulation.Signed-off-by Tom Herbert <[email protected]>--- net/ipv6/ip6_offload.c | 3 +++ 1 file changed, 3 insertions(+)net/ipv6/ip6_offload.c +0
-0
net/ipv6/ip6_offload.c static struct sk_buff **ipv6_gro_receive(struct sk_buff **head, · line 269
Quoted 9 lines
net/ipv6/ip6_offload.c +3
-0@@ -261,6 +261,9 @@ static struct sk_buff **ipv6_gro_receive(struct sk_buff **head,/* flush if Traffic Class fields are different */ NAPI_GRO_CB (p)->flush |= !!(first_word &htonl (0x0FF00000 ));NAPI_GRO_CB (p)->flush |= flush; + +/* Clear flush_id, there's really no concept of ID in IPv6. */ +NAPI_GRO_CB (p)->flush_id =0 ; }NAPI_GRO_CB (skb)->flush |= flush;
Yeah, I mentioned this problem months ago and apparently forgot to push the fix.
http://patchwork.ozlabs.org/patch/311212/
Re: [PATCH net-next 1/3] ipv6: Clear flush_id to make GRO work
Quoted 33 lines · Eric Dumazet
Quoted 32 lines · Tom Herbert
In TCP gro we check
flush_idwhich is derived from the IP identifier. In IPv4 gro path theflush_idis set with the expectation that every matched packet increments IP identifier. In IPv6, theflush_idis never set and thus is uinitialized. What's worse is that in IPv6 over IPv4 encapsulation, the IP identifier is taken from the outer header which is currently not incremented on every packet for Linux stack, so GRO in this case never matches packets (identifier is not increasing).This patch clears
flush_idfor every time for a matched packet in IPv6gro_receive. We need to do this each time to overwrite the setting that would be done in IPv4gro_receiveper the outer header in IPv6 over Ipv4 encapsulation.Signed-off-by Tom Herbert <[email protected]>--- net/ipv6/ip6_offload.c | 3 +++ 1 file changed, 3 insertions(+)net/ipv6/ip6_offload.c +3
-0@@ -261,6 +261,9 @@ static struct sk_buff **ipv6_gro_receive(struct sk_buff **head,/* flush if Traffic Class fields are different */ NAPI_GRO_CB (p)->flush |= !!(first_word &htonl (0x0FF00000 ));NAPI_GRO_CB (p)->flush |= flush; + +/* Clear flush_id, there's really no concept of ID in IPv6. */ +NAPI_GRO_CB (p)->flush_id =0 ; }
Quoted 5 lines
net/ipv6/ip6_offload.c static struct sk_buff **ipv6_gro_receive(struct sk_buff **head, · line 269
Quoted 1 line
net/ipv6/ip6_offload.c +0
-0NAPI_GRO_CB (skb)->flush |= flush;Yeah, I mentioned this problem months ago and apparently forgot to push the fix.
Unfortunately, probably not the last GRO issue. This whole thing is really rather fragile and probably easy to break undetected...
Quoted 5 lines
http://patchwork.ozlabs.org/patch/311212/
Signed-off-by Eric Dumazet <[email protected]>
PATCH 2/3 ipip: Add gro callbacks to ipip offload
In reply to Tom HerbertAdd inet_gro_receive and inet_gro_complete to ipip_offload to support GRO.
--- net/ipv4/af_inet.c | 2 ++ 1 file changed, 2 insertions(+)
net/ipv4/af_inet.c +2 -0
@@ -1670,6 +1670,8 @@ static const struct net_offload ipip_offload = { .callbacks = { .gso_send_check = inet_gso_send_check, .gso_segment = inet_gso_segment, + .gro_receive = inet_gro_receive, + .gro_complete = inet_gro_complete, }, };
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH 3/3 sit: Add gro callbacks to sit_offload
In reply to Tom HerbertAdd ipv6_gro_receive and ipv6_gro_complete to sit_offload to support GRO.
--- net/ipv6/ip6_offload.c | 2 ++ 1 file changed, 2 insertions(+)
net/ipv6/ip6_offload.c +2 -0
@@ -317,6 +317,8 @@ static const struct net_offload sit_offload = { .callbacks = { .gso_send_check = ipv6_gso_send_check, .gso_segment = ipv6_gso_segment, + .gro_receive = ipv6_gro_receive, + .gro_complete = ipv6_gro_complete, }, };
Signature · 1 line
2.1.0.rc2.206.gedb03e5
David Miller
maintainerFrom: Tom Herbert <[email protected]>
Date: Tue, 9 Sep 2014 11:23:13 -0700
Quoted 4 lines
This patch sets populates the IPIP and SIT offload structures with
gro_receiveandgro_completefunctions. This enables use of GRO for these. Also, fixed a problem in IPv6 where we were not properly initializingflush_id.
We're going to need to sort this out, because these functions are now static after Eric's patches to fix sparse warnings.
You're going to have to respin this with that patch you intended to add from the previous series that exported these routines.
Quoted 8 lines · David Miller
From: Tom Herbert <[email protected]>
Date: Tue, 9 Sep 2014 11:23:13 -0700Quoted 4 lines
This patch sets populates the IPIP and SIT offload structures with
gro_receiveandgro_completefunctions. This enables use of GRO for these. Also, fixed a problem in IPv6 where we were not properly initializingflush_id.
Quoted 5 lines
We're going to need to sort this out, because these functions are now static after Eric's patches to fix sparse warnings.
You're going to have to respin this with that patch you intended to add from the previous series that exported these routines.
I might be missing something, but I believe the functions referred in this patch series were already static.
David Miller
maintainerFrom: Tom Herbert <[email protected]>
Date: Tue, 9 Sep 2014 21:04:27 -0700
Quoted 9 lines · David Miller
Quoted 8 lines
From: Tom Herbert <[email protected]>
Date: Tue, 9 Sep 2014 11:23:13 -0700Quoted 4 lines
This patch sets populates the IPIP and SIT offload structures with
gro_receiveandgro_completefunctions. This enables use of GRO for these. Also, fixed a problem in IPv6 where we were not properly initializingflush_id.
Quoted 8 lines
Quoted 5 lines
We're going to need to sort this out, because these functions are now static after Eric's patches to fix sparse warnings.
You're going to have to respin this with that patch you intended to add from the previous series that exported these routines.
I might be missing something, but I believe the functions referred in this patch series were already static.
Oops my bad, let me rereview this...
Yep it's fine, applied, thanks!