gro: some minor optimization
Subsystems (2)
Paolo Abeni
maintainercover gro: some minor optimization
This series collects a few small optimization for the GRO engine. I measure a 10% performance improvements in micro-benchmarks around dev_gro_receive(), but deltas are within noise range in tput tests.
Still with big TCP coming every cycle saved from the GRO engine will count - I hope ;)
The only change from the RFC is in patch 2, as per Alexander feedback
Paolo Abeni (3):
net: gro: avoid re-computing truesize twice on recycle
net: gro: minor optimization for dev_gro_receive()
net: gro: register gso and gro offload on separate lists
include/linux/netdevice.h | 3 +- include/net/gro.h | 52 +++++++++--------- net/core/gro.c | 109 +++++++++++++++++++++----------------- 3 files changed, 91 insertions(+), 73 deletions(-)
Signature · 1 line
2.34.1
Paolo Abeni
maintainerPATCH 1/3 net: gro: avoid re-computing truesize twice on recycle
After commit 5e10da5385d2 ("skbuff: allow 'slow_gro' for skb carring sock reference") and commit af352460b465 ("net: fix GRO skb truesize update") the truesize of freed skb is properly updated by the GRO engine, we don't need anymore resetting it at recycle time.
--- net/core/gro.c | 1 - 1 file changed, 1 deletion(-)
net/core/gro.c +0 -1
@@ -634,7 +634,6 @@ static void napi_reuse_skb(struct napi_struct *napi, struct sk_buff *skb) skb->encapsulation =0 ;skb_shinfo (skb)->gso_type =0 ; - skb->truesize =SKB_TRUESIZE (skb_end_offset (skb));if (unlikely (skb->slow_gro)) {skb_orphan (skb);skb_ext_reset (skb);
Signature · 1 line
2.34.1
Paolo Abeni
maintainerPATCH 2/3 net: gro: minor optimization for dev_gro_receive()
In reply to Paolo AbeniWhile inspecting some perf report, I noticed that the compiler emits suboptimal code for the napi CB initialization, fetching and storing multiple times the memory for flags bitfield. This is with gcc 10.3.1, but I observed the same with older compiler versions.
We can help the compiler to do a nicer work clearing several fields at once using an u32 alias. The generated code is quite smaller, with the same number of conditional.
Before:
objdump -t net/core/gro.o | grep " F .text"0000000000000bb0 l F .text 0000000000000357 dev_gro_receive
After:0000000000000bb0 l F .text 000000000000033c dev_gro_receive
RFC -> v1:
- use __struct_group to delimt the zeroed area (Alexander)
--- include/net/gro.h | 52 +++++++++++++++++++++++++---------------------- net/core/gro.c | 18 +++++++--------- 2 files changed, 35 insertions(+), 35 deletions(-)
include/net/gro.h +28 -24
@@ -29,46 +29,50 @@ struct napi_gro_cb {/* Number of segments aggregated. */ u16 count; -/* Start offset for remote checksum offload */ -u16 gro_remcsum_start; +/* Used in ipv6_gro_receive() and foo-over-udp */ +u16 proto;/* jiffies when first packet was created/queued */ unsigned long age; -/* Used in ipv6_gro_receive() and foo-over-udp */ -u16 proto; +/* portion of the cb set to zero at every gro iteration */ + __struct_group(/* no tag */, zeroed, /* no attrs */, + +/* Start offset for remote checksum offload */ + u16 gro_remcsum_start; -/* This is non-zero if the packet may be of the same flow. */ - u8 same_flow:1; +/* This is non-zero if the packet may be of the same flow. */ + u8 same_flow:1; -/* Used in tunnel GRO receive */ - u8 encap_mark:1; +/* Used in tunnel GRO receive */ + u8 encap_mark:1; -/* GRO checksum is valid */ - u8 csum_valid:1; +/* GRO checksum is valid */ + u8 csum_valid:1; -/* Number of checksums via CHECKSUM_UNNECESSARY */ - u8 csum_cnt:3; +/* Number of checksums via CHECKSUM_UNNECESSARY */ + u8 csum_cnt:3; -/* Free the skb? */ - u8 free:2; +/* Free the skb? */ + u8 free:2;#define NAPI_GRO_FREE 1#define NAPI_GRO_FREE_STOLEN_HEAD 2 -/* Used in foo-over-udp, set in udp[46]_gro_receive */ - u8 is_ipv6:1; +/* Used in foo-over-udp, set in udp[46]_gro_receive */ + u8 is_ipv6:1; -/* Used in GRE, set in fou/gue_gro_receive */ - u8 is_fou:1; +/* Used in GRE, set in fou/gue_gro_receive */ + u8 is_fou:1; -/* Used to determine if flush_id can be ignored */ - u8 is_atomic:1; +/* Used to determine if flush_id can be ignored */ + u8 is_atomic:1; -/* Number of gro_receive callbacks this packet already went through */ - u8 recursion_counter:4; +/* Number of gro_receive callbacks this packet already went through */ + u8 recursion_counter:4; -/* GRO is done by frag_list pointer chaining. */ - u8 is_flist:1; +/* GRO is done by frag_list pointer chaining. */ + u8 is_flist:1; + );/* used to support CHECKSUM_COMPLETE for tunneling protocols */ __wsum csum;
net/core/gro.c +7 -11
@@ -435,6 +435,9 @@ static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head)napi_gro_complete (napi, oldest); } +#define zeroed_len (offsetof(struct napi_gro_cb, zeroed_end) - \ + offsetof(struct napi_gro_cb, zeroed_start)) +static enum gro_result dev_gro_receive (struct napi_struct *napi,struct sk_buff *skb) {u32 bucket =skb_get_hash_raw (skb) & (GRO_HASH_BUCKETS -1 );
@@ -459,29 +462,22 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buffskb_set_network_header (skb,skb_gro_offset (skb));skb_reset_mac_len (skb); -NAPI_GRO_CB (skb)->same_flow =0 ; + BUILD_BUG_ON(sizeof_field(struct napi_gro_cb, zeroed) != sizeof(u32)); +BUILD_BUG_ON (!IS_ALIGNED (offsetof(struct napi_gro_cb , zeroed), +sizeof (u32)));/* Avoid slow unaligned acc */ + *(u32 *)&NAPI_GRO_CB (skb)->zeroed =0 ;NAPI_GRO_CB (skb)->flush =skb_is_gso (skb) ||skb_has_frag_list (skb); -NAPI_GRO_CB (skb)->free =0 ; -NAPI_GRO_CB (skb)->encap_mark =0 ; -NAPI_GRO_CB (skb)->recursion_counter =0 ; -NAPI_GRO_CB (skb)->is_fou =0 ;NAPI_GRO_CB (skb)->is_atomic =1 ; -NAPI_GRO_CB (skb)->gro_remcsum_start =0 ;/* Setup for GRO checksum validation */ switch (skb->ip_summed) {case CHECKSUM_COMPLETE :NAPI_GRO_CB (skb)->csum = skb->csum;NAPI_GRO_CB (skb)->csum_valid =1 ; -NAPI_GRO_CB (skb)->csum_cnt =0 ;break ;case CHECKSUM_UNNECESSARY :NAPI_GRO_CB (skb)->csum_cnt = skb->csum_level +1 ; -NAPI_GRO_CB (skb)->csum_valid =0 ;break ; -default : -NAPI_GRO_CB (skb)->csum_cnt =0 ; -NAPI_GRO_CB (skb)->csum_valid =0 ; } pp = INDIRECT_CALL_INET(ptype->callbacks.gro_receive,
Signature · 1 line
2.34.1
Paolo Abeni
maintainerPATCH 3/3 net: gro: register gso and gro offload on separate lists
In reply to Paolo AbeniSo that we know each element in gro_list has valid gro callbacks (and the same for gso). This allows dropping a bunch of conditional in fastpath.
Before:
objdump -t net/core/gro.o | grep " F .text"0000000000000bb0 l F .text 000000000000033c dev_gro_receive
After:0000000000000bb0 l F .text 0000000000000325 dev_gro_receive
--- include/linux/netdevice.h | 3 +- net/core/gro.c | 90 +++++++++++++++++++++++---------------- 2 files changed, 56 insertions(+), 37 deletions(-)
include/linux/netdevice.h +2 -1
@@ -2564,7 +2564,8 @@ struct packet_offload {__be16 type;/* This is really htons(ether_type). */ u16 priority;struct offload_callbacks callbacks; -struct list_head list; +struct list_head gro_list; +struct list_head gso_list; };/* often modified stats are per-CPU, other are shared (netdev->stats) */
net/core/gro.c +54 -36
176 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH net-next 2/3] net: gro: minor optimization for dev_gro_receive()
In reply to Paolo AbeniFrom: Paolo Abeni <[email protected]>
Date: Thu, 3 Feb 2022 16:48:22 +0100
Quoted 14 lines
While inspecting some perf report, I noticed that the compiler emits suboptimal code for the napi CB initialization, fetching and storing multiple times the memory for flags bitfield. This is with gcc 10.3.1, but I observed the same with older compiler versions.
We can help the compiler to do a nicer work clearing several fields at once using an u32 alias. The generated code is quite smaller, with the same number of conditional.
Before:
objdump -tnet/core/gro.o| grep " F .text"0000000000000bb0l F .text 0000000000000357dev_gro_receive
Quoted 5 lines
After:
0000000000000bb0l F .text000000000000033cdev_gro_receiveRFC -> v1:
- use__struct_groupto delimt the zeroed area (Alexander)
"delimit"?
Quoted 17 lines
Signed-off-by Paolo Abeni <[email protected]>--- include/net/gro.h | 52 +++++++++++++++++++++++++---------------------- net/core/gro.c | 18 +++++++--------- 2 files changed, 35 insertions(+), 35 deletions(-)include/net/gro.h +0
-2@@ -29,46 +29,50 @@ struct napi_gro_cb {/* Number of segments aggregated. */ u16 count; -/* Start offset for remote checksum offload */ -u16 gro_remcsum_start;
include/net/gro.h struct napi_gro_cb { · line 39
Quoted 10 lines
include/net/gro.h +4
-2+/* Used in ipv6_gro_receive() and foo-over-udp */ +u16 proto;/* jiffies when first packet was created/queued */ unsigned long age; -/* Used in ipv6_gro_receive() and foo-over-udp */ -u16 proto; +/* portion of the cb set to zero at every gro iteration */ + __struct_group(/* no tag */, zeroed, /* no attrs */,
Oh, only after sending this in my reply I noticed that there's a shorthand for this
struct_group(name, ...)
means exactly
__struct_group(/* no tag */, name, /* no attrs */, ...)
Sorry for that. I got confused by that Kees used __struct_group() directly in one place although there was a possibility to use short struct_group() instead.
Since there should already be a v2 (see below), probably worth changing.
Quoted 64 lines
include/net/gro.h +24
-20+ +/* Start offset for remote checksum offload */ +u16 gro_remcsum_start; -/* This is non-zero if the packet may be of the same flow. */ - u8 same_flow:1; +/* This is non-zero if the packet may be of the same flow. */ + u8 same_flow:1; -/* Used in tunnel GRO receive */ - u8 encap_mark:1; +/* Used in tunnel GRO receive */ + u8 encap_mark:1; -/* GRO checksum is valid */ - u8 csum_valid:1; +/* GRO checksum is valid */ + u8 csum_valid:1; -/* Number of checksums via CHECKSUM_UNNECESSARY */ - u8 csum_cnt:3; +/* Number of checksums via CHECKSUM_UNNECESSARY */ + u8 csum_cnt:3; -/* Free the skb? */ - u8 free:2; +/* Free the skb? */ + u8 free:2;#define NAPI_GRO_FREE 1#define NAPI_GRO_FREE_STOLEN_HEAD 2 -/* Used in foo-over-udp, set in udp[46]_gro_receive */ - u8 is_ipv6:1; +/* Used in foo-over-udp, set in udp[46]_gro_receive */ + u8 is_ipv6:1; -/* Used in GRE, set in fou/gue_gro_receive */ - u8 is_fou:1; +/* Used in GRE, set in fou/gue_gro_receive */ + u8 is_fou:1; -/* Used to determine if flush_id can be ignored */ - u8 is_atomic:1; +/* Used to determine if flush_id can be ignored */ + u8 is_atomic:1; -/* Number of gro_receive callbacks this packet already went through */ - u8 recursion_counter:4; +/* Number of gro_receive callbacks this packet already went through */ + u8 recursion_counter:4; -/* GRO is done by frag_list pointer chaining. */ - u8 is_flist:1; +/* GRO is done by frag_list pointer chaining. */ + u8 is_flist:1; + );/* used to support CHECKSUM_COMPLETE for tunneling protocols */ __wsum csum;net/core/gro.c +0
-0
net/core/gro.c static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head) · line 440
Quoted 6 lines
net/core/gro.c +3
-0@@ -435,6 +435,9 @@ static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head)napi_gro_complete (napi, oldest); } +#define zeroed_len (offsetof(struct napi_gro_cb, zeroed_end) - \ + offsetof(struct napi_gro_cb, zeroed_start)) +
A leftover from the RFC I guess? It's not used anywhere in the code, so it doesn't break build.
net/core/gro.c static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff · line 468
Quoted 12 lines
net/core/gro.c +4
-1static enum gro_result dev_gro_receive (struct napi_struct *napi,struct sk_buff *skb) {u32 bucket =skb_get_hash_raw (skb) & (GRO_HASH_BUCKETS -1 );@@ -459,29 +462,22 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buffskb_set_network_header (skb,skb_gro_offset (skb));skb_reset_mac_len (skb); -NAPI_GRO_CB (skb)->same_flow =0 ; + BUILD_BUG_ON(sizeof_field(struct napi_gro_cb, zeroed) != sizeof(u32)); +BUILD_BUG_ON (!IS_ALIGNED (offsetof(struct napi_gro_cb , zeroed), +sizeof (u32)));/* Avoid slow unaligned acc */ + *(u32 *)&NAPI_GRO_CB (skb)->zeroed =0 ;
Looks relatively elegant and self-explanatory to me, nice.
Quoted 27 lines
net/core/gro.c +0
-10NAPI_GRO_CB (skb)->flush =skb_is_gso (skb) ||skb_has_frag_list (skb); -NAPI_GRO_CB (skb)->free =0 ; -NAPI_GRO_CB (skb)->encap_mark =0 ; -NAPI_GRO_CB (skb)->recursion_counter =0 ; -NAPI_GRO_CB (skb)->is_fou =0 ;NAPI_GRO_CB (skb)->is_atomic =1 ; -NAPI_GRO_CB (skb)->gro_remcsum_start =0 ;/* Setup for GRO checksum validation */ switch (skb->ip_summed) {case CHECKSUM_COMPLETE :NAPI_GRO_CB (skb)->csum = skb->csum;NAPI_GRO_CB (skb)->csum_valid =1 ; -NAPI_GRO_CB (skb)->csum_cnt =0 ;break ;case CHECKSUM_UNNECESSARY :NAPI_GRO_CB (skb)->csum_cnt = skb->csum_level +1 ; -NAPI_GRO_CB (skb)->csum_valid =0 ;break ; -default : -NAPI_GRO_CB (skb)->csum_cnt =0 ; -NAPI_GRO_CB (skb)->csum_valid =0 ; } pp = INDIRECT_CALL_INET(ptype->callbacks.gro_receive,Signature · 1 line
2.34.1
Al
Re: [PATCH net-next 3/3] net: gro: register gso and gro offload on separate lists
In reply to Paolo AbeniQuoted 23 lines · Paolo Abeni
So that we know each element in
gro_listhas valid gro callbacks (and the same for gso). This allows dropping a bunch of conditional in fastpath.Before:
objdump -tnet/core/gro.o| grep " F .text"0000000000000bb0l F .text000000000000033cdev_gro_receiveAfter:
0000000000000bb0l F .text 0000000000000325dev_gro_receiveSigned-off-by Paolo Abeni <[email protected]>--- include/linux/netdevice.h | 3 +- net/core/gro.c | 90 +++++++++++++++++++++++---------------- 2 files changed, 56 insertions(+), 37 deletions(-)include/linux/netdevice.h +0
-0
include/linux/netdevice.h struct packet_offload { · line 2570
Quoted 8 lines
include/linux/netdevice.h +2
-1@@ -2564,7 +2564,8 @@ struct packet_offload {__be16 type;/* This is really htons(ether_type). */ u16 priority;struct offload_callbacks callbacks; -struct list_head list; +struct list_head gro_list; +struct list_head gso_list; };
On the other hand, this makes this object bigger, increasing the risk of spanning cache lines.
It would be nice to group all struct packet_offload together in the same section to increase data locality.
I played in the past with a similar idea, but splitting struct packet_offload in two structures, one for GRO, one for GSO. (Note that GSO is hardly ever use with modern NIC)
But the gains were really marginal.
Re: [PATCH net-next 1/3] net: gro: avoid re-computing truesize twice on recycle
In reply to Paolo AbeniFrom: Paolo Abeni <[email protected]>
Date: Thu, 3 Feb 2022 16:48:21 +0100
Quoted 3 lines
After commit
5e10da5385d2("skbuff: allow 'slow_gro' for skb carring sock reference") and commitaf352460b465("net: fix GRO skb truesize update") the truesize of freed skb is properly updated
^^^^^
One nit here, I'd change this to "truesize of skb with stolen head" or so. It took me a bit of time to get why we should update the truesize of skb already freed (: Right, napi_reuse_skb() makes use of stolen-data skbs.
Quoted 21 lines
by the GRO engine, we don't need anymore resetting it at recycle time.
Signed-off-by Paolo Abeni <[email protected]>--- net/core/gro.c | 1 - 1 file changed, 1 deletion(-)net/core/gro.c +0
-1@@ -634,7 +634,6 @@ static void napi_reuse_skb(struct napi_struct *napi, struct sk_buff *skb) skb->encapsulation =0 ;skb_shinfo (skb)->gso_type =0 ; - skb->truesize =SKB_TRUESIZE (skb_end_offset (skb));if (unlikely (skb->slow_gro)) {skb_orphan (skb);skb_ext_reset (skb);Signature · 1 line
2.34.1
Thanks,
Al
Re: [PATCH net-next 3/3] net: gro: register gso and gro offload on separate lists
In reply to Eric DumazetFrom: Eric Dumazet <[email protected]>
Date: Thu, 3 Feb 2022 08:11:43 -0800
Quoted 19 lines · Paolo Abeni
Quoted 18 lines
So that we know each element in
gro_listhas valid gro callbacks (and the same for gso). This allows dropping a bunch of conditional in fastpath.Before:
objdump -tnet/core/gro.o| grep " F .text"0000000000000bb0l F .text000000000000033cdev_gro_receiveAfter:
0000000000000bb0l F .text 0000000000000325dev_gro_receiveSigned-off-by Paolo Abeni <[email protected]>--- include/linux/netdevice.h | 3 +- net/core/gro.c | 90 +++++++++++++++++++++++---------------- 2 files changed, 56 insertions(+), 37 deletions(-)
Quoted 16 lines
include/linux/netdevice.h struct packet_offload { · line 2570
Quoted 13 lines
include/linux/netdevice.h +2
-1@@ -2564,7 +2564,8 @@ struct packet_offload {__be16 type;/* This is really htons(ether_type). */ u16 priority;struct offload_callbacks callbacks; -struct list_head list; +struct list_head gro_list; +struct list_head gso_list; };On the other hand, this makes this object bigger, increasing the risk of spanning cache lines.
As you said, GSO callbacks are barely used with most modern NICs. Plus GRO and GSO callbacks are used in the completely different operations.gro_list occupies the same place where the list previously was. Does it make a lot of sense to care about gso_list being placed in a cacheline separate from gro_list?
Quoted 9 lines
It would be nice to group all struct
packet_offloadtogether in the same section to increase data locality.I played in the past with a similar idea, but splitting struct
packet_offloadin two structures, one for GRO, one for GSO. (Note that GSO is hardly ever use with modern NIC)But the gains were really marginal.
Thanks,
Al
Re: [PATCH net-next 2/3] net: gro: minor optimization for dev_gro_receive()
In reply to Paolo AbeniQuoted 39 lines · Paolo Abeni
While inspecting some perf report, I noticed that the compiler emits suboptimal code for the napi CB initialization, fetching and storing multiple times the memory for flags bitfield. This is with gcc 10.3.1, but I observed the same with older compiler versions.
We can help the compiler to do a nicer work clearing several fields at once using an u32 alias. The generated code is quite smaller, with the same number of conditional.
Before:
objdump -tnet/core/gro.o| grep " F .text"0000000000000bb0l F .text 0000000000000357dev_gro_receiveAfter:
0000000000000bb0l F .text 000000000000033cdev_gro_receiveRFC -> v1:
- use__struct_groupto delimt the zeroed area (Alexander)Signed-off-by Paolo Abeni <[email protected]>---
include/net/gro.h | 52 +++++++++++++++++++++++++-------------------- --
net/core/gro.c | 18 +++++++---------2 files changed, 35 insertions(+), 35 deletions(-)include/net/gro.h +0
-0@@ -29,46 +29,50 @@ struct napi_gro_cb { /* Number of segments aggregated. */ u16 count;
- /* Start offset for remote checksum offload */ - u16 gro_remcsum_start;
+ /* Used inipv6_gro_receive()and foo-over-udp */ + u16 proto;
Quoted 8 lines
/* jiffies when first packet was created/queued */ unsigned long age;
- /* Used inipv6_gro_receive()and foo-over-udp */ - u16 proto;
+ /* portion of the cb set to zero at every gro iteration */ + __struct_group(/* no tag */, zeroed, /* no attrs */,
Any specific reason for using __struct_group here rather than the struct_group macro instead?
Re: [PATCH net-next 3/3] net: gro: register gso and gro offload on separate lists
In reply to Alexander LobakinQuoted 40 lines · Alexander Lobakin
From: Eric Dumazet <[email protected]>
Date: Thu, 3 Feb 2022 08:11:43 -0800Quoted 19 lines · Paolo Abeni
Quoted 18 lines
So that we know each element in
gro_listhas valid gro callbacks (and the same for gso). This allows dropping a bunch of conditional in fastpath.Before:
objdump -tnet/core/gro.o| grep " F .text"0000000000000bb0l F .text000000000000033cdev_gro_receiveAfter:
0000000000000bb0l F .text 0000000000000325dev_gro_receiveSigned-off-by Paolo Abeni <[email protected]>--- include/linux/netdevice.h | 3 +- net/core/gro.c | 90 +++++++++++++++++++++++---------------- 2 files changed, 56 insertions(+), 37 deletions(-)Quoted 16 lines
include/linux/netdevice.h struct packet_offload { · line 2570
Quoted 13 lines
include/linux/netdevice.h +2
-1@@ -2564,7 +2564,8 @@ struct packet_offload {__be16 type;/* This is really htons(ether_type). */ u16 priority;struct offload_callbacks callbacks; -struct list_head list; +struct list_head gro_list; +struct list_head gso_list; };On the other hand, this makes this object bigger, increasing the risk of spanning cache lines.
Quoted 6 lines
As you said, GSO callbacks are barely used with most modern NICs. Plus GRO and GSO callbacks are used in the completely different operations.
gro_listoccupies the same place where thelistpreviously was. Does it make a lot of sense to care aboutgso_listbeing placed in a cacheline separate fromgro_list?
Not sure what you are asking in this last sentence.
Putting together all struct packet_offload and struct net_offload would reduce number of cache lines,
but making these objects bigger is conflicting.
Another approach to avoiding conditional tests would be to provide non NULL values
for all "struct offload_callbacks" members, just in case we missed something.
I think the test over ptype->type == type should be enough, right ?
Paolo Abeni
maintainerRe: [PATCH net-next 2/3] net: gro: minor optimization for dev_gro_receive()
In reply to Alexander H DuyckQuoted 43 lines · Alexander H Duyck
Quoted 42 lines · Paolo Abeni
While inspecting some perf report, I noticed that the compiler emits suboptimal code for the napi CB initialization, fetching and storing multiple times the memory for flags bitfield. This is with gcc 10.3.1, but I observed the same with older compiler versions.
We can help the compiler to do a nicer work clearing several fields at once using an u32 alias. The generated code is quite smaller, with the same number of conditional.
Before:
objdump -tnet/core/gro.o| grep " F .text"0000000000000bb0l F .text 0000000000000357dev_gro_receiveAfter:
0000000000000bb0l F .text 000000000000033cdev_gro_receiveRFC -> v1:
- use__struct_groupto delimt the zeroed area (Alexander)Signed-off-by Paolo Abeni <[email protected]>---
include/net/gro.h | 52 +++++++++++++++++++++++++-------------------- --
net/core/gro.c | 18 +++++++---------2 files changed, 35 insertions(+), 35 deletions(-)include/net/gro.h +0
-0@@ -29,46 +29,50 @@ struct napi_gro_cb { /* Number of segments aggregated. */ u16 count;
- /* Start offset for remote checksum offload */ - u16 gro_remcsum_start;
+ /* Used inipv6_gro_receive()and foo-over-udp */ + u16 proto;
/* jiffies when first packet was created/queued */ unsigned long age;
Quoted 8 lines
Quoted 5 lines
- /* Used inipv6_gro_receive()and foo-over-udp */ - u16 proto;
+ /* portion of the cb set to zero at every gro iteration */ + __struct_group(/* no tag */, zeroed, /* no attrs */,Any specific reason for using
__struct_grouphere rather than thestruct_groupmacro instead?
Just sheer ignorance on my side. I'll fix on the next iteration.
Thanks!
Paolo
Paolo Abeni
maintainerRe: [PATCH net-next 3/3] net: gro: register gso and gro offload on separate lists
In reply to Eric DumazetQuoted 55 lines · Eric Dumazet
Quoted 40 lines · Alexander Lobakin
From: Eric Dumazet <[email protected]>
Date: Thu, 3 Feb 2022 08:11:43 -0800Quoted 19 lines · Paolo Abeni
Quoted 18 lines
So that we know each element in
gro_listhas valid gro callbacks (and the same for gso). This allows dropping a bunch of conditional in fastpath.Before:
objdump -tnet/core/gro.o| grep " F .text"0000000000000bb0l F .text000000000000033cdev_gro_receiveAfter:
0000000000000bb0l F .text 0000000000000325dev_gro_receiveSigned-off-by Paolo Abeni <[email protected]>--- include/linux/netdevice.h | 3 +- net/core/gro.c | 90 +++++++++++++++++++++++---------------- 2 files changed, 56 insertions(+), 37 deletions(-)Quoted 16 lines
include/linux/netdevice.h struct packet_offload { · line 2570
Quoted 13 lines
include/linux/netdevice.h +2
-1@@ -2564,7 +2564,8 @@ struct packet_offload {__be16 type;/* This is really htons(ether_type). */ u16 priority;struct offload_callbacks callbacks; -struct list_head list; +struct list_head gro_list; +struct list_head gso_list; };On the other hand, this makes this object bigger, increasing the risk of spanning cache lines.
Quoted 6 lines
As you said, GSO callbacks are barely used with most modern NICs. Plus GRO and GSO callbacks are used in the completely different operations.
gro_listoccupies the same place where thelistpreviously was. Does it make a lot of sense to care aboutgso_listbeing placed in a cacheline separate fromgro_list?Not sure what you are asking in this last sentence.
Putting together all struct
packet_offloadand structnet_offloadwould reduce number of cache lines,
but making these objects bigger is conflicting.
Quoted 5 lines
Another approach to avoiding conditional tests would be to provide non NULL values
for all "structoffload_callbacks" members, just in case we missed something.I think the test over
ptype->type== type should be enough, right ?
I'll try this later approach, it looks simpler.
Also I'll keep this patch separate, as the others looks less controversial.
Thanks!
Paolo