net: use bulk free in kfree_skb_list
Subsystems (2)
PATCH v3 1/1 net: use bulk free in kfree_skb_list
Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in napi_consume_skb.
---
v3: reorder checks to prevent skb double unref
v2: call kmem_cache_free_bulk once the skb array is full instead of
falling back to kfree_skb
net/core/skbuff.c | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-)
net/core/skbuff.c +36 -4
@@ -666,12 +666,44 @@ EXPORT_SYMBOL(kfree_skb);void kfree_skb_list (struct sk_buff *segs) { -while (segs) { -struct sk_buff *next = segs->next; +struct sk_buff *next = segs; +void *skbs[16 ]; +int n_skbs =0 ; -kfree_skb (segs); - segs = next; +while ((segs = next) !=NULL ) { + next = segs->next; + +if (segs->fclone !=SKB_FCLONE_UNAVAILABLE ) { +kfree_skb (segs); +continue ; + } + +if (!skb_unref (segs)) +continue ; + +trace_kfree_skb (segs,__builtin_return_address (0 )); + +/* drop skb->head and call any destructors for packet */ +skb_release_all (segs); + +#ifdef CONFIG_SLUB +/* SLUB writes into objects when freeing */ +prefetchw (segs); +#endif + + skbs[n_skbs++] = segs; + +if (n_skbs <ARRAY_SIZE (skbs)) +continue ; + +kmem_cache_free_bulk (skbuff_head_cache, n_skbs, skbs); + n_skbs =0 ; } + +if (!n_skbs) +return ; + +kmem_cache_free_bulk (skbuff_head_cache, n_skbs, skbs); }EXPORT_SYMBOL (kfree_skb_list);
Signature · 1 line
2.17.0
Quoted 39 lines · Felix Fietkau
Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in
napi_consume_skb.Signed-off-by Felix Fietkau <[email protected]>---
v3: reorder checks to prevent skb double unref
v2: callkmem_cache_free_bulkonce the skb array is full instead of
falling back tokfree_skbnet/core/skbuff.c | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-)net/core/skbuff.c +15
-4@@ -666,12 +666,44 @@ EXPORT_SYMBOL(kfree_skb);void kfree_skb_list (struct sk_buff *segs) { -while (segs) { -struct sk_buff *next = segs->next; +struct sk_buff *next = segs; +void *skbs[16 ]; +int n_skbs =0 ; -kfree_skb (segs); - segs = next; +while ((segs = next) !=NULL ) { + next = segs->next; + +if (segs->fclone !=SKB_FCLONE_UNAVAILABLE ) { +kfree_skb (segs); +continue ; + } + +if (!skb_unref (segs)) +continue ; + +trace_kfree_skb (segs,__builtin_return_address (0 ));
net/core/skbuff.c EXPORT_SYMBOL(kfree_skb); · line 694
Quoted 10 lines
net/core/skbuff.c +8
-0+ +/* drop skb->head and call any destructors for packet */ +skb_release_all (segs); + +#ifdef CONFIG_SLUB +/* SLUB writes into objects when freeing */ +prefetchw (segs); +#endif
This is done too late :
You should probably either remove this prefetchw()
or do it before reading segs->next at the beginning of the loop.
Quoted 18 lines
net/core/skbuff.c +13
-0+ + skbs[n_skbs++] = segs; + +if (n_skbs <ARRAY_SIZE (skbs)) +continue ; + +kmem_cache_free_bulk (skbuff_head_cache, n_skbs, skbs); + n_skbs =0 ; } + +if (!n_skbs) +return ; + +kmem_cache_free_bulk (skbuff_head_cache, n_skbs, skbs); }
EXPORT_SYMBOL(kfree_skb_list);
Quoted 8 lines · On Mon, 25 Mar 2019 02:27:14 -0700 Eric Dumazet <[email protected]> wrote:
net/core/skbuff.c EXPORT_SYMBOL(kfree_skb); · line 692
Quoted 4 lines
net/core/skbuff.c +4
-0+#ifdef CONFIG_SLUB +/* SLUB writes into objects when freeing */ +prefetchw (segs); +#endif This is done too late :
You should probably either remove thisprefetchw()
or do it before readingsegs->nextat the beginning of the loop.
Agree. Not sure the prefetchw optimization makes sense here. IHMO just drop it in this patch.
Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer