net: use bulk free in kfree_skb_list
Subsystems (2)
PATCH v2 1/1 net: use bulk free in kfree_skb_list
Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in napi_consume_skb.
---
v2: call kmem_cache_free_bulk once the skb array is full instead of
falling back to kfree_skb
net/core/skbuff.c | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-)
net/core/skbuff.c +36 -4
@@ -666,12 +666,44 @@ EXPORT_SYMBOL(kfree_skb);void kfree_skb_list (struct sk_buff *segs) { -while (segs) { -struct sk_buff *next = segs->next; +struct sk_buff *next = segs; +void *skbs[16 ]; +int n_skbs =0 ; -kfree_skb (segs); - segs = next; +while ((segs = next) !=NULL ) { + next = segs->next; + +if (!skb_unref (segs)) +continue ; + +if (segs->fclone !=SKB_FCLONE_UNAVAILABLE ) { +kfree_skb (segs); +continue ; + } + +trace_kfree_skb (segs,__builtin_return_address (0 )); + +/* drop skb->head and call any destructors for packet */ +skb_release_all (segs); + +#ifdef CONFIG_SLUB +/* SLUB writes into objects when freeing */ +prefetchw (segs); +#endif + + skbs[n_skbs++] = segs; + +if (n_skbs <ARRAY_SIZE (skbs)) +continue ; + +kmem_cache_free_bulk (skbuff_head_cache, n_skbs, skbs); + n_skbs =0 ; } + +if (!n_skbs) +return ; + +kmem_cache_free_bulk (skbuff_head_cache, n_skbs, skbs); }EXPORT_SYMBOL (kfree_skb_list);
Signature · 1 line
2.17.0
Quoted 7 lines · On Sun, 24 Mar 2019 17:56:44 +0100 Felix Fietkau <[email protected]> wrote:
Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in
napi_consume_skb.Signed-off-by Felix Fietkau <[email protected]>---
v2: callkmem_cache_free_bulkonce the skb array is full instead of
falling back tokfree_skb
Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
Quoted 3 lines · Felix Fietkau
Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in
napi_consume_skb.
I do not believe kfree_skb_list() is used in the fast path, so do we really need to make it so complex ?
Quoted 9 lines · Eric Dumazet
Quoted 3 lines · Felix Fietkau
Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in
napi_consume_skb.I do not believe
kfree_skb_list()is used in the fast path, so do we really need to make it so complex ?
mac80211 uses it to free the fraglist from A-MSDU aggregated packets in the tx status path. That's one fast path where it gets used right now and the reason it was showing up in my perf traces.
- Felix
Quoted 8 lines · Felix Fietkau
Quoted 7 lines · Eric Dumazet
Quoted 3 lines · Felix Fietkau
Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in
napi_consume_skb.
Quoted 5 lines
Quoted 2 lines
I do not believe
kfree_skb_list()is used in the fast path, so do we really need to make it so complex ?mac80211 uses it to free the fraglist from A-MSDU aggregated packets in the tx status path. That's one fast path where it gets used right now and the reason it was showing up in my perf traces.
This is not drop monitor friendly then....
TX completion should use consume_skb() or dev_kfree_skb()
BTW, I wonder what drop-monitor signal bulk free is sending ?
Quoted 17 lines · Eric Dumazet
Quoted 8 lines · Felix Fietkau
Quoted 7 lines · Eric Dumazet
Quoted 3 lines · Felix Fietkau
Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in
napi_consume_skb.Quoted 5 lines
Quoted 2 lines
I do not believe
kfree_skb_list()is used in the fast path, so do we really need to make it so complex ?mac80211 uses it to free the fraglist from A-MSDU aggregated packets in the tx status path. That's one fast path where it gets used right now and the reason it was showing up in my perf traces.
Quoted 5 lines
This is not drop monitor friendly then....
TX completion should use
consume_skb()ordev_kfree_skb()BTW, I wonder what drop-monitor signal bulk free is sending ?
Good point about the drop monitor. Would you prefer that I replace this patch with one that adds a consume_skb_list function and another one that makes mac80211 use it?
- Felix
Quoted 9 lines · On Mon, 25 Mar 2019 10:35:35 +0100 Felix Fietkau <[email protected]> wrote:
Quoted 8 lines · Eric Dumazet
Quoted 7 lines · Felix Fietkau
Quoted 6 lines · Eric Dumazet
Quoted 4 lines · Felix Fietkau
Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in
napi_consume_skb.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
I do not believe
kfree_skb_list()is used in the fast path, so do we really need to make it so complex ?mac80211 uses it to free the fraglist from A-MSDU aggregated packets in the tx status path. That's one fast path where it gets used right now and the reason it was showing up in my perf traces.
Notice that kfree_skb_list(to_free) is used in (what I consider) "fast-path" when the qdisc system drops packets, happens via to_free in bottom of __dev_xmit_skb(). And fq_codel (which is default in most distro's today) have an optimization (in fq_codel_drop()) for bulk freeing SKBs, which will also benefit from this.
Thus, I still think it will be valuable to optimize kfree_skb_list().
Quoted 9 lines
Quoted 5 lines
This is not drop monitor friendly then....
TX completion should use
consume_skb()ordev_kfree_skb()BTW, I wonder what drop-monitor signal bulk free is sending ?
Good point about the drop monitor. Would you prefer that I replace this patch with one that adds a
consume_skb_listfunction and another one that makes mac80211 use it?
Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
Paolo Abeni
maintainerHi,
Quoted 26 lines · Felix Fietkau
Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in
napi_consume_skb.Signed-off-by Felix Fietkau <[email protected]>---
v2: callkmem_cache_free_bulkonce the skb array is full instead of
falling back tokfree_skbnet/core/skbuff.c | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-)net/core/skbuff.c +3
-4@@ -666,12 +666,44 @@ EXPORT_SYMBOL(kfree_skb);void kfree_skb_list (struct sk_buff *segs) { -while (segs) { -struct sk_buff *next = segs->next; +struct sk_buff *next = segs; +void *skbs[16 ]; +int n_skbs =0 ; -kfree_skb (segs); - segs = next;
net/core/skbuff.c EXPORT_SYMBOL(kfree_skb); · line 682
Quoted 10 lines
net/core/skbuff.c +10
-0+while ((segs = next) !=NULL ) { + next = segs->next; + +if (!skb_unref (segs)) +continue ; + +if (fclone !=SKB_FCLONE_UNAVAILABLE ) { +kfree_skb (segs); +continue ; + }
I think you should swap the order of skb_unref() and the above check, or skbs with 'fclone != SKB_FCLONE_UNAVAILABLE' will go twice in skb_unref() (kfree_skb() calls skb_unref(), too).
Other than that LGTM,
Thanks,
Paolo