Skip to contentSkip to conversation

net: use bulk free in kfree_skb_list

PATCH Superseded Felix Fietkau 4 participants 8 messages Mar 24, 2019 – Mar 25, 2019

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 net: use bulk free in kfree_skb_list

Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in napi_consume_skb.

---
v2: call kmem_cache_free_bulk once the skb array is full instead of
falling back to kfree_skb

 net/core/skbuff.c | 40 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 36 insertions(+), 4 deletions(-)
net/core/skbuff.c +36 -4
index 2415d9cb9b89..1eeaa264d2a4 100644
@@ -666,12 +666,44 @@ EXPORT_SYMBOL(kfree_skb);
 
 void kfree_skb_list(struct sk_buff *segs)
 {
-	while (segs) {
-		struct sk_buff *next = segs->next;
+	struct sk_buff *next = segs;
+	void *skbs[16];
+	int n_skbs = 0;
 
-		kfree_skb(segs);
-		segs = next;
+	while ((segs = next) != NULL) {
+		next = segs->next;
+
+		if (!skb_unref(segs))
+			continue;
+
+		if (segs->fclone != SKB_FCLONE_UNAVAILABLE) {
+			kfree_skb(segs);
+			continue;
+		}
+
+		trace_kfree_skb(segs, __builtin_return_address(0));
+
+		/* drop skb->head and call any destructors for packet */
+		skb_release_all(segs);
+
+#ifdef CONFIG_SLUB
+		/* SLUB writes into objects when freeing */
+		prefetchw(segs);
+#endif
+
+		skbs[n_skbs++] = segs;
+
+		if (n_skbs < ARRAY_SIZE(skbs))
+			continue;
+
+		kmem_cache_free_bulk(skbuff_head_cache, n_skbs, skbs);
+		n_skbs = 0;
 	}
+
+	if (!n_skbs)
+		return;
+
+	kmem_cache_free_bulk(skbuff_head_cache, n_skbs, skbs);
 }
 EXPORT_SYMBOL(kfree_skb_list);
 
Signature · 1 line
2.17.0
Acked-by on v2 1/1
#1 lore raw
Quoted 7 lines · On Sun, 24 Mar 2019 17:56:44 +0100 Felix Fietkau <[email protected]> wrote:

Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in napi_consume_skb.

---
v2: call kmem_cache_free_bulk once the skb array is full instead of
falling back to kfree_skb

Signature · 4 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer
#2 lore raw
In reply to Felix Fietkau
Quoted 3 lines · Felix Fietkau

Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in napi_consume_skb.

I do not believe kfree_skb_list() is used in the fast path, so do we really need to make it so complex ?

#3 lore raw
Quoted 9 lines · Eric Dumazet
Quoted 3 lines · Felix Fietkau

Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in napi_consume_skb.

I do not believe kfree_skb_list() is used in the fast path, so do we really need to make it so complex ?

mac80211 uses it to free the fraglist from A-MSDU aggregated packets in the tx status path. That's one fast path where it gets used right now and the reason it was showing up in my perf traces.

- Felix

#4 lore raw
Quoted 8 lines · Felix Fietkau
Quoted 7 lines · Eric Dumazet
Quoted 3 lines · Felix Fietkau

Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in napi_consume_skb.

Quoted 5 lines
Quoted 2 lines

I do not believe kfree_skb_list() is used in the fast path, so do we really need to make it so complex ?

mac80211 uses it to free the fraglist from A-MSDU aggregated packets in the tx status path. That's one fast path where it gets used right now and the reason it was showing up in my perf traces.

This is not drop monitor friendly then....

TX completion should use consume_skb() or dev_kfree_skb()

BTW, I wonder what drop-monitor signal bulk free is sending ?

#5 lore raw
Quoted 17 lines · Eric Dumazet
Quoted 8 lines · Felix Fietkau
Quoted 7 lines · Eric Dumazet
Quoted 3 lines · Felix Fietkau

Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in napi_consume_skb.

Quoted 5 lines
Quoted 2 lines

I do not believe kfree_skb_list() is used in the fast path, so do we really need to make it so complex ?

mac80211 uses it to free the fraglist from A-MSDU aggregated packets in the tx status path. That's one fast path where it gets used right now and the reason it was showing up in my perf traces.

Quoted 5 lines

This is not drop monitor friendly then....

TX completion should use consume_skb() or dev_kfree_skb()

BTW, I wonder what drop-monitor signal bulk free is sending ?

Good point about the drop monitor. Would you prefer that I replace this patch with one that adds a consume_skb_list function and another one that makes mac80211 use it?

- Felix

#6 lore raw
Quoted 9 lines · On Mon, 25 Mar 2019 10:35:35 +0100 Felix Fietkau <[email protected]> wrote:
Quoted 8 lines · Eric Dumazet
Quoted 7 lines · Felix Fietkau
Quoted 6 lines · Eric Dumazet
Quoted 4 lines · Felix Fietkau

Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in napi_consume_skb.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

I do not believe kfree_skb_list() is used in the fast path, so do we really need to make it so complex ?

mac80211 uses it to free the fraglist from A-MSDU aggregated packets in the tx status path. That's one fast path where it gets used right now and the reason it was showing up in my perf traces.

Notice that kfree_skb_list(to_free) is used in (what I consider) "fast-path" when the qdisc system drops packets, happens via to_free in bottom of __dev_xmit_skb(). And fq_codel (which is default in most distro's today) have an optimization (in fq_codel_drop()) for bulk freeing SKBs, which will also benefit from this.

Thus, I still think it will be valuable to optimize kfree_skb_list().

Quoted 9 lines
Quoted 5 lines

This is not drop monitor friendly then....

TX completion should use consume_skb() or dev_kfree_skb()

BTW, I wonder what drop-monitor signal bulk free is sending ?

Good point about the drop monitor. Would you prefer that I replace this patch with one that adds a consume_skb_list function and another one that makes mac80211 use it?

Signature · 4 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer
maintainer
#7 lore raw
In reply to Felix Fietkau

Hi,

Quoted 26 lines · Felix Fietkau

Since we're freeing multiple skbs, we might as well use bulk free to save a few cycles. Use the same conditions for bulk free as in napi_consume_skb.

---
v2: call kmem_cache_free_bulk once the skb array is full instead of
falling back to kfree_skb

 net/core/skbuff.c | 40 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 36 insertions(+), 4 deletions(-)
net/core/skbuff.c +3 -4
index 2415d9cb9b89..1eeaa264d2a4 100644
@@ -666,12 +666,44 @@ EXPORT_SYMBOL(kfree_skb);
 
 void kfree_skb_list(struct sk_buff *segs)
 {
-	while (segs) {
-		struct sk_buff *next = segs->next;
+	struct sk_buff *next = segs;
+	void *skbs[16];
+	int n_skbs = 0;
 
-		kfree_skb(segs);
-		segs = next;

net/core/skbuff.c EXPORT_SYMBOL(kfree_skb); · line 682

Quoted 10 lines
net/core/skbuff.c +10 -0
+	while ((segs = next) != NULL) {
+		next = segs->next;
+
+		if (!skb_unref(segs))
+			continue;
+
+		if (fclone != SKB_FCLONE_UNAVAILABLE) {
+			kfree_skb(segs);
+			continue;
+		}

I think you should swap the order of skb_unref() and the above check, or skbs with 'fclone != SKB_FCLONE_UNAVAILABLE' will go twice in skb_unref() (kfree_skb() calls skb_unref(), too).

Other than that LGTM,

Thanks,

Paolo