Skip to contentSkip to conversation

net: save some cycles when doing skb_attempt_defer_free()

PATCH Superseded Jason Xing 3 participants 7 messages Apr 11, 2024

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Jason Xing

author
#0 lore raw

PATCH 1/1 net: save some cycles when doing skb_attempt_defer_free()

From: Jason Xing <[email protected]>

Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as skb->alloc_cpu.

One simple test that can help us see the frequency of this statement 'cpu == raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to capture skb_attempt_defer_free()

I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.

---
 net/core/skbuff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
net/core/skbuff.c +2 -2
index ab970ded8a7b..b4f252dc91fb 100644
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)
 	unsigned int defer_max;
 	bool kick;
 
-	if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
+	if (cpu == raw_smp_processor_id() ||
 	    !cpu_online(cpu) ||
-	    cpu == raw_smp_processor_id()) {
+	    WARN_ON_ONCE(cpu >= nr_cpu_ids)) {
 nodefer:	kfree_skb_napi_cache(skb);
 		return;
 	}
Signature · 1 line
2.37.3
#1 lore raw
Quoted 27 lines · Jason Xing

From: Jason Xing <[email protected]>

Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as skb->alloc_cpu.

One simple test that can help us see the frequency of this statement 'cpu == raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to capture skb_attempt_defer_free()

I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.

---
 net/core/skbuff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
net/core/skbuff.c +0 -0
index ab970ded8a7b..b4f252dc91fb 100644
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)
        unsigned int defer_max;

net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010

Quoted 10 lines
net/core/skbuff.c +2 -2
        bool kick;
 
-       if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
+       if (cpu == raw_smp_processor_id() ||
            !cpu_online(cpu) ||
-           cpu == raw_smp_processor_id()) {
+           WARN_ON_ONCE(cpu >= nr_cpu_ids)) {
 nodefer:       kfree_skb_napi_cache(skb);
                return;
        }

Wrong patch.

cpu_online(X) is undefined and might crash if X is out of bounds on CONFIG_SMP=y

author
#2 lore raw
Quoted 42 lines · Eric Dumazet
Quoted 27 lines · Jason Xing

From: Jason Xing <[email protected]>

Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as skb->alloc_cpu.

One simple test that can help us see the frequency of this statement 'cpu == raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to capture skb_attempt_defer_free()

I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.

---
 net/core/skbuff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
net/core/skbuff.c +0 -0
index ab970ded8a7b..b4f252dc91fb 100644
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)
        unsigned int defer_max;

net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010

Quoted 10 lines
net/core/skbuff.c +2 -2
        bool kick;
 
-       if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
+       if (cpu == raw_smp_processor_id() ||
            !cpu_online(cpu) ||
-           cpu == raw_smp_processor_id()) {
+           WARN_ON_ONCE(cpu >= nr_cpu_ids)) {
 nodefer:       kfree_skb_napi_cache(skb);
                return;
        }

Wrong patch.

Quoted 1 line

cpu_online(X) is undefined and might crash if X is out of bounds on CONFIG_SMP=y

Even if skb->alloc_cpu is larger than nr_cpu_ids, I don't know why the integer test statement could cause crashing the kernel. It's just a simple comparison. And if the statement is true, raw_smp_processor_id() can guarantee the validation, right?

#3 lore raw
Quoted 44 lines · Jason Xing
Quoted 42 lines · Eric Dumazet
Quoted 27 lines · Jason Xing

From: Jason Xing <[email protected]>

Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as skb->alloc_cpu.

One simple test that can help us see the frequency of this statement 'cpu == raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to capture skb_attempt_defer_free()

I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.

---
 net/core/skbuff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
net/core/skbuff.c +0 -0
index ab970ded8a7b..b4f252dc91fb 100644
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)
        unsigned int defer_max;

net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010

Quoted 10 lines
net/core/skbuff.c +2 -2
        bool kick;
 
-       if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
+       if (cpu == raw_smp_processor_id() ||
            !cpu_online(cpu) ||
-           cpu == raw_smp_processor_id()) {
+           WARN_ON_ONCE(cpu >= nr_cpu_ids)) {
 nodefer:       kfree_skb_napi_cache(skb);
                return;
        }

Wrong patch.

Quoted 6 lines
Quoted 1 line

cpu_online(X) is undefined and might crash if X is out of bounds on CONFIG_SMP=y

Even if skb->alloc_cpu is larger than nr_cpu_ids, I don't know why the integer test statement could cause crashing the kernel. It's just a simple comparison. And if the statement is true, raw_smp_processor_id() can guarantee the validation, right?

Please read again the code you wrote, or run it with skb->alloc_cpu being set to 45000 on a full DEBUG kernel.

You are focusing on skb->alloc_cpu == raw_smp_processor_id(), I am focusing on what happens
when this condition is not true.

author
#4 lore raw
Quoted 53 lines · Eric Dumazet
Quoted 44 lines · Jason Xing
Quoted 42 lines · Eric Dumazet
Quoted 27 lines · Jason Xing

From: Jason Xing <[email protected]>

Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as skb->alloc_cpu.

One simple test that can help us see the frequency of this statement 'cpu == raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to capture skb_attempt_defer_free()

I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.

---
 net/core/skbuff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
net/core/skbuff.c +0 -0
index ab970ded8a7b..b4f252dc91fb 100644
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)
        unsigned int defer_max;

net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010

Quoted 10 lines
net/core/skbuff.c +2 -2
        bool kick;
 
-       if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
+       if (cpu == raw_smp_processor_id() ||
            !cpu_online(cpu) ||
-           cpu == raw_smp_processor_id()) {
+           WARN_ON_ONCE(cpu >= nr_cpu_ids)) {
 nodefer:       kfree_skb_napi_cache(skb);
                return;
        }

Wrong patch.

Quoted 6 lines
Quoted 1 line

cpu_online(X) is undefined and might crash if X is out of bounds on CONFIG_SMP=y

Even if skb->alloc_cpu is larger than nr_cpu_ids, I don't know why the integer test statement could cause crashing the kernel. It's just a simple comparison. And if the statement is true, raw_smp_processor_id() can guarantee the validation, right?

Quoted 6 lines

Please read again the code you wrote, or run it with skb->alloc_cpu being set to 45000 on a full DEBUG kernel.

You are focusing on skb->alloc_cpu == raw_smp_processor_id(), I am focusing on what happens
when this condition is not true.

Sorry. My bad. I put the wrong order of '!cpu_online(cpu)' and 'cpu >= nr_cpu_ids'. I didn't consider the out-of-bound issue. I should have done more checks :(

The correct patch should be:

net/core/skbuff.c +3 -3
index ab970ded8a7b..6dc577a3ea6a 100644
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)
        unsigned int defer_max;
        bool kick;
 
-       if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
-           !cpu_online(cpu) ||
-           cpu == raw_smp_processor_id()) {
+       if (cpu == raw_smp_processor_id() ||
+           WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
+           !cpu_online(cpu)) {
 nodefer:       kfree_skb_napi_cache(skb);
                return;
        }

I will submit V2 tomorrow.

Thanks,
Jason

Reviewed-by
#5 lore raw

From: Jason Xing <[email protected]>
Date: Thu, 11 Apr 2024 15:31:23 +0800

Quoted 71 lines · Eric Dumazet
Quoted 53 lines · Jason Xing
Quoted 44 lines · Eric Dumazet
Quoted 42 lines · Jason Xing
Quoted 27 lines

From: Jason Xing <[email protected]>

Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as skb->alloc_cpu.

One simple test that can help us see the frequency of this statement 'cpu == raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to capture skb_attempt_defer_free()

I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.

---
 net/core/skbuff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
net/core/skbuff.c +0 -0
index ab970ded8a7b..b4f252dc91fb 100644
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)
        unsigned int defer_max;

net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010

Quoted 10 lines
net/core/skbuff.c +2 -2
        bool kick;
 
-       if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
+       if (cpu == raw_smp_processor_id() ||
            !cpu_online(cpu) ||
-           cpu == raw_smp_processor_id()) {
+           WARN_ON_ONCE(cpu >= nr_cpu_ids)) {
 nodefer:       kfree_skb_napi_cache(skb);
                return;
        }

Wrong patch.

Quoted 6 lines
Quoted 1 line

cpu_online(X) is undefined and might crash if X is out of bounds on CONFIG_SMP=y

Even if skb->alloc_cpu is larger than nr_cpu_ids, I don't know why the integer test statement could cause crashing the kernel. It's just a simple comparison. And if the statement is true, raw_smp_processor_id() can guarantee the validation, right?

Quoted 6 lines

Please read again the code you wrote, or run it with skb->alloc_cpu being set to 45000 on a full DEBUG kernel.

You are focusing on skb->alloc_cpu == raw_smp_processor_id(), I am focusing on what happens
when this condition is not true.

Sorry. My bad. I put the wrong order of '!cpu_online(cpu)' and 'cpu >= nr_cpu_ids'. I didn't consider the out-of-bound issue. I should have done more checks :(

The correct patch should be:

net/core/skbuff.c +0 -0
index ab970ded8a7b..6dc577a3ea6a 100644

net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7007

Quoted 9 lines
net/core/skbuff.c +3 -3
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)
        unsigned int defer_max;
        bool kick;
 
-       if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
-           !cpu_online(cpu) ||
-           cpu == raw_smp_processor_id()) {
+       if (cpu == raw_smp_processor_id() ||
+           WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
+           !cpu_online(cpu)) {

This one looks good to me.
Feel free to add

To your v2 before sending.

Quoted 8 lines
net/core/skbuff.c +0 -0
 nodefer:       kfree_skb_napi_cache(skb);
                return;
        }

I will submit V2 tomorrow.

Thanks,
Jason

Thanks,
Olek

author
#6 lore raw
Quoted 88 lines · Alexander Lobakin

From: Jason Xing <[email protected]>
Date: Thu, 11 Apr 2024 15:31:23 +0800

Quoted 71 lines · Eric Dumazet
Quoted 53 lines · Jason Xing
Quoted 44 lines · Eric Dumazet
Quoted 42 lines · Jason Xing
Quoted 27 lines

From: Jason Xing <[email protected]>

Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as skb->alloc_cpu.

One simple test that can help us see the frequency of this statement 'cpu == raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to capture skb_attempt_defer_free()

I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.

---
 net/core/skbuff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
net/core/skbuff.c +0 -0
index ab970ded8a7b..b4f252dc91fb 100644
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)
        unsigned int defer_max;

net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010

Quoted 10 lines
net/core/skbuff.c +2 -2
        bool kick;
 
-       if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
+       if (cpu == raw_smp_processor_id() ||
            !cpu_online(cpu) ||
-           cpu == raw_smp_processor_id()) {
+           WARN_ON_ONCE(cpu >= nr_cpu_ids)) {
 nodefer:       kfree_skb_napi_cache(skb);
                return;
        }

Wrong patch.

Quoted 6 lines
Quoted 1 line

cpu_online(X) is undefined and might crash if X is out of bounds on CONFIG_SMP=y

Even if skb->alloc_cpu is larger than nr_cpu_ids, I don't know why the integer test statement could cause crashing the kernel. It's just a simple comparison. And if the statement is true, raw_smp_processor_id() can guarantee the validation, right?

Quoted 6 lines

Please read again the code you wrote, or run it with skb->alloc_cpu being set to 45000 on a full DEBUG kernel.

You are focusing on skb->alloc_cpu == raw_smp_processor_id(), I am focusing on what happens
when this condition is not true.

Sorry. My bad. I put the wrong order of '!cpu_online(cpu)' and 'cpu >= nr_cpu_ids'. I didn't consider the out-of-bound issue. I should have done more checks :(

The correct patch should be:

net/core/skbuff.c +0 -0
index ab970ded8a7b..6dc577a3ea6a 100644

net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7007

Quoted 9 lines
net/core/skbuff.c +3 -3
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)
        unsigned int defer_max;
        bool kick;
 
-       if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
-           !cpu_online(cpu) ||
-           cpu == raw_smp_processor_id()) {
+       if (cpu == raw_smp_processor_id() ||
+           WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
+           !cpu_online(cpu)) {

This one looks good to me.
Feel free to add

Quoted 3 lines

To your v2 before sending.

Thanks! I will:)