net: save some cycles when doing skb_attempt_defer_free()
Subsystems (2)
PATCH 1/1 net: save some cycles when doing skb_attempt_defer_free()
From: Jason Xing <[email protected]>
Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as skb->alloc_cpu.
One simple test that can help us see the frequency of this statement 'cpu == raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to capture skb_attempt_defer_free()
I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.
--- net/core/skbuff.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
net/core/skbuff.c +2 -2
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)unsigned int defer_max;bool kick; -if (WARN_ON_ONCE (cpu >= nr_cpu_ids) || +if (cpu ==raw_smp_processor_id () || !cpu_online (cpu) || - cpu ==raw_smp_processor_id ()) { +WARN_ON_ONCE (cpu >= nr_cpu_ids)) { nodefer:kfree_skb_napi_cache (skb);return ; }
Signature · 1 line
2.37.3
Quoted 27 lines · Jason Xing
From: Jason Xing <[email protected]>
Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as
skb->alloc_cpu.One simple test that can help us see the frequency of this statement 'cpu ==
raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to captureskb_attempt_defer_free()I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.
Signed-off-by Jason Xing <[email protected]>--- net/core/skbuff.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)net/core/skbuff.c +0
-0@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)unsigned int defer_max;
net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010
Quoted 10 lines
net/core/skbuff.c +2
-2bool kick; -if (WARN_ON_ONCE (cpu >= nr_cpu_ids) || +if (cpu ==raw_smp_processor_id () || !cpu_online (cpu) || - cpu ==raw_smp_processor_id ()) { +WARN_ON_ONCE (cpu >= nr_cpu_ids)) { nodefer:kfree_skb_napi_cache (skb);return ; }
Wrong patch.
cpu_online(X) is undefined and might crash if X is out of bounds on CONFIG_SMP=y
Quoted 42 lines · Eric Dumazet
Quoted 27 lines · Jason Xing
From: Jason Xing <[email protected]>
Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as
skb->alloc_cpu.One simple test that can help us see the frequency of this statement 'cpu ==
raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to captureskb_attempt_defer_free()I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.
Signed-off-by Jason Xing <[email protected]>--- net/core/skbuff.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)net/core/skbuff.c +0
-0@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)unsigned int defer_max;net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010
Quoted 10 lines
net/core/skbuff.c +2
-2bool kick; -if (WARN_ON_ONCE (cpu >= nr_cpu_ids) || +if (cpu ==raw_smp_processor_id () || !cpu_online (cpu) || - cpu ==raw_smp_processor_id ()) { +WARN_ON_ONCE (cpu >= nr_cpu_ids)) { nodefer:kfree_skb_napi_cache (skb);return ; }Wrong patch.
Quoted 1 line
cpu_online(X)is undefined and might crash if X is out of bounds onCONFIG_SMP=y
Even if skb->alloc_cpu is larger than nr_cpu_ids, I don't know why the integer test statement could cause crashing the kernel. It's just a simple comparison. And if the statement is true, raw_smp_processor_id() can guarantee the validation, right?
Quoted 44 lines · Jason Xing
Quoted 42 lines · Eric Dumazet
Quoted 27 lines · Jason Xing
From: Jason Xing <[email protected]>
Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as
skb->alloc_cpu.One simple test that can help us see the frequency of this statement 'cpu ==
raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to captureskb_attempt_defer_free()I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.
Signed-off-by Jason Xing <[email protected]>--- net/core/skbuff.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)net/core/skbuff.c +0
-0@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)unsigned int defer_max;net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010
Quoted 10 lines
net/core/skbuff.c +2
-2bool kick; -if (WARN_ON_ONCE (cpu >= nr_cpu_ids) || +if (cpu ==raw_smp_processor_id () || !cpu_online (cpu) || - cpu ==raw_smp_processor_id ()) { +WARN_ON_ONCE (cpu >= nr_cpu_ids)) { nodefer:kfree_skb_napi_cache (skb);return ; }Wrong patch.
Quoted 6 lines
Quoted 1 line
cpu_online(X)is undefined and might crash if X is out of bounds onCONFIG_SMP=yEven if
skb->alloc_cpuis larger thannr_cpu_ids, I don't know why the integer test statement could cause crashing the kernel. It's just a simple comparison. And if the statement is true,raw_smp_processor_id()can guarantee the validation, right?
Please read again the code you wrote, or run it with skb->alloc_cpu being set to 45000 on a full DEBUG kernel.
You are focusing on skb->alloc_cpu == raw_smp_processor_id(), I am focusing on what happens
when this condition is not true.
Quoted 53 lines · Eric Dumazet
Quoted 44 lines · Jason Xing
Quoted 42 lines · Eric Dumazet
Quoted 27 lines · Jason Xing
From: Jason Xing <[email protected]>
Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as
skb->alloc_cpu.One simple test that can help us see the frequency of this statement 'cpu ==
raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to captureskb_attempt_defer_free()I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.
Signed-off-by Jason Xing <[email protected]>--- net/core/skbuff.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)net/core/skbuff.c +0
-0@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)unsigned int defer_max;net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010
Quoted 10 lines
net/core/skbuff.c +2
-2bool kick; -if (WARN_ON_ONCE (cpu >= nr_cpu_ids) || +if (cpu ==raw_smp_processor_id () || !cpu_online (cpu) || - cpu ==raw_smp_processor_id ()) { +WARN_ON_ONCE (cpu >= nr_cpu_ids)) { nodefer:kfree_skb_napi_cache (skb);return ; }Wrong patch.
Quoted 6 lines
Quoted 1 line
cpu_online(X)is undefined and might crash if X is out of bounds onCONFIG_SMP=yEven if
skb->alloc_cpuis larger thannr_cpu_ids, I don't know why the integer test statement could cause crashing the kernel. It's just a simple comparison. And if the statement is true,raw_smp_processor_id()can guarantee the validation, right?
Quoted 6 lines
Please read again the code you wrote, or run it with
skb->alloc_cpubeing set to 45000 on a full DEBUG kernel.You are focusing on
skb->alloc_cpu==raw_smp_processor_id(), I am focusing on what happens
when this condition is not true.
Sorry. My bad. I put the wrong order of '!cpu_online(cpu)' and 'cpu >= nr_cpu_ids'. I didn't consider the out-of-bound issue. I should have done more checks :(
The correct patch should be:
net/core/skbuff.c +3 -3
@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)unsigned int defer_max;bool kick; -if (WARN_ON_ONCE (cpu >= nr_cpu_ids) || - !cpu_online (cpu) || - cpu ==raw_smp_processor_id ()) { +if (cpu ==raw_smp_processor_id () || +WARN_ON_ONCE (cpu >= nr_cpu_ids) || + !cpu_online (cpu)) { nodefer:kfree_skb_napi_cache (skb);return ; }
I will submit V2 tomorrow.
Thanks,
Jason
Alexander Lobakin
Reviewed-byFrom: Jason Xing <[email protected]>
Date: Thu, 11 Apr 2024 15:31:23 +0800
Quoted 71 lines · Eric Dumazet
Quoted 53 lines · Jason Xing
Quoted 44 lines · Eric Dumazet
Quoted 42 lines · Jason Xing
Quoted 27 lines
From: Jason Xing <[email protected]>
Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as
skb->alloc_cpu.One simple test that can help us see the frequency of this statement 'cpu ==
raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to captureskb_attempt_defer_free()I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.
Signed-off-by Jason Xing <[email protected]>--- net/core/skbuff.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)net/core/skbuff.c +0
-0@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)unsigned int defer_max;net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010
Quoted 10 lines
net/core/skbuff.c +2
-2bool kick; -if (WARN_ON_ONCE (cpu >= nr_cpu_ids) || +if (cpu ==raw_smp_processor_id () || !cpu_online (cpu) || - cpu ==raw_smp_processor_id ()) { +WARN_ON_ONCE (cpu >= nr_cpu_ids)) { nodefer:kfree_skb_napi_cache (skb);return ; }Wrong patch.
Quoted 6 lines
Quoted 1 line
cpu_online(X)is undefined and might crash if X is out of bounds onCONFIG_SMP=yEven if
skb->alloc_cpuis larger thannr_cpu_ids, I don't know why the integer test statement could cause crashing the kernel. It's just a simple comparison. And if the statement is true,raw_smp_processor_id()can guarantee the validation, right?Quoted 6 lines
Please read again the code you wrote, or run it with
skb->alloc_cpubeing set to 45000 on a full DEBUG kernel.You are focusing on
skb->alloc_cpu==raw_smp_processor_id(), I am focusing on what happens
when this condition is not true.Sorry. My bad. I put the wrong order of '!cpu_online(cpu)' and 'cpu >=
nr_cpu_ids'. I didn't consider the out-of-bound issue. I should have done more checks :(The correct patch should be:
net/core/skbuff.c +0
-0
net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7007
Quoted 9 lines
net/core/skbuff.c +3
-3@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)unsigned int defer_max;bool kick; -if (WARN_ON_ONCE (cpu >= nr_cpu_ids) || - !cpu_online (cpu) || - cpu ==raw_smp_processor_id ()) { +if (cpu ==raw_smp_processor_id () || +WARN_ON_ONCE (cpu >= nr_cpu_ids) || + !cpu_online (cpu)) {
This one looks good to me.
Feel free to add
To your v2 before sending.
Quoted 8 lines
net/core/skbuff.c +0
-0nodefer:kfree_skb_napi_cache (skb);return ; }I will submit V2 tomorrow.
Thanks,
Jason
Thanks,
Olek
Quoted 88 lines · Alexander Lobakin
From: Jason Xing <[email protected]>
Date: Thu, 11 Apr 2024 15:31:23 +0800Quoted 71 lines · Eric Dumazet
Quoted 53 lines · Jason Xing
Quoted 44 lines · Eric Dumazet
Quoted 42 lines · Jason Xing
Quoted 27 lines
From: Jason Xing <[email protected]>
Normally, we don't face these two exceptions very often meanwhile we have some chance to meet the condition where the current cpu id is the same as
skb->alloc_cpu.One simple test that can help us see the frequency of this statement 'cpu ==
raw_smp_processor_id()':
1. running iperf -s and iperf -c [ip] -P [MAX CPU]
2. using BPF to captureskb_attempt_defer_free()I can see around 4% chance that happens to satisfy the statement. So moving this statement at the beginning can save some cycles in most cases.
Signed-off-by Jason Xing <[email protected]>--- net/core/skbuff.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)net/core/skbuff.c +0
-0@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)unsigned int defer_max;net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7010
Quoted 10 lines
net/core/skbuff.c +2
-2bool kick; -if (WARN_ON_ONCE (cpu >= nr_cpu_ids) || +if (cpu ==raw_smp_processor_id () || !cpu_online (cpu) || - cpu ==raw_smp_processor_id ()) { +WARN_ON_ONCE (cpu >= nr_cpu_ids)) { nodefer:kfree_skb_napi_cache (skb);return ; }Wrong patch.
Quoted 6 lines
Quoted 1 line
cpu_online(X)is undefined and might crash if X is out of bounds onCONFIG_SMP=yEven if
skb->alloc_cpuis larger thannr_cpu_ids, I don't know why the integer test statement could cause crashing the kernel. It's just a simple comparison. And if the statement is true,raw_smp_processor_id()can guarantee the validation, right?Quoted 6 lines
Please read again the code you wrote, or run it with
skb->alloc_cpubeing set to 45000 on a full DEBUG kernel.You are focusing on
skb->alloc_cpu==raw_smp_processor_id(), I am focusing on what happens
when this condition is not true.Sorry. My bad. I put the wrong order of '!cpu_online(cpu)' and 'cpu >=
nr_cpu_ids'. I didn't consider the out-of-bound issue. I should have done more checks :(The correct patch should be:
net/core/skbuff.c +0
-0net/core/skbuff.c void skb_attempt_defer_free(struct sk_buff *skb) · line 7007
Quoted 9 lines
net/core/skbuff.c +3
-3@@ -7002,9 +7002,9 @@ void skb_attempt_defer_free(struct sk_buff *skb)unsigned int defer_max;bool kick; -if (WARN_ON_ONCE (cpu >= nr_cpu_ids) || - !cpu_online (cpu) || - cpu ==raw_smp_processor_id ()) { +if (cpu ==raw_smp_processor_id () || +WARN_ON_ONCE (cpu >= nr_cpu_ids) || + !cpu_online (cpu)) {This one looks good to me.
Feel free to add
Quoted 3 lines
Reviewed-by Alexander Lobakin <[email protected]>To your v2 before sending.
Thanks! I will:)