Skip to contentSkip to conversation

WARNING: at include/net/sock.h:417 udp_lib_unhash

PATCH In review Tantilov, Emil S 6 participants 29 messages Jul 1, 2009 – Jul 17, 2009

Subsystems (8)
lore
Share & tools
lore
#0 lore raw

I see the following trace during netperf stress mixed UDP/TCP IPv4/6 traffic. This is on recent pulls from net-2.6 and net-next.

[45197.989163] ------------[ cut here ]------------
[45197.994309] WARNING: at include/net/sock.h:417 udp_lib_unhash+0x81/0xab()
[45197.994311] Hardware name: X7DA8
[45197.994314] Modules linked in: e1000 [last unloaded: e1000]
[45197.994326] Pid: 7110, comm: netserver Tainted: G        W  2.6.31-rc1-net-next-e1000-06250902 #8
[45197.994331] Call Trace:
[45197.994336]  [<ffffffff8135e0dc>] ? udp_lib_unhash+0x81/0xab
[45197.994344]  [<ffffffff8103cac9>] warn_slowpath_common+0x77/0x8f
[45197.994349]  [<ffffffff8103caf0>] warn_slowpath_null+0xf/0x11
[45197.994352]  [<ffffffff8135e0dc>] udp_lib_unhash+0x81/0xab
[45197.994357]  [<ffffffff81301acb>] sk_common_release+0x2f/0xb4
[45197.994364]  [<ffffffff813a0256>] udp_lib_close+0x9/0xb
[45197.994369]  [<ffffffff81364259>] inet_release+0x58/0x5f
[45197.994374]  [<ffffffff8138c8bd>] inet6_release+0x30/0x35
[45197.994383]  [<ffffffff812ff273>] sock_release+0x1a/0x6c
[45197.994386]  [<ffffffff812ff763>] sock_close+0x22/0x26
[45197.994392]  [<ffffffff810c69a0>] __fput+0xf0/0x18c
[45197.994395]  [<ffffffff810c6d00>] fput+0x15/0x19
[45197.994399]  [<ffffffff810c3c3e>] filp_close+0x5c/0x67
[45197.994404]  [<ffffffff810c3cc4>] sys_close+0x7b/0xb6
[45197.994412]  [<ffffffff8100baeb>] system_call_fastpath+0x16/0x1b
[45197.994418] ---[ end trace 5acab6fc0afdaaa3 ]---

Emil

#1 lore raw
Quoted 25 lines · Emil S

I see the following trace during netperf stress mixed UDP/TCP IPv4/6 traffic. This is on recent pulls from net-2.6 and net-next.

[45197.989163] ------------[ cut here ]------------
[45197.994309] WARNING: at include/net/sock.h:417 udp_lib_unhash+0x81/0xab()
[45197.994311] Hardware name: X7DA8
[45197.994314] Modules linked in: e1000 [last unloaded: e1000]
[45197.994326] Pid: 7110, comm: netserver Tainted: G        W  2.6.31-rc1-net-next-e1000-06250902 #8
[45197.994331] Call Trace:
[45197.994336]  [<ffffffff8135e0dc>] ? udp_lib_unhash+0x81/0xab
[45197.994344]  [<ffffffff8103cac9>] warn_slowpath_common+0x77/0x8f
[45197.994349]  [<ffffffff8103caf0>] warn_slowpath_null+0xf/0x11
[45197.994352]  [<ffffffff8135e0dc>] udp_lib_unhash+0x81/0xab
[45197.994357]  [<ffffffff81301acb>] sk_common_release+0x2f/0xb4
[45197.994364]  [<ffffffff813a0256>] udp_lib_close+0x9/0xb
[45197.994369]  [<ffffffff81364259>] inet_release+0x58/0x5f
[45197.994374]  [<ffffffff8138c8bd>] inet6_release+0x30/0x35
[45197.994383]  [<ffffffff812ff273>] sock_release+0x1a/0x6c
[45197.994386]  [<ffffffff812ff763>] sock_close+0x22/0x26
[45197.994392]  [<ffffffff810c69a0>] __fput+0xf0/0x18c
[45197.994395]  [<ffffffff810c6d00>] fput+0x15/0x19
[45197.994399]  [<ffffffff810c3c3e>] filp_close+0x5c/0x67
[45197.994404]  [<ffffffff810c3cc4>] sys_close+0x7b/0xb6
[45197.994412]  [<ffffffff8100baeb>] system_call_fastpath+0x16/0x1b
[45197.994418] ---[ end trace 5acab6fc0afdaaa3 ]---
Quoted 1 line

Emil--

Thanks for this report Emil.

I could not find a recent change in this area in last kernels. If struct sk is hashed (sk_hashed() true), then sk_refcnt was incremented in sk_nulls_add_node_rcu(), thus its value should be >= 2.

Maybe we have a missing memory barrier somewhere or a list corruption.

1) Could you try CONFIG_DEBUG_LIST=y ?

2) Could you give model of cpu, since it reminds me the ongoing discussion raised by Jiri Olsa.

CPU1 does an atomic_inc(&sk->sk_refcnt) : refcnt changes from 1 to 2 then CPU2 does an atomic_read(&sk->sk_refcnt) and reads 1 instead of 2

David, maybe this test is not safe and if we really want to do a check we need to use a stronger atomic function.

If you can reproduce this problem easily could you try following patch ?

Thank you

include/net/sock.h +6 -5
index 352f06b..96ab278 100644
@@ -393,8 +393,9 @@ static __inline__ int sk_del_node_init(struct sock *sk)
 
 	if (rc) {
 		/* paranoid for a while -acme */
-		WARN_ON(atomic_read(&sk->sk_refcnt) == 1);
-		__sock_put(sk);
+		int res = atomic_dec_return(&sk->sk_refcnt);
+
+		WARN_ON(res <= 0);
 	}
 	return rc;
 }
@@ -413,9 +414,9 @@ static __inline__ int sk_nulls_del_node_init_rcu(struct sock *sk)
 	int rc = __sk_nulls_del_node_init_rcu(sk);
 
 	if (rc) {
-		/* paranoid for a while -acme */
-		WARN_ON(atomic_read(&sk->sk_refcnt) == 1);
-		__sock_put(sk);
+		int res = atomic_dec_return(&sk->sk_refcnt);
+
+		WARN_ON(res <= 0);
 	}
 	return rc;
 }
#2 lore raw
Quoted 30 lines · Eric Dumazet
Quoted 25 lines · Emil S

I see the following trace during netperf stress mixed UDP/TCP IPv4/6 traffic. This is on recent pulls from net-2.6 and net-next.

[45197.989163] ------------[ cut here ]------------
[45197.994309] WARNING: at include/net/sock.h:417 udp_lib_unhash+0x81/0xab()
[45197.994311] Hardware name: X7DA8
[45197.994314] Modules linked in: e1000 [last unloaded: e1000]
[45197.994326] Pid: 7110, comm: netserver Tainted: G        W  2.6.31-rc1-net-next-e1000-06250902 #8
[45197.994331] Call Trace:
[45197.994336]  [<ffffffff8135e0dc>] ? udp_lib_unhash+0x81/0xab
[45197.994344]  [<ffffffff8103cac9>] warn_slowpath_common+0x77/0x8f
[45197.994349]  [<ffffffff8103caf0>] warn_slowpath_null+0xf/0x11
[45197.994352]  [<ffffffff8135e0dc>] udp_lib_unhash+0x81/0xab
[45197.994357]  [<ffffffff81301acb>] sk_common_release+0x2f/0xb4
[45197.994364]  [<ffffffff813a0256>] udp_lib_close+0x9/0xb
[45197.994369]  [<ffffffff81364259>] inet_release+0x58/0x5f
[45197.994374]  [<ffffffff8138c8bd>] inet6_release+0x30/0x35
[45197.994383]  [<ffffffff812ff273>] sock_release+0x1a/0x6c
[45197.994386]  [<ffffffff812ff763>] sock_close+0x22/0x26
[45197.994392]  [<ffffffff810c69a0>] __fput+0xf0/0x18c
[45197.994395]  [<ffffffff810c6d00>] fput+0x15/0x19
[45197.994399]  [<ffffffff810c3c3e>] filp_close+0x5c/0x67
[45197.994404]  [<ffffffff810c3cc4>] sys_close+0x7b/0xb6
[45197.994412]  [<ffffffff8100baeb>] system_call_fastpath+0x16/0x1b
[45197.994418] ---[ end trace 5acab6fc0afdaaa3 ]---
Quoted 1 line

Emil--

Thanks for this report Emil.

Quoted 7 lines

I could not find a recent change in this area in last kernels. If struct sk is hashed (sk_hashed() true), then sk_refcnt was incremented in sk_nulls_add_node_rcu(), thus its value should be >= 2.

Maybe we have a missing memory barrier somewhere or a list corruption.

1) Could you try CONFIG_DEBUG_LIST=y ?

I am running a test with this option now. Sorry for the delayed response, I was out last week.

Quoted 1 line

2) Could you give model of cpu, since it reminds me the ongoing discussion raised by Jiri Olsa.

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5450 @ 3.00GHz
stepping : 6
cpu MHz : 2999.790
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi flexpriority
bogomips : 5999.58
clflush size : 64

2 quad core Xeons, I only included the output from the first to reduce clutter.

Quoted 7 lines

CPU1 does an atomic_inc(&sk->sk_refcnt)  : refcnt changes from 1 to 2 then CPU2 does an atomic_read(&sk->sk_refcnt) and reads 1 instead of 2

David, maybe this test is not safe and if we really want to do a check we need to use a stronger atomic function.

If you can reproduce this problem easily could you try following patch ?

It varies from few minutes to hours, but it does reproduce consistently in my tests so far. I will try your patch once I manage to get a trace with CONFIG_DEBUG_LIST

Thanks,
Emil

#3 lore raw
Quoted 73 lines · Emil S Tantilov
Quoted 29 lines · Eric Dumazet
Quoted 25 lines · Emil S

I see the following trace during netperf stress mixed UDP/TCP IPv4/6 traffic. This is on recent pulls from net-2.6 and net-next.

[45197.989163] ------------[ cut here ]------------
[45197.994309] WARNING: at include/net/sock.h:417 udp_lib_unhash+0x81/0xab()
[45197.994311] Hardware name: X7DA8
[45197.994314] Modules linked in: e1000 [last unloaded: e1000]
[45197.994326] Pid: 7110, comm: netserver Tainted: G        W  2.6.31-rc1-net-next-e1000-06250902 #8
[45197.994331] Call Trace:
[45197.994336]  [<ffffffff8135e0dc>] ? udp_lib_unhash+0x81/0xab
[45197.994344]  [<ffffffff8103cac9>] warn_slowpath_common+0x77/0x8f
[45197.994349]  [<ffffffff8103caf0>] warn_slowpath_null+0xf/0x11
[45197.994352]  [<ffffffff8135e0dc>] udp_lib_unhash+0x81/0xab
[45197.994357]  [<ffffffff81301acb>] sk_common_release+0x2f/0xb4
[45197.994364]  [<ffffffff813a0256>] udp_lib_close+0x9/0xb
[45197.994369]  [<ffffffff81364259>] inet_release+0x58/0x5f
[45197.994374]  [<ffffffff8138c8bd>] inet6_release+0x30/0x35
[45197.994383]  [<ffffffff812ff273>] sock_release+0x1a/0x6c
[45197.994386]  [<ffffffff812ff763>] sock_close+0x22/0x26
[45197.994392]  [<ffffffff810c69a0>] __fput+0xf0/0x18c
[45197.994395]  [<ffffffff810c6d00>] fput+0x15/0x19
[45197.994399]  [<ffffffff810c3c3e>] filp_close+0x5c/0x67
[45197.994404]  [<ffffffff810c3cc4>] sys_close+0x7b/0xb6
[45197.994412]  [<ffffffff8100baeb>] system_call_fastpath+0x16/0x1b
[45197.994418] ---[ end trace 5acab6fc0afdaaa3 ]---
Quoted 1 line

Emil--

Thanks for this report Emil.

Quoted 7 lines

I could not find a recent change in this area in last kernels. If struct sk is hashed (sk_hashed() true), then sk_refcnt was incremented in sk_nulls_add_node_rcu(), thus its value should be >= 2.

Maybe we have a missing memory barrier somewhere or a list corruption.

1) Could you try CONFIG_DEBUG_LIST=y ?

I am running a test with this option now. Sorry for the delayed response, I was out last week.

Quoted 1 line

2) Could you give model of cpu, since it reminds me the ongoing discussion raised by Jiri Olsa.

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5450 @ 3.00GHz
stepping : 6
cpu MHz : 2999.790
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi flexpriority
bogomips : 5999.58
clflush size : 64

2 quad core Xeons, I only included the output from the first to reduce clutter.

Quoted 3 lines

CPU1 does an atomic_inc(&sk->sk_refcnt) : refcnt changes from 1 to 2 then CPU2 does an atomic_read(&sk->sk_refcnt) and reads 1 instead of 2

Quoted 9 lines
Quoted 4 lines

David, maybe this test is not safe and if we really want to do a check we need to use a stronger atomic function.

If you can reproduce this problem easily could you try following patch ?

It varies from few minutes to hours, but it does reproduce consistently in my tests so far. I will try your patch once I manage to get a trace with CONFIG_DEBUG_LIST

Eventually, could you also use following debug/quick&dirty patch ?

include/net/sock.h +2 -0
index 352f06b..548f822 100644
@@ -376,6 +376,7 @@ static __inline__ int __sk_del_node_init(struct sock *sk)
 
 static inline void sock_hold(struct sock *sk)
 {
+	WARN_ON(atomic_read(&sk->sk_refcnt) == 0);
 	atomic_inc(&sk->sk_refcnt);
 }
 
@@ -385,6 +386,7 @@ static inline void sock_hold(struct sock *sk)
 static inline void __sock_put(struct sock *sk)
 {
 	atomic_dec(&sk->sk_refcnt);
+	WARN_ON(atomic_read(&sk->sk_refcnt) == 0);
 }
 
 static __inline__ int sk_del_node_init(struct sock *sk)
#4 lore raw
Quoted 100 lines · Eric Dumazet
Quoted 72 lines · Emil S Tantilov
Quoted 29 lines · Eric Dumazet
Quoted 25 lines · Emil S

I see the following trace during netperf stress mixed UDP/TCP IPv4/6 traffic. This is on recent pulls from net-2.6 and net-next.

[45197.989163] ------------[ cut here ]------------
[45197.994309] WARNING: at include/net/sock.h:417 udp_lib_unhash+0x81/0xab()
[45197.994311] Hardware name: X7DA8
[45197.994314] Modules linked in: e1000 [last unloaded: e1000]
[45197.994326] Pid: 7110, comm: netserver Tainted: G        W  2.6.31-rc1-net-next-e1000-06250902 #8
[45197.994331] Call Trace:
[45197.994336]  [<ffffffff8135e0dc>] ? udp_lib_unhash+0x81/0xab
[45197.994344]  [<ffffffff8103cac9>] warn_slowpath_common+0x77/0x8f
[45197.994349]  [<ffffffff8103caf0>] warn_slowpath_null+0xf/0x11
[45197.994352]  [<ffffffff8135e0dc>] udp_lib_unhash+0x81/0xab
[45197.994357]  [<ffffffff81301acb>] sk_common_release+0x2f/0xb4
[45197.994364]  [<ffffffff813a0256>] udp_lib_close+0x9/0xb
[45197.994369]  [<ffffffff81364259>] inet_release+0x58/0x5f
[45197.994374]  [<ffffffff8138c8bd>] inet6_release+0x30/0x35
[45197.994383]  [<ffffffff812ff273>] sock_release+0x1a/0x6c
[45197.994386]  [<ffffffff812ff763>] sock_close+0x22/0x26
[45197.994392]  [<ffffffff810c69a0>] __fput+0xf0/0x18c
[45197.994395]  [<ffffffff810c6d00>] fput+0x15/0x19
[45197.994399]  [<ffffffff810c3c3e>] filp_close+0x5c/0x67
[45197.994404]  [<ffffffff810c3cc4>] sys_close+0x7b/0xb6
[45197.994412]  [<ffffffff8100baeb>] system_call_fastpath+0x16/0x1b
[45197.994418] ---[ end trace 5acab6fc0afdaaa3 ]---
Quoted 1 line

Emil--

Thanks for this report Emil.

Quoted 7 lines

I could not find a recent change in this area in last kernels. If struct sk is hashed (sk_hashed() true), then sk_refcnt was incremented in sk_nulls_add_node_rcu(), thus its value should be >= 2.

Maybe we have a missing memory barrier somewhere or a list corruption.

1) Could you try CONFIG_DEBUG_LIST=y ?

I am running a test with this option now. Sorry for the delayed response, I was out last week.

Quoted 1 line

2) Could you give model of cpu, since it reminds me the ongoing discussion raised by Jiri Olsa.

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5450 @ 3.00GHz
stepping : 6
cpu MHz : 2999.790
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi flexpriority
bogomips : 5999.58
clflush size : 64

2 quad core Xeons, I only included the output from the first to reduce clutter.

Quoted 3 lines

CPU1 does an atomic_inc(&sk->sk_refcnt) : refcnt changes from 1 to 2 then CPU2 does an atomic_read(&sk->sk_refcnt) and reads 1 instead of 2

Quoted 8 lines
Quoted 4 lines

David, maybe this test is not safe and if we really want to do a check we need to use a stronger atomic function.

If you can reproduce this problem easily could you try following patch ?

It varies from few minutes to hours, but it does reproduce consistently in my tests so far. I will try your patch once I manage to get a trace with CONFIG_DEBUG_LIST

Eventually, could you also use following debug/quick&dirty patch ?

include/net/sock.h +1 -0
index 352f06b..548f822 100644
@@ -376,6 +376,7 @@ static __inline__ int __sk_del_node_init(struct sock *sk)
 
 static inline void sock_hold(struct sock *sk)
 {
+	WARN_ON(atomic_read(&sk->sk_refcnt) == 0);
 	atomic_inc(&sk->sk_refcnt);
 }
 

include/net/sock.h static inline void sock_hold(struct sock *sk) · line 392

Quoted 7 lines
include/net/sock.h +1 -0
@@ -385,6 +386,7 @@ static inline void sock_hold(struct sock *sk)
 static inline void __sock_put(struct sock *sk)
 {
 	atomic_dec(&sk->sk_refcnt);
+	WARN_ON(atomic_read(&sk->sk_refcnt) == 0);
 }
 
 static __inline__ int sk_del_node_init(struct sock *sk)

Hold on, I found the problem and will submit a patch (after testing it) in following hours

Thanks

#5 lore raw

PATCH 1/1 net: sk_alloc() should not blindly overwrite memory

Quoted 4 lines · Eric Dumazet

Hold on, I found the problem and will submit a patch (after testing it) in following hours

David, while looking at Emil case, I found following problem.

I am not sure this bug is responsible for the BUG hit by Emil, but it seems a candidate for stable as well...

Thanks

[PATCH] net: sk_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Current sk_alloc() implementation doesnt respect this hypothesis, calling kmem_alloc with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

---
 include/net/sock.h            |   24 +++++++++++-------
 net/core/sock.c               |   41 ++++++++++++++++++++++++++++----
 net/ipv4/inet_timewait_sock.c |    2 -
 3 files changed, 52 insertions(+), 15 deletions(-)
include/net/sock.h +15 -9
index 352f06b..3f6284e 100644
@@ -103,15 +103,15 @@ struct net;
 
 /**
  *	struct sock_common - minimal network layer representation of sockets
+ *	@skc_node: main hash linkage for various protocol lookup tables
+ *	@skc_nulls_node: main hash linkage for UDP/UDP-Lite protocol
+ *	@skc_refcnt: reference count
+ *	@skc_hash: hash value used with various protocol lookup tables
  *	@skc_family: network address family
  *	@skc_state: Connection state
  *	@skc_reuse: %SO_REUSEADDR setting
  *	@skc_bound_dev_if: bound device index if != 0
- *	@skc_node: main hash linkage for various protocol lookup tables
- *	@skc_nulls_node: main hash linkage for UDP/UDP-Lite protocol
  *	@skc_bind_node: bind hash linkage for various protocol lookup tables
- *	@skc_refcnt: reference count
- *	@skc_hash: hash value used with various protocol lookup tables
  *	@skc_prot: protocol handlers inside a network family
  *	@skc_net: reference to the network namespace of this socket
  *
@@ -119,17 +119,23 @@ struct net;
  *	for struct sock and struct inet_timewait_sock.
  */
 struct sock_common {
-	unsigned short		skc_family;
-	volatile unsigned char	skc_state;
-	unsigned char		skc_reuse;
-	int			skc_bound_dev_if;
+	/*
+	 * WARNING : skc_node, skc_refcnt must be first elements of this struct
+	 * (sk_prot_alloc() should not clear skc_node.next & skc_refcnt because
+	 *  of RCU lookups)
+	 */
 	union {
 		struct hlist_node	skc_node;
 		struct hlist_nulls_node skc_nulls_node;
 	};
-	struct hlist_node	skc_bind_node;
 	atomic_t		skc_refcnt;
+
 	unsigned int		skc_hash;
+	unsigned short		skc_family;
+	volatile unsigned char	skc_state;
+	unsigned char		skc_reuse;
+	int			skc_bound_dev_if;
+	struct hlist_node	skc_bind_node;
 	struct proto		*skc_prot;
 #ifdef CONFIG_NET_NS
 	struct net	 	*skc_net;
net/core/sock.c +36 -5

77 lines not on this page · raw message

net/ipv4/inet_timewait_sock.c +1 -1
index 61283f9..a2997df 100644
@@ -139,7 +139,7 @@ struct inet_timewait_sock *inet_twsk_alloc(const struct sock *sk, const int stat
 		tw->tw_transparent  = inet->transparent;
 		tw->tw_prot	    = sk->sk_prot_creator;
 		twsk_net_set(tw, hold_net(sock_net(sk)));
-		atomic_set(&tw->tw_refcnt, 1);
+		atomic_inc(&tw->tw_refcnt);
 		inet_twsk_dead_node_init(tw);
 		__module_get(tw->tw_prot->owner);
 	}
#6 lore raw

RE: [PATCH] net: sk_alloc() should not blindly overwrite memory

Quoted 181 lines · Eric Dumazet
Quoted 5 lines · Eric Dumazet

Hold on, I found the problem and will submit a patch (after testing it) in following hours

David, while looking at Emil case, I found following problem.

I am not sure this bug is responsible for the BUG hit by Emil, but it seems a candidate for stable as well...

Thanks

[PATCH] net: sk_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Current sk_alloc() implementation doesnt respect this hypothesis, calling kmem_alloc with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

---
 include/net/sock.h            |   24 +++++++++++-------
 net/core/sock.c               |   41 ++++++++++++++++++++++++++++----
 net/ipv4/inet_timewait_sock.c |    2 -
 3 files changed, 52 insertions(+), 15 deletions(-)
include/net/sock.h +0 -0
index 352f06b..3f6284e 100644
@@ -103,15 +103,15 @@ struct net;
 
 /**
  *	struct sock_common - minimal network layer representation of

sockets + * @skc_node: main hash linkage for various protocol lookup
tables + * @skc_nulls_node: main hash linkage for UDP/UDP-Lite
protocol + * @skc_refcnt: reference count
+ * @skc_hash: hash value used with various protocol lookup tables
* @skc_family: network address family
* @skc_state: Connection state
* @skc_reuse: %SO_REUSEADDR setting
* @skc_bound_dev_if: bound device index if != 0
- * @skc_node: main hash linkage for various protocol lookup tables
- * @skc_nulls_node: main hash linkage for UDP/UDP-Lite protocol
* @skc_bind_node: bind hash linkage for various protocol lookup
tables
- * @skc_refcnt: reference count
- * @skc_hash: hash value used with various protocol lookup tables
* @skc_prot: protocol handlers inside a network family
* @skc_net: reference to the network namespace of this socket
*

include/net/sock.h +2 -4
@@ -119,17 +119,23 @@ struct net;
  *	for struct sock and struct inet_timewait_sock.
  */
 struct sock_common {
-	unsigned short		skc_family;
-	volatile unsigned char	skc_state;
-	unsigned char		skc_reuse;
-	int			skc_bound_dev_if;
+	/*
+	 * WARNING : skc_node, skc_refcnt must be first elements of this

struct + * (sk_prot_alloc() should not clear skc_node.next &
skc_refcnt because + * of RCU lookups)

include/net/sock.h +7 -1
+	 */
 	union {
 		struct hlist_node	skc_node;
 		struct hlist_nulls_node skc_nulls_node;
 	};
-	struct hlist_node	skc_bind_node;
 	atomic_t		skc_refcnt;
+
 	unsigned int		skc_hash;
+	unsigned short		skc_family;
+	volatile unsigned char	skc_state;
+	unsigned char		skc_reuse;
+	int			skc_bound_dev_if;
+	struct hlist_node	skc_bind_node;
 	struct proto		*skc_prot;
 #ifdef CONFIG_NET_NS
 	struct net	 	*skc_net;
net/core/sock.c +19 -2
index b0ba569..daadcc5 100644
@@ -994,8 +994,25 @@ struct sock *sk_alloc(struct net *net, int
 family, gfp_t priority, {
 	struct sock *sk;
 
-	sk = sk_prot_alloc(prot, priority | __GFP_ZERO, family);
+	sk = sk_prot_alloc(prot, priority, family);
 	if (sk) {
+		/*
+		 * caches using SLAB_DESTROY_BY_RCU should let sk_node
+		 * and sk_refcnt untouched.
+		 *
+		 * Following makes sense only if sk first fields are
+		 * in following order : sk_node, refcnt, sk_hash
+		 */
+		BUILD_BUG_ON(offsetof(struct sock, sk_node) != 0);
+		BUILD_BUG_ON(offsetof(struct sock, sk_refcnt) !=
+			     sizeof(sk->sk_node));
+		BUILD_BUG_ON(offsetof(struct sock, sk_hash) !=
+			     sizeof(sk->sk_node) + sizeof(sk->sk_refcnt));
+		if (prot->slab_flags & SLAB_DESTROY_BY_RCU)
+			memset(&sk->sk_hash, 0,
+			       prot->obj_size - offsetof(struct sock, sk_hash));
+		else
+			memset(sk, 0, prot->obj_size);
 		sk->sk_family = family;
 		/*
 		 * See comment in struct sock definition to understand
@@ -1125,7 +1142,7 @@ struct sock *sk_clone(const struct sock *sk,
 const gfp_t priority) 
 
 		newsk->sk_err	   = 0;
 		newsk->sk_priority = 0;
-		atomic_set(&newsk->sk_refcnt, 2);
+		atomic_add(2, &newsk->sk_refcnt);
 
 		/*
net/core/sock.c +1 -1
 		 * Increment the counter in the same struct proto as the master
@@ -1840,7 +1857,7 @@ void sock_init_data(struct socket *sock, struct
 sock *sk) 
 
 	sk->sk_stamp = ktime_set(-1L, 0);
 
-	atomic_set(&sk->sk_refcnt, 1);
+	atomic_inc(&sk->sk_refcnt);
 	atomic_set(&sk->sk_wmem_alloc, 1);
 	atomic_set(&sk->sk_drops, 0);

}

net/core/sock.c +2 -0
@@ -2140,12 +2157,25 @@ static inline void release_proto_idx(struct
 proto *prot) }
 #endif
 
+/*
+ * We need to initialize skc->skc_refcnt to 0, and

skc->skc_node.pprev to NULL + * but only on newly allocated objects (check sk_prot_alloc()) + */

net/core/sock.c +12 -2
+static void sk_init_once(void *arg)
+{
+	struct sock_common *skc = arg;
+
+	atomic_set(&skc->skc_refcnt, 0);
+	skc->skc_node.pprev = NULL;
+}
+
 int proto_register(struct proto *prot, int alloc_slab)
 {
 	if (alloc_slab) {
 		prot->slab = kmem_cache_create(prot->name, prot->obj_size, 0,
 					SLAB_HWCACHE_ALIGN | prot->slab_flags,
-					NULL);
+					prot->slab_flags & SLAB_DESTROY_BY_RCU ?
+					    sk_init_once : NULL);
 
 		if (prot->slab == NULL) {
 			printk(KERN_CRIT "%s: Can't create sock SLAB cache!\n",
@@ -2187,7 +2217,8 @@ int proto_register(struct proto *prot, int
 						  alloc_slab) 0,
 						  SLAB_HWCACHE_ALIGN |
 							prot->slab_flags,
-						  NULL);
+						  prot->slab_flags & SLAB_DESTROY_BY_RCU ?
+							sk_init_once : NULL);
 			if (prot->twsk_prot->twsk_slab == NULL)
 				goto out_free_timewait_sock_slab_name;
 		}
net/ipv4/inet_timewait_sock.c +0 -0
index 61283f9..a2997df 100644

net/ipv4/inet_timewait_sock.c struct inet_timewait_sock *inet_twsk_alloc(const · line 145

Quoted 8 lines
net/ipv4/inet_timewait_sock.c +1 -1
@@ -139,7 +139,7 @@ struct inet_timewait_sock *inet_twsk_alloc(const
 		struct sock *sk, const int stat tw->tw_transparent  =
 		inet->transparent; tw->tw_prot	    = sk->sk_prot_creator;
 		twsk_net_set(tw, hold_net(sock_net(sk)));
-		atomic_set(&tw->tw_refcnt, 1);
+		atomic_inc(&tw->tw_refcnt);
 		inet_twsk_dead_node_init(tw);
 		__module_get(tw->tw_prot->owner);
 	}

Eric,

With this patch applied, I get panic on boot:

[    5.334653] BUG: unable to handle kernel NULL pointer dereference at 0000000000000010
[    5.344225] IP: [<ffffffff811ed364>] selinux_socket_post_create+0x78/0x95
[    5.352263] PGD 0 
[    5.354952] Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
[    5.360599] last sysfs file: 
[    5.364292] CPU 0 
[    5.366985] Modules linked in:
[    5.370836] Pid: 1, comm: swapper Not tainted 2.6.31-rc1-net-2.6-igb-ed-patch-07071123 #2 S5520HC
[    5.381465] RIP: 0010:[<ffffffff811ed364>]  [<ffffffff811ed364>] selinux_socket_post_create+0x78/0x95
[    5.392551] RSP: 0018:ffff8801ef0a5d60  EFLAGS: 00010286
[    5.398871] RAX: 0000000000000001 RBX: ffff88036e5ace80 RCX: 0000000000000006
[    5.407227] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000002
[    5.415587] RBP: ffff8801ef0a5d80 R08: 0000000000000001 R09: 0000000000000000
[    5.423945] R10: ffff88036e000000 R11: ffff8801ef0a5c20 R12: ffff88036ec09c80
[    5.432300] R13: 0000000000000002 R14: 0000000000000002 R15: 0000000000000003
[    5.440662] FS:  0000000000000000(0000) GS:ffffc90000000000(0000) knlGS:0000000000000000
[    5.450386] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
[    5.457189] CR2: 0000000000000010 CR3: 0000000001001000 CR4: 00000000000006b0
[    5.465545] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    5.473908] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[    5.482267] Process swapper (pid: 1, threadinfo ffff8801ef0a4000, task ffff88036f040000)
[    5.491994] Stack:
[    5.494606]  01ffffff816d0040 00000000ffffff9f ffffffff817244a0 ffff88036ec09c80
[    5.503262] <0> ffff8801ef0a5d90 ffffffff811e5ef0 ffff8801ef0a5df0 ffffffff8138c041
[    5.512853] <0> ffffffff8138bf93 00000001ef0a5de0 ffff8801ef0a5e18 0000000681478fb3
[    5.523046] Call Trace:
[    5.526158]  [<ffffffff811e5ef0>] security_socket_post_create+0x11/0x13
[    5.533932]  [<ffffffff8138c041>] __sock_create+0x1a4/0x1ff
[    5.540541]  [<ffffffff8138bf93>] ? __sock_create+0xf6/0x1ff
[    5.547243]  [<ffffffff8184028c>] ? inet_init+0x0/0x204
[    5.553463]  [<ffffffff8138c0bb>] sock_create_kern+0x1f/0x21
[    5.560162]  [<ffffffff813edd5a>] inet_ctl_sock_create+0x29/0x5d
[    5.567255]  [<ffffffff8184028c>] ? inet_init+0x0/0x204
[    5.573473]  [<ffffffff8183fb60>] tcp_sk_init+0x25/0x27
[    5.579687]  [<ffffffff81396716>] register_pernet_operations+0x18/0x1a
[    5.587365]  [<ffffffff81396824>] register_pernet_subsys+0x29/0x3d
[    5.594657]  [<ffffffff8184028c>] ? inet_init+0x0/0x204
[    5.600882]  [<ffffffff8183fb27>] tcp_v4_init+0x1c/0x30
[    5.607106]  [<ffffffff818403cd>] inet_init+0x141/0x204
[    5.613333]  [<ffffffff8100905c>] do_one_initcall+0x56/0x130
[    5.620038]  [<ffffffff810a2a72>] ? register_irq_proc+0xae/0xca
[    5.627041]  [<ffffffff81130000>] ? d_add+0x16/0x1d
[    5.632875]  [<ffffffff81811915>] kernel_init+0x15e/0x1b4
[    5.639294]  [<ffffffff81028b9a>] child_rip+0xa/0x20
[    5.645214]  [<ffffffff8102857c>] ? restore_args+0x0/0x30
[    5.651622]  [<ffffffff818117b7>] ? kernel_init+0x0/0x1b4
[    5.658033]  [<ffffffff81028b90>] ? child_rip+0x0/0x20
[    5.664158] Code: ca 44 89 ef e8 d8 b6 ff ff c6 43 22 01 66 89 43 20 31 c0 49 8b 54 24 60 48 85 d2 74 22 8b 43 1c 48 8b 92 a8 03 00 00 41 0f b7 f5 <89> 42 10 8b 43 20 66 89 42 18 49 8b 7c 24 60 e8 de 4a 00 00 41 
[    5.690779] RIP  [<ffffffff811ed364>] selinux_socket_post_create+0x78/0x95
[    5.698921]  RSP <ffff8801ef0a5d60>
[    5.703202] CR2: 0000000000000010
[    5.707298] ---[ end trace a7919e7f17c0a725 ]---
[    5.712848] swapper used greatest stack depth: 4680 bytes left
[    5.727678] Kernel panic - not syncing: Attempted to kill init!
[    5.734672] Pid: 1, comm: swapper Tainted: G      D    2.6.31-rc1-net-2.6-igb-ed-patch-07071123 #2
[    5.745375] Call Trace:
[    5.748485]  [<ffffffff81057cfd>] panic+0xdb/0x190
[    5.754217]  [<ffffffff8105ae3b>] ? do_exit+0x35f/0x6d8
[    5.760439]  [<ffffffff8105adeb>] ? do_exit+0x30f/0x6d8
[    5.766659]  [<ffffffff8105adeb>] ? do_exit+0x30f/0x6d8
[    5.772875]  [<ffffffff8107bdbb>] ? trace_hardirqs_on+0xd/0xf
[    5.779688]  [<ffffffff8147a295>] ? _write_unlock_irq+0x2b/0x31
[    5.786690]  [<ffffffff8105ab55>] do_exit+0x79/0x6d8
[    5.792622]  [<ffffffff8147b07d>] oops_end+0xb2/0xba
[    5.798555]  [<ffffffff810407b4>] no_context+0x1ef/0x1fe
[    5.804875]  [<ffffffff8107ba3b>] ? mark_lock+0x22/0x1fb
[    5.811196]  [<ffffffff8107af82>] ? save_trace+0x3f/0x96
[    5.817516]  [<ffffffff81040a03>] __bad_area_nosemaphore+0x186/0x1a9
[    5.825009]  [<ffffffff813e5bbf>] ? raw_hash_sk+0x2a/0x73
[    5.831421]  [<ffffffff8147c354>] ? do_page_fault+0xbd/0x22e
[    5.838128]  [<ffffffff81040a9c>] bad_area_nosemaphore+0xe/0x10
[    5.845132]  [<ffffffff8147c3b6>] do_page_fault+0x11f/0x22e
[    5.851747]  [<ffffffff8147a4ef>] page_fault+0x1f/0x30
[    5.857871]  [<ffffffff811ed364>] ? selinux_socket_post_create+0x78/0x95
[    5.865754]  [<ffffffff811e5ef0>] security_socket_post_create+0x11/0x13
[    5.873536]  [<ffffffff8138c041>] __sock_create+0x1a4/0x1ff
[    5.880149]  [<ffffffff8138bf93>] ? __sock_create+0xf6/0x1ff
[    5.886859]  [<ffffffff8184028c>] ? inet_init+0x0/0x204
[    5.893072]  [<ffffffff8138c0bb>] sock_create_kern+0x1f/0x21
[    5.899774]  [<ffffffff813edd5a>] inet_ctl_sock_create+0x29/0x5d
[    5.906863]  [<ffffffff8184028c>] ? inet_init+0x0/0x204
[    5.913080]  [<ffffffff8183fb60>] tcp_sk_init+0x25/0x27
[    5.919299]  [<ffffffff81396716>] register_pernet_operations+0x18/0x1a
[    5.926986]  [<ffffffff81396824>] register_pernet_subsys+0x29/0x3d
[    5.934274]  [<ffffffff8184028c>] ? inet_init+0x0/0x204
[    5.940504]  [<ffffffff8183fb27>] tcp_v4_init+0x1c/0x30
[    5.946730]  [<ffffffff818403cd>] inet_init+0x141/0x204
[    5.952957]  [<ffffffff8100905c>] do_one_initcall+0x56/0x130
[    5.959668]  [<ffffffff810a2a72>] ? register_irq_proc+0xae/0xca
[    5.966666]  [<ffffffff81130000>] ? d_add+0x16/0x1d
[    5.972495]  [<ffffffff81811915>] kernel_init+0x15e/0x1b4
[    5.978909]  [<ffffffff81028b9a>] child_rip+0xa/0x20
[    5.984845]  [<ffffffff8102857c>] ? restore_args+0x0/0x30
[    5.991266]  [<ffffffff818117b7>] ? kernel_init+0x0/0x1b4
[    5.997686]  [<ffffffff81028b90>] ? child_rip+0x0/0x20

Emil

#7 lore raw

PATCH 1/1 net: sk_prot_alloc() should not blindly overwrite memory

Quoted 24 lines · Emil S
Quoted 23 lines · Eric Dumazet
Quoted 4 lines · Eric Dumazet

Hold on, I found the problem and will submit a patch (after testing it) in following hours

David, while looking at Emil case, I found following problem.

I am not sure this bug is responsible for the BUG hit by Emil, but it seems a candidate for stable as well...

Thanks

[PATCH] net: sk_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Current sk_alloc() implementation doesnt respect this hypothesis, calling kmem_alloc with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

Quoted 6 lines
Quoted 2 lines

Eric,

With this patch applied, I get panic on boot:

Oops, this stuff has to be done inside sk_prot_alloc() or breaks selinux

Thanks for testing Emil, here is a second try !

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

---
 include/net/sock.h            |   24 +++++++++------
 net/core/sock.c               |   49 ++++++++++++++++++++++++++++----
 net/ipv4/inet_timewait_sock.c |    2 -
 3 files changed, 59 insertions(+), 16 deletions(-)
include/net/sock.h +15 -9
index 352f06b..3f6284e 100644
@@ -103,15 +103,15 @@ struct net;
 
 /**
  *	struct sock_common - minimal network layer representation of sockets
+ *	@skc_node: main hash linkage for various protocol lookup tables
+ *	@skc_nulls_node: main hash linkage for UDP/UDP-Lite protocol
+ *	@skc_refcnt: reference count
+ *	@skc_hash: hash value used with various protocol lookup tables
  *	@skc_family: network address family
  *	@skc_state: Connection state
  *	@skc_reuse: %SO_REUSEADDR setting
  *	@skc_bound_dev_if: bound device index if != 0
- *	@skc_node: main hash linkage for various protocol lookup tables
- *	@skc_nulls_node: main hash linkage for UDP/UDP-Lite protocol
  *	@skc_bind_node: bind hash linkage for various protocol lookup tables
- *	@skc_refcnt: reference count
- *	@skc_hash: hash value used with various protocol lookup tables
  *	@skc_prot: protocol handlers inside a network family
  *	@skc_net: reference to the network namespace of this socket
  *
@@ -119,17 +119,23 @@ struct net;
  *	for struct sock and struct inet_timewait_sock.
  */
 struct sock_common {
-	unsigned short		skc_family;
-	volatile unsigned char	skc_state;
-	unsigned char		skc_reuse;
-	int			skc_bound_dev_if;
+	/*
+	 * WARNING : skc_node, skc_refcnt must be first elements of this struct
+	 * (sk_prot_alloc() should not clear skc_node.next & skc_refcnt because
+	 *  of RCU lookups)
+	 */
 	union {
 		struct hlist_node	skc_node;
 		struct hlist_nulls_node skc_nulls_node;
 	};
-	struct hlist_node	skc_bind_node;
 	atomic_t		skc_refcnt;
+
 	unsigned int		skc_hash;
+	unsigned short		skc_family;
+	volatile unsigned char	skc_state;
+	unsigned char		skc_reuse;
+	int			skc_bound_dev_if;
+	struct hlist_node	skc_bind_node;
 	struct proto		*skc_prot;
 #ifdef CONFIG_NET_NS
 	struct net	 	*skc_net;
net/core/sock.c +43 -6

84 lines not on this page · raw message

net/ipv4/inet_timewait_sock.c +1 -1
index 61283f9..a2997df 100644
@@ -139,7 +139,7 @@ struct inet_timewait_sock *inet_twsk_alloc(const struct sock *sk, const int stat
 		tw->tw_transparent  = inet->transparent;
 		tw->tw_prot	    = sk->sk_prot_creator;
 		twsk_net_set(tw, hold_net(sock_net(sk)));
-		atomic_set(&tw->tw_refcnt, 1);
+		atomic_inc(&tw->tw_refcnt);
 		inet_twsk_dead_node_init(tw);
 		__module_get(tw->tw_prot->owner);
 	}
maintainer
#8 lore raw

Re: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

From: Eric Dumazet <[email protected]>
Date: Wed, 08 Jul 2009 00:33:29 +0200

Quoted 5 lines

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

Quoted 8 lines

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

I've applied this but will wait for some more testing before I push it out for real to kernel.org

#9 lore raw

Re: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Quoted 14 lines · David Miller

From: Eric Dumazet <[email protected]>
Date: Wed, 08 Jul 2009 00:33:29 +0200

Quoted 11 lines

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

Quoted 5 lines
Quoted 2 lines

I've applied this but will wait for some more testing before I push it out for real to kernel.org

Thanks David

I forgot to CC Paul and Patrick, so I'll ask them to look at this patch.

Patrick, a similar fix is needed in conntrack as well, we currently uses "ct = kmem_cache_zalloc(nf_conntrack_cachep, gfp);" and thus
overwrite struct hlist_nulls_node hnnode; contained
in "struct nf_conntrack_tuple_hash", while lockless readers still potentialy need them. Setting hnnode.next to NULL is dangerous since last bit is not set (not a nulls value), a reader could try to dereference this NULL pointer and trap.

Here is the patch again so that Paul & Patrick can comment on it.

I am not sure about the refcnt thing (blindly setting it to 0 again should be OK in fact, since no reader should/can to the atomic_inc_if_not_zero on it), but the nulls.next thing is problematic.

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

---
 include/net/sock.h            |   24 +++++++++------
 net/core/sock.c               |   49 ++++++++++++++++++++++++++++----
 net/ipv4/inet_timewait_sock.c |    2 -
 3 files changed, 59 insertions(+), 16 deletions(-)
include/net/sock.h +15 -9
index 352f06b..3f6284e 100644
@@ -103,15 +103,15 @@ struct net;
 
 /**
  *	struct sock_common - minimal network layer representation of sockets
+ *	@skc_node: main hash linkage for various protocol lookup tables
+ *	@skc_nulls_node: main hash linkage for UDP/UDP-Lite protocol
+ *	@skc_refcnt: reference count
+ *	@skc_hash: hash value used with various protocol lookup tables
  *	@skc_family: network address family
  *	@skc_state: Connection state
  *	@skc_reuse: %SO_REUSEADDR setting
  *	@skc_bound_dev_if: bound device index if != 0
- *	@skc_node: main hash linkage for various protocol lookup tables
- *	@skc_nulls_node: main hash linkage for UDP/UDP-Lite protocol
  *	@skc_bind_node: bind hash linkage for various protocol lookup tables
- *	@skc_refcnt: reference count
- *	@skc_hash: hash value used with various protocol lookup tables
  *	@skc_prot: protocol handlers inside a network family
  *	@skc_net: reference to the network namespace of this socket
  *
@@ -119,17 +119,23 @@ struct net;
  *	for struct sock and struct inet_timewait_sock.
  */
 struct sock_common {
-	unsigned short		skc_family;
-	volatile unsigned char	skc_state;
-	unsigned char		skc_reuse;
-	int			skc_bound_dev_if;
+	/*
+	 * WARNING : skc_node, skc_refcnt must be first elements of this struct
+	 * (sk_prot_alloc() should not clear skc_node.next & skc_refcnt because
+	 *  of RCU lookups)
+	 */
 	union {
 		struct hlist_node	skc_node;
 		struct hlist_nulls_node skc_nulls_node;
 	};
-	struct hlist_node	skc_bind_node;
 	atomic_t		skc_refcnt;
+
 	unsigned int		skc_hash;
+	unsigned short		skc_family;
+	volatile unsigned char	skc_state;
+	unsigned char		skc_reuse;
+	int			skc_bound_dev_if;
+	struct hlist_node	skc_bind_node;
 	struct proto		*skc_prot;
 #ifdef CONFIG_NET_NS
 	struct net	 	*skc_net;
net/core/sock.c +43 -6

84 lines not on this page · raw message

net/ipv4/inet_timewait_sock.c +1 -1
index 61283f9..a2997df 100644
@@ -139,7 +139,7 @@ struct inet_timewait_sock *inet_twsk_alloc(const struct sock *sk, const int stat
 		tw->tw_transparent  = inet->transparent;
 		tw->tw_prot	    = sk->sk_prot_creator;
 		twsk_net_set(tw, hold_net(sock_net(sk)));
-		atomic_set(&tw->tw_refcnt, 1);
+		atomic_inc(&tw->tw_refcnt);
 		inet_twsk_dead_node_init(tw);
 		__module_get(tw->tw_prot->owner);
 	}
#10 lore raw

Re: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Quoted 33 lines · Eric Dumazet
Quoted 10 lines · David Miller

From: Eric Dumazet <[email protected]>
Date: Wed, 08 Jul 2009 00:33:29 +0200

Quoted 7 lines

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Quoted 8 lines
Quoted 6 lines

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

I've applied this but will wait for some more testing before I push it out for real to kernel.org

Thanks David

I forgot to CC Paul and Patrick, so I'll ask them to look at this patch.

Patrick, a similar fix is needed in conntrack as well, we currently uses "ct = kmem_cache_zalloc(nf_conntrack_cachep, gfp);" and thus
overwrite struct hlist_nulls_node hnnode; contained
in "struct nf_conntrack_tuple_hash", while lockless readers still potentialy need them. Setting hnnode.next to NULL is dangerous since last bit is not set (not a nulls value), a reader could try to dereference this NULL pointer and trap.

Quoted 5 lines

Here is the patch again so that Paul & Patrick can comment on it.

I am not sure about the refcnt thing (blindly setting it to 0 again should be OK in fact, since no reader should/can to the atomic_inc_if_not_zero on it), but the nulls.next thing is problematic.

Here is an updated and much simpler patch, taking care of sk_node.next being not set to 0

This patch applies to >= 2.6.29 kernels

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code correctness depends on sk->sk_nulls_node.next being always valid. A NULL value is not allowed as it might fault a lockless reader.

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO. Just call memset() around the forbidden field.

---

net/core/sock.c +17 -2
index b0ba569..7b87ec0 100644
@@ -939,8 +939,23 @@ static struct sock *sk_prot_alloc(struct proto *prot, gfp_t priority,
 	struct kmem_cache *slab;
 
 	slab = prot->slab;
-	if (slab != NULL)
-		sk = kmem_cache_alloc(slab, priority);
+	if (slab != NULL) {
+		sk = kmem_cache_alloc(slab, priority & ~__GFP_ZERO);
+		if (!sk)
+			return sk;
+		if (priority & __GFP_ZERO) {
+			/*
+			 * caches using SLAB_DESTROY_BY_RCU should let
+			 * sk_node.next un-modified. Special care is taken
+			 * when initializing object to zero.
+			 */
+			if (offsetof(struct sock, sk_node.next) != 0)
+				memset(sk, 0, offsetof(struct sock, sk_node.next));
+			memset(&sk->sk_node.pprev, 0,
+			       prot->obj_size - offsetof(struct sock,
+							 sk_node.pprev));
+		}
+	}
 	else
 		sk = kmalloc(prot->obj_size, priority);
 
#11 lore raw

Re: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Quoted 36 lines · Eric Dumazet
Quoted 35 lines · Eric Dumazet
Quoted 10 lines · David Miller

From: Eric Dumazet <[email protected]>
Date: Wed, 08 Jul 2009 00:33:29 +0200

Quoted 7 lines

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Quoted 8 lines
Quoted 6 lines

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

I've applied this but will wait for some more testing before I push it out for real to kernel.org

Thanks David

I forgot to CC Paul and Patrick, so I'll ask them to look at this patch.

Patrick, a similar fix is needed in conntrack as well, we currently uses "ct = kmem_cache_zalloc(nf_conntrack_cachep, gfp);" and thus
overwrite struct hlist_nulls_node hnnode; contained
in "struct nf_conntrack_tuple_hash", while lockless readers still potentialy need them. Setting hnnode.next to NULL is dangerous since last bit is not set (not a nulls value), a reader could try to dereference this NULL pointer and trap.

Here is the patch again so that Paul & Patrick can comment on it.

Quoted 7 lines
Quoted 3 lines

I am not sure about the refcnt thing (blindly setting it to 0 again should be OK in fact, since no reader should/can to the atomic_inc_if_not_zero on it), but the nulls.next thing is problematic.

Here is an updated and much simpler patch, taking care of sk_node.next being not set to 0

This patch applies to >= 2.6.29 kernels

Does this one also need the rearrangement of struct elements in the earlier patch? (And apologies about being slow to get to that one.)

Thanx, Paul

Quoted 42 lines

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code correctness depends on sk->sk_nulls_node.next being always valid. A NULL value is not allowed as it might fault a lockless reader.

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO. Just call memset() around the forbidden field.

---

net/core/sock.c +17 -2
index b0ba569..7b87ec0 100644
@@ -939,8 +939,23 @@ static struct sock *sk_prot_alloc(struct proto *prot, gfp_t priority,
 	struct kmem_cache *slab;
 
 	slab = prot->slab;
-	if (slab != NULL)
-		sk = kmem_cache_alloc(slab, priority);
+	if (slab != NULL) {
+		sk = kmem_cache_alloc(slab, priority & ~__GFP_ZERO);
+		if (!sk)
+			return sk;
+		if (priority & __GFP_ZERO) {
+			/*
+			 * caches using SLAB_DESTROY_BY_RCU should let
+			 * sk_node.next un-modified. Special care is taken
+			 * when initializing object to zero.
+			 */
+			if (offsetof(struct sock, sk_node.next) != 0)
+				memset(sk, 0, offsetof(struct sock, sk_node.next));
+			memset(&sk->sk_node.pprev, 0,
+			       prot->obj_size - offsetof(struct sock,
+							 sk_node.pprev));
+		}
+	}
 	else
 		sk = kmalloc(prot->obj_size, priority);
 
#12 lore raw

Re: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Quoted 39 lines · Paul E. McKenney
Quoted 38 lines · Eric Dumazet
Quoted 37 lines · Eric Dumazet
Quoted 10 lines · David Miller

From: Eric Dumazet <[email protected]>
Date: Wed, 08 Jul 2009 00:33:29 +0200

Quoted 7 lines

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Quoted 8 lines
Quoted 6 lines

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

I've applied this but will wait for some more testing before I push it out for real to kernel.org

Thanks David

I forgot to CC Paul and Patrick, so I'll ask them to look at this patch.

Patrick, a similar fix is needed in conntrack as well, we currently uses "ct = kmem_cache_zalloc(nf_conntrack_cachep, gfp);" and thus
overwrite struct hlist_nulls_node hnnode; contained
in "struct nf_conntrack_tuple_hash", while lockless readers still potentialy need them. Setting hnnode.next to NULL is dangerous since last bit is not set (not a nulls value), a reader could try to dereference this NULL pointer and trap.

Here is the patch again so that Paul & Patrick can comment on it.

I am not sure about the refcnt thing (blindly setting it to 0 again should be OK in fact, since no reader should/can to the atomic_inc_if_not_zero on it), but the nulls.next thing is problematic.

Quoted 7 lines
Quoted 3 lines

Here is an updated and much simpler patch, taking care of sk_node.next being not set to 0

This patch applies to >= 2.6.29 kernels

Does this one also need the rearrangement of struct elements in the earlier patch? (And apologies about being slow to get to that one.)

No, because only one field (sk_node.next) needs special attention, I felt it was not really necessary to reorder fields. First memset is inlined because of constant size, so small cost.

Thanks

Quoted 44 lines

Thanx, Paul

Quoted 42 lines

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code correctness depends on sk->sk_nulls_node.next being always valid. A NULL value is not allowed as it might fault a lockless reader.

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO. Just call memset() around the forbidden field.

---

net/core/sock.c +17 -2
index b0ba569..7b87ec0 100644
@@ -939,8 +939,23 @@ static struct sock *sk_prot_alloc(struct proto *prot, gfp_t priority,
 	struct kmem_cache *slab;
 
 	slab = prot->slab;
-	if (slab != NULL)
-		sk = kmem_cache_alloc(slab, priority);
+	if (slab != NULL) {
+		sk = kmem_cache_alloc(slab, priority & ~__GFP_ZERO);
+		if (!sk)
+			return sk;
+		if (priority & __GFP_ZERO) {
+			/*
+			 * caches using SLAB_DESTROY_BY_RCU should let
+			 * sk_node.next un-modified. Special care is taken
+			 * when initializing object to zero.
+			 */
+			if (offsetof(struct sock, sk_node.next) != 0)
+				memset(sk, 0, offsetof(struct sock, sk_node.next));
+			memset(&sk->sk_node.pprev, 0,
+			       prot->obj_size - offsetof(struct sock,
+							 sk_node.pprev));
+		}
+	}
 	else
 		sk = kmalloc(prot->obj_size, priority);
 
maintainer
#13 lore raw

Re: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

In reply to Eric Dumazet

From: Eric Dumazet <[email protected]>
Date: Thu, 09 Jul 2009 07:36:05 +0200

Quoted 11 lines

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code correctness depends on sk->sk_nulls_node.next being always valid. A NULL value is not allowed as it might fault a lockless reader.

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO. Just call memset() around the forbidden field.

APplied and queued up for -stable.

#14 lore raw

Re: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Quoted 9 lines · David Miller

From: Eric Dumazet <[email protected]>
Date: Thu, 09 Jul 2009 07:36:05 +0200

Quoted 6 lines

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code correctness depends on sk->sk_nulls_node.next being always valid. A NULL value is not allowed as it might fault a lockless reader.

Quoted 7 lines
Quoted 5 lines

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO. Just call memset() around the forbidden field.

APplied and queued up for -stable.

I'll try to find some time to fix netfilter conntrack as well.

Thanks

#15 lore raw

PATCH 1/1 net: nf_conntrack_alloc() should not use kmem_cache_zalloc()

David, Patrick

Here is a fix for nf_conntrack, candidate for linux-2.6.31 and stable (linux-2.6.30)

Thank you

[PATCH] net: nf_conntrack_alloc() should not use kmem_cache_zalloc()

When a slab cache uses SLAB_DESTROY_BY_RCU, we must be careful when allocating objects, since slab allocator could give a freed object still used by lockless readers.

In particular, nf_conntrack RCU lookups rely on ct->tuplehash[xxx].hnnode.next being always valid (ie containing a valid 'nulls' value, or a valid pointer to next object in hash chain.)

kmem_cache_zalloc() setups object with NULL values, but a NULL value is not valid for ct->tuplehash[xxx].hnnode.next.

Fix is to call kmem_cache_alloc() and do the zeroing ourself.

---

net/netfilter/nf_conntrack_core.c +13 -2
index 7508f11..23feafa 100644
@@ -561,17 +561,28 @@ struct nf_conn *nf_conntrack_alloc(struct net *net,
 		}
 	}
 
-	ct = kmem_cache_zalloc(nf_conntrack_cachep, gfp);
+	/*
+	 * Do not use kmem_cache_zalloc(), as this cache uses
+	 * SLAB_DESTROY_BY_RCU.
+	 */
+	ct = kmem_cache_alloc(nf_conntrack_cachep, gfp);
 	if (ct == NULL) {
 		pr_debug("nf_conntrack_alloc: Can't alloc conntrack.\n");
 		atomic_dec(&net->ct.count);
 		return ERR_PTR(-ENOMEM);
 	}
-
+	/*
+	 * Let ct->tuplehash[IP_CT_DIR_ORIGINAL].hnnode.next
+	 * and ct->tuplehash[IP_CT_DIR_REPLY].hnnode.next unchanged.
+	 */
+	memset(&ct->tuplehash[IP_CT_DIR_MAX], 0,
+	       sizeof(*ct) - offsetof(struct nf_conn, tuplehash[IP_CT_DIR_MAX]));
 	spin_lock_init(&ct->lock);
 	atomic_set(&ct->ct_general.use, 1);
 	ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple = *orig;
+	ct->tuplehash[IP_CT_DIR_ORIGINAL].hnnode.pprev = NULL;
 	ct->tuplehash[IP_CT_DIR_REPLY].tuple = *repl;
+	ct->tuplehash[IP_CT_DIR_REPLY].hnnode.pprev = NULL;
 	/* Don't set timer yet: wait for confirmation */
 	setup_timer(&ct->timeout, death_by_timeout, (unsigned long)ct);
 #ifdef CONFIG_NET_NS
#16 lore raw

Re: [PATCH] net: nf_conntrack_alloc() should not use kmem_cache_zalloc()

Quoted 6 lines · Eric Dumazet

[PATCH] net: nf_conntrack_alloc() should not use kmem_cache_zalloc()

When a slab cache uses SLAB_DESTROY_BY_RCU, we must be careful when allocating objects, since slab allocator could give a freed object still used by lockless readers.

Quoted 8 lines

In particular, nf_conntrack RCU lookups rely on ct->tuplehash[xxx].hnnode.next being always valid (ie containing a valid 'nulls' value, or a valid pointer to next object in hash chain.)

kmem_cache_zalloc() setups object with NULL values, but a NULL value is not valid for ct->tuplehash[xxx].hnnode.next.

Fix is to call kmem_cache_alloc() and do the zeroing ourself.

I think this is still racy, please see below:

Quoted 10 lines
net/netfilter/nf_conntrack_core.c +1 -1
index 7508f11..23feafa 100644
@@ -561,17 +561,28 @@ struct nf_conn *nf_conntrack_alloc(struct net *net,
 		}
 	}
 
-	ct = kmem_cache_zalloc(nf_conntrack_cachep, gfp);
+	/*

net/netfilter/nf_conntrack_core.c struct nf_conn *nf_conntrack_alloc(struct net *net, · line 570

Quoted 10 lines
net/netfilter/nf_conntrack_core.c +4 -1
+	 * Do not use kmem_cache_zalloc(), as this cache uses
+	 * SLAB_DESTROY_BY_RCU.
+	 */
+	ct = kmem_cache_alloc(nf_conntrack_cachep, gfp);
 	if (ct == NULL) {
 		pr_debug("nf_conntrack_alloc: Can't alloc conntrack.\n");
 		atomic_dec(&net->ct.count);
 		return ERR_PTR(-ENOMEM);
 	}
-

__nf_conntrack_find() on another CPU finds the entry at this point.

net/netfilter/nf_conntrack_core.c struct nf_conn *nf_conntrack_alloc(struct net *net, · line 581

Quoted 8 lines
net/netfilter/nf_conntrack_core.c +6 -0
+	/*
+	 * Let ct->tuplehash[IP_CT_DIR_ORIGINAL].hnnode.next
+	 * and ct->tuplehash[IP_CT_DIR_REPLY].hnnode.next unchanged.
+	 */
+	memset(&ct->tuplehash[IP_CT_DIR_MAX], 0,
+	       sizeof(*ct) - offsetof(struct nf_conn, tuplehash[IP_CT_DIR_MAX]));
 	spin_lock_init(&ct->lock);
 	atomic_set(&ct->ct_general.use, 1);

nf_conntrack_find_get() successfully tries to atomic_inc_not_zero() at this point, following by another tuple comparison which is also successful.

Am I missing something? I think we need to make sure the reference count is not increased until the new tuples are visible.

Quoted 8 lines
net/netfilter/nf_conntrack_core.c +2 -0
 	ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple = *orig;
+	ct->tuplehash[IP_CT_DIR_ORIGINAL].hnnode.pprev = NULL;
 	ct->tuplehash[IP_CT_DIR_REPLY].tuple = *repl;
+	ct->tuplehash[IP_CT_DIR_REPLY].hnnode.pprev = NULL;
 	/* Don't set timer yet: wait for confirmation */
 	setup_timer(&ct->timeout, death_by_timeout, (unsigned long)ct);
 #ifdef CONFIG_NET_NS
#17 lore raw

PATCH 1/1 net: nf_conntrack_alloc() fixes

Quoted 11 lines · Patrick McHardy
Quoted 10 lines · Eric Dumazet

[PATCH] net: nf_conntrack_alloc() should not use kmem_cache_zalloc()

When a slab cache uses SLAB_DESTROY_BY_RCU, we must be careful when allocating objects, since slab allocator could give a freed object still used by lockless readers.

In particular, nf_conntrack RCU lookups rely on ct->tuplehash[xxx].hnnode.next being always valid (ie containing a valid 'nulls' value, or a valid pointer to next object in hash chain.)

Quoted 6 lines
Quoted 4 lines

kmem_cache_zalloc() setups object with NULL values, but a NULL value is not valid for ct->tuplehash[xxx].hnnode.next.

Fix is to call kmem_cache_alloc() and do the zeroing ourself.

I think this is still racy, please see below:

Nice catch indeed !

Quoted 37 lines
Quoted 11 lines
net/netfilter/nf_conntrack_core.c +1 -1
index 7508f11..23feafa 100644
@@ -561,17 +561,28 @@ struct nf_conn *nf_conntrack_alloc(struct net *net,
 		}
 	}
 
-	ct = kmem_cache_zalloc(nf_conntrack_cachep, gfp);
+	/*

net/netfilter/nf_conntrack_core.c struct nf_conn *nf_conntrack_alloc(struct net *net, · line 570

Quoted 10 lines
net/netfilter/nf_conntrack_core.c +4 -1
+	 * Do not use kmem_cache_zalloc(), as this cache uses
+	 * SLAB_DESTROY_BY_RCU.
+	 */
+	ct = kmem_cache_alloc(nf_conntrack_cachep, gfp);
 	if (ct == NULL) {
 		pr_debug("nf_conntrack_alloc: Can't alloc conntrack.\n");
 		atomic_dec(&net->ct.count);
 		return ERR_PTR(-ENOMEM);
 	}
-

__nf_conntrack_find() on another CPU finds the entry at this point.

net/netfilter/nf_conntrack_core.c struct nf_conn *nf_conntrack_alloc(struct net *net, · line 581

Quoted 8 lines
net/netfilter/nf_conntrack_core.c +6 -0
+	/*
+	 * Let ct->tuplehash[IP_CT_DIR_ORIGINAL].hnnode.next
+	 * and ct->tuplehash[IP_CT_DIR_REPLY].hnnode.next unchanged.
+	 */
+	memset(&ct->tuplehash[IP_CT_DIR_MAX], 0,
+	       sizeof(*ct) - offsetof(struct nf_conn, tuplehash[IP_CT_DIR_MAX]));
 	spin_lock_init(&ct->lock);
 	atomic_set(&ct->ct_general.use, 1);

nf_conntrack_find_get() successfully tries to atomic_inc_not_zero() at this point, following by another tuple comparison which is also successful.

Quoted 2 lines

Am I missing something? I think we need to make sure the reference count is not increased until the new tuples are visible.

Yes you are right, and Documentation/RCU/rculist_nulls.txt should be updated to reflect this as well (in insert algo, must use smp_wmb() between obj->key assignment and refcnt assigment)

We'll have to change socket allocation too, this will be addressed by a followup patch

Thanks Patrick !

[PATCH] net: nf_conntrack_alloc() fixes

When a slab cache uses SLAB_DESTROY_BY_RCU, we must be careful when allocating objects, since slab allocator could give a freed object still used by lockless readers.

In particular, nf_conntrack RCU lookups rely on ct->tuplehash[xxx].hnnode.next being always valid (ie containing a valid 'nulls' value, or a valid pointer to next object in hash chain.)

kmem_cache_zalloc() setups object with NULL values, but a NULL value is not valid for ct->tuplehash[xxx].hnnode.next.

Fix is to call kmem_cache_alloc() and do the zeroing ourself.

As spotted by Patrick, we also need to make sure lookup keys are committed to memory before setting refcount to 1, or a lockless reader could get a reference on the old version of the object. Its key re-check could then pass the barrier.

---
 Documentation/RCU/rculist_nulls.txt |    7 ++++++-
 net/netfilter/nf_conntrack_core.c   |   21 ++++++++++++++++++---
 2 files changed, 24 insertions(+), 4 deletions(-)
Documentation/RCU/rculist_nulls.txt +6 -1
index 93cb28d..18f9651 100644
@@ -83,11 +83,12 @@ not detect it missed following items in original chain.
 obj = kmem_cache_alloc(...);
 lock_chain(); // typically a spin_lock()
 obj->key = key;
-atomic_inc(&obj->refcnt);
 /*
  * we need to make sure obj->key is updated before obj->next
+ * or obj->refcnt
  */
 smp_wmb();
+atomic_set(&obj->refcnt, 1);
 hlist_add_head_rcu(&obj->obj_node, list);
 unlock_chain(); // typically a spin_unlock()
 
@@ -159,6 +160,10 @@ out:
 obj = kmem_cache_alloc(cachep);
 lock_chain(); // typically a spin_lock()
 obj->key = key;
+/*
+ * changes to obj->key must be visible before refcnt one
+ */
+smp_wmb();
 atomic_set(&obj->refcnt, 1);
 /*
  * insert obj in RCU way (readers might be traversing chain)
net/netfilter/nf_conntrack_core.c +18 -3
index 7508f11..b5869b9 100644
@@ -561,23 +561,38 @@ struct nf_conn *nf_conntrack_alloc(struct net *net,
 		}
 	}
 
-	ct = kmem_cache_zalloc(nf_conntrack_cachep, gfp);
+	/*
+	 * Do not use kmem_cache_zalloc(), as this cache uses
+	 * SLAB_DESTROY_BY_RCU.
+	 */
+	ct = kmem_cache_alloc(nf_conntrack_cachep, gfp);
 	if (ct == NULL) {
 		pr_debug("nf_conntrack_alloc: Can't alloc conntrack.\n");
 		atomic_dec(&net->ct.count);
 		return ERR_PTR(-ENOMEM);
 	}
-
+	/*
+	 * Let ct->tuplehash[IP_CT_DIR_ORIGINAL].hnnode.next
+	 * and ct->tuplehash[IP_CT_DIR_REPLY].hnnode.next unchanged.
+	 */
+	memset(&ct->tuplehash[IP_CT_DIR_MAX], 0,
+	       sizeof(*ct) - offsetof(struct nf_conn, tuplehash[IP_CT_DIR_MAX]));
 	spin_lock_init(&ct->lock);
-	atomic_set(&ct->ct_general.use, 1);
 	ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple = *orig;
+	ct->tuplehash[IP_CT_DIR_ORIGINAL].hnnode.pprev = NULL;
 	ct->tuplehash[IP_CT_DIR_REPLY].tuple = *repl;
+	ct->tuplehash[IP_CT_DIR_REPLY].hnnode.pprev = NULL;
 	/* Don't set timer yet: wait for confirmation */
 	setup_timer(&ct->timeout, death_by_timeout, (unsigned long)ct);
 #ifdef CONFIG_NET_NS
 	ct->ct_net = net;
 #endif
 
+	/*
+	 * changes to lookup keys must be done before setting refcnt to 1
+	 */
+	smp_wmb();
+	atomic_set(&ct->ct_general.use, 1);
 	return ct;
 }
 EXPORT_SYMBOL_GPL(nf_conntrack_alloc);
#18 lore raw

PATCH 1/1 net: sock_copy() fixes

David, I believe following patch could be 2.6.31 candidate (and stable (2.6.29/2.6.30) as well)

Note : Because we clear all sock fields in sk_prot_alloc(), setting sk_refcnt to 1 in sock_init_data() is safe, even if sock not yet added to hash table (with correct keys). Lockless readers have to re-check their key after getting a reference on the found object anyway, their re-check should not catch old key values.

Next problem to fix (but it can wait 2.6.32 I believe) is to delay sk_refcnt initialization *after* sock lookup keys are setup for new socket, not after being set to 0.

A lockless reader could :
- find an old sock, with old keys
(at this point, another cpu re-allocate this sock, clear fields,
and sets sk_refcnt to 1)
- succeed to increment sk_refcnt
(at this point, other cpu is filling lookup fields (remote address/port, local address...)
- re-check keys and find a match, while socket still being initialized by other cpu.

Only way to fix this is to set sk_refcnt to a non null value only at the point it is safe for a reader to find the socket and use it.

Thanks

[PATCH] net: sock_copy() fixes

Commit e912b1142be8f1e2c71c71001dc992c6e5eb2ec1 (net: sk_prot_alloc() should not blindly overwrite memory) took care of not zeroing whole new socket at allocation time.

sock_copy() is another spot where we should be very careful. We should not set refcnt to a non null value, until we are sure other fields are correctly setup, or
a lockless reader could catch this socket by mistake, while not fully (re)initialized.

This patch puts sk_node & sk_refcnt to the very beginning of struct sock to ease sock_copy() & sk_prot_alloc() job.

We add appropriate smp_wmb() before sk_refcnt initializations to match our RCU requirements (changes to sock keys should be committed to memory before sk_refcnt setting)

---
 include/net/sock.h |   32 +++++++++++++++++++-------------
 net/core/sock.c    |   20 ++++++++++++++++++--
 2 files changed, 37 insertions(+), 15 deletions(-)
include/net/sock.h +19 -13
index 2c0da92..ae0906e 100644
@@ -104,15 +104,15 @@ struct net;
 
 /**
  *	struct sock_common - minimal network layer representation of sockets
+ *	@skc_node: main hash linkage for various protocol lookup tables
+ *	@skc_nulls_node: main hash linkage for UDP/UDP-Lite protocol
+ *	@skc_refcnt: reference count
+ *	@skc_hash: hash value used with various protocol lookup tables
  *	@skc_family: network address family
  *	@skc_state: Connection state
  *	@skc_reuse: %SO_REUSEADDR setting
  *	@skc_bound_dev_if: bound device index if != 0
- *	@skc_node: main hash linkage for various protocol lookup tables
- *	@skc_nulls_node: main hash linkage for UDP/UDP-Lite protocol
  *	@skc_bind_node: bind hash linkage for various protocol lookup tables
- *	@skc_refcnt: reference count
- *	@skc_hash: hash value used with various protocol lookup tables
  *	@skc_prot: protocol handlers inside a network family
  *	@skc_net: reference to the network namespace of this socket
  *
@@ -120,17 +120,21 @@ struct net;
  *	for struct sock and struct inet_timewait_sock.
  */
 struct sock_common {
-	unsigned short		skc_family;
-	volatile unsigned char	skc_state;
-	unsigned char		skc_reuse;
-	int			skc_bound_dev_if;
+	/*
+	 * first fields are not copied in sock_copy()
+	 */
 	union {
 		struct hlist_node	skc_node;
 		struct hlist_nulls_node skc_nulls_node;
 	};
-	struct hlist_node	skc_bind_node;
 	atomic_t		skc_refcnt;
+
 	unsigned int		skc_hash;
+	unsigned short		skc_family;
+	volatile unsigned char	skc_state;
+	unsigned char		skc_reuse;
+	int			skc_bound_dev_if;
+	struct hlist_node	skc_bind_node;
 	struct proto		*skc_prot;
 #ifdef CONFIG_NET_NS
 	struct net	 	*skc_net;
@@ -208,15 +212,17 @@ struct sock {
 	 * don't add nothing before this first member (__sk_common) --acme
 	 */
 	struct sock_common	__sk_common;
+#define sk_node			__sk_common.skc_node
+#define sk_nulls_node		__sk_common.skc_nulls_node
+#define sk_refcnt		__sk_common.skc_refcnt
+
+#define sk_copy_start		__sk_common.skc_hash
+#define sk_hash			__sk_common.skc_hash
 #define sk_family		__sk_common.skc_family
 #define sk_state		__sk_common.skc_state
 #define sk_reuse		__sk_common.skc_reuse
 #define sk_bound_dev_if		__sk_common.skc_bound_dev_if
-#define sk_node			__sk_common.skc_node
-#define sk_nulls_node		__sk_common.skc_nulls_node
 #define sk_bind_node		__sk_common.skc_bind_node
-#define sk_refcnt		__sk_common.skc_refcnt
-#define sk_hash			__sk_common.skc_hash
 #define sk_prot			__sk_common.skc_prot
 #define sk_net			__sk_common.skc_net
 	kmemcheck_bitfield_begin(flags);
net/core/sock.c +18 -2
index ba5d211..d9eec15 100644
@@ -919,13 +919,19 @@ static inline void sock_lock_init(struct sock *sk)
 			af_family_keys + sk->sk_family);
 }
 
+/*
+ * Copy all fields from osk to nsk but nsk->sk_refcnt must not change yet,
+ * even temporarly, because of RCU lookups. sk_node should also be left as is.
+ */
 static void sock_copy(struct sock *nsk, const struct sock *osk)
 {
 #ifdef CONFIG_SECURITY_NETWORK
 	void *sptr = nsk->sk_security;
 #endif
-
-	memcpy(nsk, osk, osk->sk_prot->obj_size);
+	BUILD_BUG_ON(offsetof(struct sock, sk_copy_start) !=
+		     sizeof(osk->sk_node) + sizeof(osk->sk_refcnt));
+	memcpy(&nsk->sk_copy_start, &osk->sk_copy_start,
+	       osk->sk_prot->obj_size - offsetof(struct sock, sk_copy_start));
 #ifdef CONFIG_SECURITY_NETWORK
 	nsk->sk_security = sptr;
 	security_sk_clone(osk, nsk);
@@ -1140,6 +1146,11 @@ struct sock *sk_clone(const struct sock *sk, const gfp_t priority)
 
 		newsk->sk_err	   = 0;
 		newsk->sk_priority = 0;
+		/*
+		 * Before updating sk_refcnt, we must commit prior changes to memory
+		 * (Documentation/RCU/rculist_nulls.txt for details)
+		 */
+		smp_wmb();
 		atomic_set(&newsk->sk_refcnt, 2);
 
 		/*
@@ -1855,6 +1866,11 @@ void sock_init_data(struct socket *sock, struct sock *sk)
 
 	sk->sk_stamp = ktime_set(-1L, 0);
 
+	/*
+	 * Before updating sk_refcnt, we must commit prior changes to memory
+	 * (Documentation/RCU/rculist_nulls.txt for details)
+	 */
+	smp_wmb();
 	atomic_set(&sk->sk_refcnt, 1);
 	atomic_set(&sk->sk_wmem_alloc, 1);
 	atomic_set(&sk->sk_drops, 0);
maintainer
#19 lore raw

Re: [PATCH] net: sock_copy() fixes

From: Eric Dumazet <[email protected]>
Date: Thu, 16 Jul 2009 11:13:10 +0200

Quoted 12 lines

[PATCH] net: sock_copy() fixes

Commit e912b1142be8f1e2c71c71001dc992c6e5eb2ec1 (net: sk_prot_alloc() should not blindly overwrite memory) took care of not zeroing whole new socket at allocation time.

sock_copy() is another spot where we should be very careful. We should not set refcnt to a non null value, until we are sure other fields are correctly setup, or
a lockless reader could catch this socket by mistake, while not fully (re)initialized.

Quoted 8 lines

This patch puts sk_node & sk_refcnt to the very beginning of struct sock to ease sock_copy() & sk_prot_alloc() job.

We add appropriate smp_wmb() before sk_refcnt initializations to match our RCU requirements (changes to sock keys should be committed to memory before sk_refcnt setting)

Applied and queued for -stable, thanks!

#20 lore raw

Re: [PATCH] net: nf_conntrack_alloc() fixes

In reply to Eric Dumazet
Quoted 10 lines · Eric Dumazet

[PATCH] net: nf_conntrack_alloc() fixes

When a slab cache uses SLAB_DESTROY_BY_RCU, we must be careful when allocating objects, since slab allocator could give a freed object still used by lockless readers.

In particular, nf_conntrack RCU lookups rely on ct->tuplehash[xxx].hnnode.next being always valid (ie containing a valid 'nulls' value, or a valid pointer to next object in hash chain.)

Quoted 8 lines

kmem_cache_zalloc() setups object with NULL values, but a NULL value is not valid for ct->tuplehash[xxx].hnnode.next.

Fix is to call kmem_cache_alloc() and do the zeroing ourself.

As spotted by Patrick, we also need to make sure lookup keys are committed to memory before setting refcount to 1, or a lockless reader could get a reference on the old version of the object. Its key re-check could then pass the barrier.

Looks good to me. Applied, thanks Eric. I'll push it to -stable with the other fixes in a couple of days.

#21 lore raw

RE: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

In reply to David Miller
Quoted 14 lines · David Miller

From: Eric Dumazet <[email protected]>
Date: Wed, 08 Jul 2009 00:33:29 +0200

Quoted 11 lines

[PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Some sockets use SLAB_DESTROY_BY_RCU, and our RCU code rely that some fields should not be blindly overwritten, even with null.

These fields are sk->sk_refcnt and sk->sk_nulls_node.next

Current sk_prot_alloc() implementation doesnt respect this hypothesis, calling kmem_cache_alloc() with __GFP_ZERO and setting sk_refcnt to 1 instead of atomically increment it.

Quoted 5 lines
Quoted 2 lines

I've applied this but will wait for some more testing before I push it out for real to kernel.org

Still seeing traces during the test even with this patch applied:

[ 1089.430093] ------------[ cut here ]------------
[ 1089.435667] WARNING: at include/net/sock.h:423 udp_lib_unhash+0x73/0xa0()
[ 1089.435670] Hardware name: S5520HC
[ 1089.435671] Modules linked in: igb dca mdio [last unloaded: ixgbe]
[ 1089.435678] Pid: 15545, comm: netserver Not tainted 2.6.31-rc1-net-2.6-igb-ed-07071641 #4
[ 1089.435681] Call Trace:
[ 1089.435686]  [<ffffffff813e8a2f>] ? udp_lib_unhash+0x73/0xa0
[ 1089.435691]  [<ffffffff81057b49>] warn_slowpath_common+0x77/0x8f
[ 1089.435696]  [<ffffffff81057b70>] warn_slowpath_null+0xf/0x11
[ 1089.435700]  [<ffffffff813e8a2f>] udp_lib_unhash+0x73/0xa0
[ 1089.435705]  [<ffffffff8138e616>] sk_common_release+0x2f/0xb4
[ 1089.435710]  [<ffffffff81429028>] udp_lib_close+0x9/0xb
[ 1089.435715]  [<ffffffff813ee62a>] inet_release+0x58/0x5f
[ 1089.435720]  [<ffffffff814158e5>] inet6_release+0x30/0x35
[ 1089.435725]  [<ffffffff8138be4b>] sock_release+0x1a/0x6c
[ 1089.435729]  [<ffffffff8138c366>] sock_close+0x22/0x26
[ 1089.435735]  [<ffffffff810ec923>] __fput+0xf0/0x18c
[ 1089.435739]  [<ffffffff810eccd1>] fput+0x15/0x18
[ 1089.435742]  [<ffffffff810e9bfa>] filp_close+0x5c/0x67
[ 1089.435746]  [<ffffffff810e9c80>] sys_close+0x7b/0xb6
[ 1089.435751]  [<ffffffff81027aab>] system_call_fastpath+0x16/0x1b
[ 1089.435755] ---[ end trace a79410bd00b8b1ac ]---

Emil

maintainer
#22 lore raw

Re: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

From: "Tantilov, Emil S" <[email protected]>
Date: Wed, 8 Jul 2009 11:02:22 -0600

Quoted 5 lines

Still seeing traces during the test even with this patch applied:

[ 1089.430093] ------------[ cut here ]------------
[ 1089.435667] WARNING: at include/net/sock.h:423 udp_lib_unhash+0x73/0xa0()
[ 1089.435670] Hardware name: S5520HC

Ok I'll back this out for now, needs more investigation obviously.

#23 lore raw

Re: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Quoted 11 lines · David Miller

From: "Tantilov, Emil S" <[email protected]>
Date: Wed, 8 Jul 2009 11:02:22 -0600

Quoted 5 lines

Still seeing traces during the test even with this patch applied:

[ 1089.430093] ------------[ cut here ]------------
[ 1089.435667] WARNING: at include/net/sock.h:423 udp_lib_unhash+0x73/0xa0()
[ 1089.435670] Hardware name: S5520HC

Ok I'll back this out for now, needs more investigation obviously.

Hmm... I never said it was supposed to fix Emil problem, just that I discovered one potential problem by code inspection.

I could not find yet sk_refcnt mismatch.
As we do less atomic ops per packet than before, some old bug could surface now...

Emil, is it easy to reproduce this problem, considering I have a similar platform than yours (dual quad core machine, E5450 cpus @ 3GHz) ?

#24 lore raw

RE: [PATCH] net: sk_prot_alloc() should not blindly overwrite memory

Quoted 16 lines · Eric Dumazet
Quoted 11 lines · David Miller

From: "Tantilov, Emil S" <[email protected]>
Date: Wed, 8 Jul 2009 11:02:22 -0600

Quoted 5 lines

Still seeing traces during the test even with this patch applied:

[ 1089.430093] ------------[ cut here ]------------
[ 1089.435667] WARNING: at include/net/sock.h:423

udp_lib_unhash+0x73/0xa0() [ 1089.435670] Hardware name: S5520HC

Ok I'll back this out for now, needs more investigation obviously.

Hmm... I never said it was supposed to fix Emil problem, just that I discovered one potential problem by code inspection.

Quoted 7 lines

I could not find yet sk_refcnt mismatch.
As we do less atomic ops per packet than before, some old bug could surface now...

Emil, is it easy to reproduce this problem, considering I have a similar platform than yours (dual quad core machine, E5450 cpus @ 3GHz) ?

Eric,

It should be easy to reproduce. At least I have been able to consistently reproduce it on several different systems with different drivers (e1000, e1000e, igb).

The test I'm running is a mix of IPV4/6 TCP/UDP traffic with netperf (also mixing different types TCP/UDP_STREAM, TCP_MAERTS, TCP_UDP_RR etc). How much this matters I don't know - it's possible that just UDP traffic would do it. I also think it may have something to do with IPv6 because of the trace, but I am not sure.

If you need more information let me know.

Thanks,
Emil

#25 lore raw

PATCH 1/1 net: ip_push_pending_frames() fix

Quoted 25 lines · Emil S
Quoted 14 lines · Eric Dumazet
Quoted 10 lines · David Miller

From: "Tantilov, Emil S" <[email protected]>
Date: Wed, 8 Jul 2009 11:02:22 -0600

Quoted 5 lines

Still seeing traces during the test even with this patch applied:

[ 1089.430093] ------------[ cut here ]------------
[ 1089.435667] WARNING: at include/net/sock.h:423

udp_lib_unhash+0x73/0xa0() [ 1089.435670] Hardware name: S5520HC

Ok I'll back this out for now, needs more investigation obviously.

Hmm... I never said it was supposed to fix Emil problem, just that I discovered one potential problem by code inspection.

Quoted 7 lines

I could not find yet sk_refcnt mismatch.
As we do less atomic ops per packet than before, some old bug could surface now...

Emil, is it easy to reproduce this problem, considering I have a similar platform than yours (dual quad core machine, E5450 cpus @ 3GHz) ?

Eric,

Quoted 8 lines

It should be easy to reproduce. At least I have been able to consistently reproduce it on several different systems with different drivers (e1000, e1000e, igb).

The test I'm running is a mix of IPV4/6 TCP/UDP traffic with netperf (also mixing different types TCP/UDP_STREAM, TCP_MAERTS, TCP_UDP_RR etc). How much this matters I don't know - it's possible that just UDP traffic would do it. I also think it may have something to do with IPv6 because of the trace, but I am not sure.

If you need more information let me know.

OK thanks, this was helpful, corking or not corking, that is the question :)

I think ip6_push_pending_frames() & ip_push_pending_frames
have a problem after recent commit 2b85a34e911bf483c27cfdd124aeb1605145dc80 (net: No more expensive sock_hold()/sock_put() on each tx)

[PATCH] net: ip_push_pending_frames() fix

After commit 2b85a34e911bf483c27cfdd124aeb1605145dc80
(net: No more expensive sock_hold()/sock_put() on each tx) we do not take any more references on sk->sk_refcnt on outgoing packets.

I forgot to delete two __sock_put() from ip_push_pending_frames() and ip6_push_pending_frames().

---

net/ipv4/ip_output.c +0 -1
index 2470262..7d08210 100644
@@ -1243,7 +1243,6 @@ int ip_push_pending_frames(struct sock *sk)
 		skb->len += tmp_skb->len;
 		skb->data_len += tmp_skb->len;
 		skb->truesize += tmp_skb->truesize;
-		__sock_put(tmp_skb->sk);
 		tmp_skb->destructor = NULL;
 		tmp_skb->sk = NULL;
 	}
net/ipv6/ip6_output.c +0 -1
index 7c76e3d..87f8419 100644
@@ -1484,7 +1484,6 @@ int ip6_push_pending_frames(struct sock *sk)
 		skb->len += tmp_skb->len;
 		skb->data_len += tmp_skb->len;
 		skb->truesize += tmp_skb->truesize;
-		__sock_put(tmp_skb->sk);
 		tmp_skb->destructor = NULL;
 		tmp_skb->sk = NULL;
 	}
#26 lore raw

RE: [PATCH] net: ip_push_pending_frames() fix

Quoted 78 lines · Eric Dumazet
Quoted 30 lines · Emil S
Quoted 14 lines · Eric Dumazet
Quoted 10 lines · David Miller

From: "Tantilov, Emil S" <[email protected]>
Date: Wed, 8 Jul 2009 11:02:22 -0600

Quoted 5 lines

Still seeing traces during the test even with this patch applied:

[ 1089.430093] ------------[ cut here ]------------
[ 1089.435667] WARNING: at include/net/sock.h:423

udp_lib_unhash+0x73/0xa0() [ 1089.435670] Hardware name: S5520HC

Ok I'll back this out for now, needs more investigation obviously.

Hmm... I never said it was supposed to fix Emil problem, just that I discovered one potential problem by code inspection.

Quoted 7 lines

I could not find yet sk_refcnt mismatch.
As we do less atomic ops per packet than before, some old bug could surface now...

Emil, is it easy to reproduce this problem, considering I have a similar platform than yours (dual quad core machine, E5450 cpus @ 3GHz) ?

Eric,

It should be easy to reproduce. At least I have been able to consistently
reproduce it on several different systems with different drivers (e1000, e1000e, igb).

Quoted 8 lines

The test I'm running is a mix of IPV4/6 TCP/UDP traffic with netperf (also mixing different types TCP/UDP_STREAM, TCP_MAERTS, TCP_UDP_RR etc). How much this matters I don't know - it's possible that just UDP traffic would do it. I also think it may have something to do with IPv6 because of the trace, but I am not sure.

If you need more information let me know.

OK thanks, this was helpful, corking or not corking, that is the question :)

I think ip6_push_pending_frames() & ip_push_pending_frames have a problem after recent commit 2b85a34e911bf483c27cfdd124aeb1605145dc80 (net: No more expensive sock_hold()/sock_put() on each tx)

[PATCH] net: ip_push_pending_frames() fix

After commit 2b85a34e911bf483c27cfdd124aeb1605145dc80 (net: No more expensive sock_hold()/sock_put() on each tx) we do not take any more references on sk->sk_refcnt on outgoing packets.

I forgot to delete two __sock_put() from ip_push_pending_frames() and ip6_push_pending_frames().

---

net/ipv4/ip_output.c +0 -1
index 2470262..7d08210 100644
@@ -1243,7 +1243,6 @@ int ip_push_pending_frames(struct sock *sk)
 		skb->len += tmp_skb->len;
 		skb->data_len += tmp_skb->len;
 		skb->truesize += tmp_skb->truesize;
-		__sock_put(tmp_skb->sk);
 		tmp_skb->destructor = NULL;
 		tmp_skb->sk = NULL;
 	}
net/ipv6/ip6_output.c +0 -0
index 7c76e3d..87f8419 100644

net/ipv6/ip6_output.c int ip6_push_pending_frames(struct sock *sk) · line 1489

Quoted 7 lines
net/ipv6/ip6_output.c +0 -1
@@ -1484,7 +1484,6 @@ int ip6_push_pending_frames(struct sock *sk)
 		skb->len += tmp_skb->len;
 		skb->data_len += tmp_skb->len;
 		skb->truesize += tmp_skb->truesize;
-		__sock_put(tmp_skb->sk);
 		tmp_skb->destructor = NULL;
 		tmp_skb->sk = NULL;
 	}

Thanks Eric,

With this patch the test ran all night without issues.

Emil

Tested-by Emil S Tantilov
#27 lore raw

Re: [PATCH] net: ip_push_pending_frames() fix

Quoted 87 lines · Emil S
Quoted 76 lines · Eric Dumazet
Quoted 29 lines · Emil S
Quoted 14 lines · Eric Dumazet
Quoted 10 lines · David Miller

From: "Tantilov, Emil S" <[email protected]>
Date: Wed, 8 Jul 2009 11:02:22 -0600

Quoted 5 lines

Still seeing traces during the test even with this patch applied:

[ 1089.430093] ------------[ cut here ]------------
[ 1089.435667] WARNING: at include/net/sock.h:423

udp_lib_unhash+0x73/0xa0() [ 1089.435670] Hardware name: S5520HC

Ok I'll back this out for now, needs more investigation obviously.

Hmm... I never said it was supposed to fix Emil problem, just that I discovered one potential problem by code inspection.

Quoted 7 lines

I could not find yet sk_refcnt mismatch.
As we do less atomic ops per packet than before, some old bug could surface now...

Emil, is it easy to reproduce this problem, considering I have a similar platform than yours (dual quad core machine, E5450 cpus @ 3GHz) ?

Eric,

It should be easy to reproduce. At least I have been able to consistently
reproduce it on several different systems with different drivers (e1000, e1000e, igb).

Quoted 8 lines

The test I'm running is a mix of IPV4/6 TCP/UDP traffic with netperf (also mixing different types TCP/UDP_STREAM, TCP_MAERTS, TCP_UDP_RR etc). How much this matters I don't know - it's possible that just UDP traffic would do it. I also think it may have something to do with IPv6 because of the trace, but I am not sure.

If you need more information let me know.

OK thanks, this was helpful, corking or not corking, that is the question :)

I think ip6_push_pending_frames() & ip_push_pending_frames have a problem after recent commit 2b85a34e911bf483c27cfdd124aeb1605145dc80 (net: No more expensive sock_hold()/sock_put() on each tx)

[PATCH] net: ip_push_pending_frames() fix

After commit 2b85a34e911bf483c27cfdd124aeb1605145dc80 (net: No more expensive sock_hold()/sock_put() on each tx) we do not take any more references on sk->sk_refcnt on outgoing packets.

I forgot to delete two __sock_put() from ip_push_pending_frames() and ip6_push_pending_frames().

---

net/ipv4/ip_output.c +0 -1
index 2470262..7d08210 100644
@@ -1243,7 +1243,6 @@ int ip_push_pending_frames(struct sock *sk)
 		skb->len += tmp_skb->len;
 		skb->data_len += tmp_skb->len;
 		skb->truesize += tmp_skb->truesize;
-		__sock_put(tmp_skb->sk);
 		tmp_skb->destructor = NULL;
 		tmp_skb->sk = NULL;
 	}
net/ipv6/ip6_output.c +0 -0
index 7c76e3d..87f8419 100644

net/ipv6/ip6_output.c int ip6_push_pending_frames(struct sock *sk) · line 1489

Quoted 7 lines
net/ipv6/ip6_output.c +0 -1
@@ -1484,7 +1484,6 @@ int ip6_push_pending_frames(struct sock *sk)
 		skb->len += tmp_skb->len;
 		skb->data_len += tmp_skb->len;
 		skb->truesize += tmp_skb->truesize;
-		__sock_put(tmp_skb->sk);
 		tmp_skb->destructor = NULL;
 		tmp_skb->sk = NULL;
 	}

Thanks Eric,

Quoted 2 lines

With this patch the test ran all night without issues.

Thanks a lot Emil for testing and your feedback.

David, could you please add another tag ?

Thanks

maintainer
#28 lore raw

Re: [PATCH] net: ip_push_pending_frames() fix

In reply to Eric Dumazet

From: Eric Dumazet <[email protected]>
Date: Thu, 09 Jul 2009 02:20:42 +0200

Quoted 11 lines

[PATCH] net: ip_push_pending_frames() fix

After commit 2b85a34e911bf483c27cfdd124aeb1605145dc80
(net: No more expensive sock_hold()/sock_put() on each tx) we do not take any more references on sk->sk_refcnt on outgoing packets.

I forgot to delete two __sock_put() from ip_push_pending_frames() and ip6_push_pending_frames().

Applied, with the Tested-by marker added.