Skip to contentSkip to conversation

do not readlock all buckets in /proc/net/tcp

PATCH In review Marcus Meissner 3 participants 7 messages Jul 5, 2004

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 do not readlock all buckets in /proc/net/tcp

Hi,

This patch makes the files /proc/net/tcp and /proc/net/tcp6 not acquire the readlock for every bucket.

On ppc64 and ia64 the readlocks are so expensive, that reading /proc/net/tcp takes 0.25 seconds on a usual p670 LPAR.

And it locks 65536 buckets where just 20 chains are used at all in a normal non-netserver setup.

Ciao, Marcus

Changelog:
Readlock only non-empty hash chains to avoid 65536 readlocks.

net/ipv4/tcp_ipv4.c renamed from net/ipv4/tcp_ipv4.c.xx +14 -4
@@ -2255,6 +2255,12 @@
 		struct hlist_node *node;
 		struct tcp_tw_bucket *tw;
 	       
+		/* Avoid taking the readlock cost if we know the chain is empty,
+		 * we have a lot of buckets.
+		 */
+		if (hlist_empty(&tcp_ehash[st->bucket].chain) &&
+		    hlist_empty(&tcp_ehash[st->bucket+tcp_ehash_size].chain))
+			continue;
 		read_lock(&tcp_ehash[st->bucket].lock);
 		sk_for_each(sk, node, &tcp_ehash[st->bucket].chain) {
 			if (sk->sk_family != st->family) {
@@ -2301,13 +2307,17 @@
 		}
 		read_unlock(&tcp_ehash[st->bucket].lock);
 		st->state = TCP_SEQ_STATE_ESTABLISHED;
-		if (++st->bucket < tcp_ehash_size) {
-			read_lock(&tcp_ehash[st->bucket].lock);
-			sk = sk_head(&tcp_ehash[st->bucket].chain);
-		} else {
+
+		while ((++st->bucket < tcp_ehash_size) &&
+		       hlist_empty(&tcp_ehash[st->bucket].chain) &&
+		       hlist_empty(&tcp_ehash[st->bucket+tcp_ehash_size].chain))
+			/*empty*/;
+		if (st->bucket >= tcp_ehash_size) {
 			cur = NULL;
 			goto out;
 		}
+		read_lock(&tcp_ehash[st->bucket].lock);
+		sk = sk_head(&tcp_ehash[st->bucket].chain);
 	} else
 		sk = sk_next(sk);
 
#1 lore raw
Quoted 9 lines · Marcus Meissner

This patch makes the files /proc/net/tcp and /proc/net/tcp6 not acquire the readlock for every bucket.

On ppc64 and ia64 the readlocks are so expensive, that reading /proc/net/tcp takes 0.25 seconds on a usual p670 LPAR.

And it locks 65536 buckets where just 20 chains are used at all in a normal non-netserver setup.

Why not use NETLINK+TCP_DIAG instead? It's much faster.

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#2 lore raw
Quoted 5 lines · Herbert Xu
Quoted 4 lines · Marcus Meissner

This patch makes the files /proc/net/tcp and /proc/net/tcp6 not acquire the readlock for every bucket.

Quoted 7 lines
Quoted 5 lines

On ppc64 and ia64 the readlocks are so expensive, that reading /proc/net/tcp takes 0.25 seconds on a usual p670 LPAR.

And it locks 65536 buckets where just 20 chains are used at all in a normal non-netserver setup.

Why not use NETLINK+TCP_DIAG instead? It's much faster.

Not sure if you want / can fix all proprietary software.

Oh, and NETLINK+TCP_DIAG seems to have the same readlock contention problem, see tcpdiag_dump().

Ciao, Marcus

#3 lore raw
Quoted 3 lines · Marcus Meissner

Oh, and NETLINK+TCP_DIAG seems to have the same readlock contention problem, see tcpdiag_dump().

Then you wouldn't mind adding this optimisation for tcp_diag as well, right? :)

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#4 lore raw

In article <[email protected]> (at Mon, 5 Jul 2004 22:06:10 +1000), Herbert Xu <[email protected]> says:

Quoted 7 lines · Marcus Meissner
Quoted 3 lines

Oh, and NETLINK+TCP_DIAG seems to have the same readlock contention problem, see tcpdiag_dump().

Then you wouldn't mind adding this optimisation for tcp_diag as well, right? :)

here it is. :-)

===== net/ipv4/tcp_diag.c 1.15 vs edited =====

net/ipv4/tcp_diag.c +6 -2
@@ -522,9 +522,13 @@
 		if (i > s_i)
 			s_num = 0;
 
-		read_lock_bh(&head->lock);
-
 		num = 0;
+
+		if (hlist_empty(&head->chain) &&
+		    (!(r->tcpdiag_states&TCPF_TIME_WAIT) || hlist_empty(&head->chain)))
+			continue;
+
+		read_lock_bh(&head->lock);
 		sk_for_each(sk, node, &head->chain) {
 			struct inet_opt *inet = inet_sk(sk);
 
Signature · 2 lines
Hideaki YOSHIFUJI @ USAGI Project <[email protected]>
GPG FP: 9022 65EB 1ECF 3AD1 0BDF  80D8 4807 F894 E062 0EEA
#5 lore raw
Quoted 18 lines · YOSHIFUJI Hideaki / 吉藤英明

In article <[email protected]> (at Mon, 5 Jul 2004 22:06:10 +1000), Herbert Xu <[email protected]> says:

Quoted 7 lines · Marcus Meissner
Quoted 3 lines

Oh, and NETLINK+TCP_DIAG seems to have the same readlock contention problem, see tcpdiag_dump().

Then you wouldn't mind adding this optimisation for tcp_diag as well, right? :)

here it is. :-)

===== net/ipv4/tcp_diag.c 1.15 vs edited =====

net/ipv4/tcp_diag.c line 529

Quoted 10 lines
net/ipv4/tcp_diag.c +4 -2
@@ -522,9 +522,13 @@
 		if (i > s_i)
 			s_num = 0;
 
-		read_lock_bh(&head->lock);
-
 		num = 0;
+
+		if (hlist_empty(&head->chain) &&
+		    (!(r->tcpdiag_states&TCPF_TIME_WAIT) || hlist_empty(&head->chain)))
+			continue;

The second hlist_empty is bad, you should be checking &tcp_ehash[i + tcp_ehash_size].chain ((head+tcp_ehash_size) I think).

Ciao, Marcus

#6 lore raw

In article <[email protected]> (at Mon, 5 Jul 2004 14:45:11 +0200), Marcus Meissner <[email protected]> says:

Quoted 2 lines

The second hlist_empty is bad, you should be checking &tcp_ehash[i + tcp_ehash_size].chain ((head+tcp_ehash_size) I think).

Oops...

===== net/ipv4/tcp_diag.c 1.15 vs edited =====

net/ipv4/tcp_diag.c +7 -2
@@ -522,9 +522,14 @@
 		if (i > s_i)
 			s_num = 0;
 
-		read_lock_bh(&head->lock);
-
 		num = 0;
+
+		if (hlist_empty(&head->chain) &&
+		    (!(r->tcpdiag_states&TCPF_TIME_WAIT) || 
+		       hlist_empty(&(head + tcp_ehash_size)->chain)))
+			continue;
+
+		read_lock_bh(&head->lock);
 		sk_for_each(sk, node, &head->chain) {
 			struct inet_opt *inet = inet_sk(sk);
 
Signature · 2 lines
Hideaki YOSHIFUJI @ USAGI Project <[email protected]>
GPG FP: 9022 65EB 1ECF 3AD1 0BDF  80D8 4807 F894 E062 0EEA