Skip to contentSkip to conversation

udp: udp_lib_get_port start at a different bucket on different processors

RFC In review Lucian Adrian Grijincu 2 participants 2 messages Dec 16, 2009 – Dec 17, 2009

Subsystems (3)
Networking Network drivers UDP
lore
Share & tools
lore

Lucian Adrian Grijincu

author
#0 lore raw

RFC 2/2 udp: udp_lib_get_port start at a different bucket on different processors

On SMP, when ports are not allocated randomly, using the same starting port on all processors will lead to bad performance as all processors will try to get the same hashbucket spinlock: only one will succeed, the rest will spin madly.

We solve this problem by making each processor start searching from a different port. To not skip possibly valid port ranges we renormalize the hint value too.

Signed-off-by Lucian Adrian Grijincu <[email protected]>
---
 net/ipv4/udp.c |   12 +++++++++---
 1 files changed, 9 insertions(+), 3 deletions(-)
net/ipv4/udp.c +9 -3
index f437d9d..5d17c71 100644
@@ -219,7 +219,9 @@ int udp_lib_get_port(struct sock *sk, unsigned short snum,
 			first = (((u64)rand * remaining) >> 32) + low;
 		} else {
 			rand = 1;
-			first = hint;
+			first = hint + smp_processor_id();
+			if (first > high)
+				first = low + (first - low) % remaining;
 		}
 		/*
 		 * force rand to be an odd multiple of UDP_HTABLE_SIZE

@@ -243,8 +245,12 @@ int udp_lib_get_port(struct sock *sk, unsigned short snum,
 			do {
 				if (low <= snum && snum <= high &&
 				    !test_bit(snum >> udptable->log, bitmap)) {
-					if (unlikely(!sysctl_udp_port_randomization))
-						hint = snum;
+					if (unlikely(!sysctl_udp_port_randomization)) {
+						int cur_hint = snum - smp_processor_id();
+						if (cur_hint < low)
+							cur_hint = high - (low - cur_hint) % remaining;
+						hint = cur_hint;
+					}
 					goto found;
 				}
 				snum += rand;
#1 lore raw
Quoted 6 lines · Lucian Adrian Grijincu

On SMP, when ports are not allocated randomly, using the same starting port on all processors will lead to bad performance as all processors will try to get the same hashbucket spinlock: only one will succeed, the rest will spin madly.

Quoted 10 lines

We solve this problem by making each processor start searching from a different port. To not skip possibly valid port ranges we renormalize the hint value too.

Signed-off-by Lucian Adrian Grijincu <[email protected]>
---
 net/ipv4/udp.c |   12 +++++++++---
 1 files changed, 9 insertions(+), 3 deletions(-)

This makes no sense. You correct a bad performance issue bringed by your previous patch ?

Please submit one patch, with no performance regression.

I suggest you use a per_cpu hint if you really want to be fast...

Thanks