Skip to contentSkip to conversation

ipv4: speedup inet_dump_ifaddr()

PATCH Applied Eric Dumazet 3 participants 6 messages Nov 12, 2009 – Nov 14, 2009

Subsystems (3)
Networking Network drivers IPv4/IPv6
lore
Share & tools
lore

Eric Dumazet

author
#0 lore raw

PATCH 1/1 ipv4: speedup inet_dump_ifaddr()

When handling large number of netdevices, inet_dump_ifaddr() is very slow because it has O(N^2) complexity.

Instead of scanning one single list, we can use the NETDEV_HASHENTRIES sub lists of the dev_index hash table, and RCU lookups.

Signed-off-by Eric Dumazet <[email protected]>
---
 net/ipv4/devinet.c |   57 ++++++++++++++++++++++++++-----------------
 1 files changed, 35 insertions(+), 22 deletions(-)
net/ipv4/devinet.c +35 -22
index c2045f9..a74cb4e 100644
@@ -1174,39 +1174,52 @@ nla_put_failure:
 static int inet_dump_ifaddr(struct sk_buff *skb, struct netlink_callback *cb)
 {
 	struct net *net = sock_net(skb->sk);
-	int idx, ip_idx;
+	int h, s_h;
+	int idx, s_idx;
+	int ip_idx, s_ip_idx;
 	struct net_device *dev;
 	struct in_device *in_dev;
 	struct in_ifaddr *ifa;
-	int s_ip_idx, s_idx = cb->args[0];
+	struct hlist_head *head;
+	struct hlist_node *node;
 
-	s_ip_idx = ip_idx = cb->args[1];
-	idx = 0;
-	for_each_netdev(net, dev) {
-		if (idx < s_idx)
-			goto cont;
-		if (idx > s_idx)
-			s_ip_idx = 0;
-		in_dev = __in_dev_get_rtnl(dev);
-		if (!in_dev)
-			goto cont;
+	s_h = cb->args[0];
+	s_idx = idx = cb->args[1];
+	s_ip_idx = ip_idx = cb->args[2];
 
-		for (ifa = in_dev->ifa_list, ip_idx = 0; ifa;
-		     ifa = ifa->ifa_next, ip_idx++) {
-			if (ip_idx < s_ip_idx)
-				continue;
-			if (inet_fill_ifaddr(skb, ifa, NETLINK_CB(cb->skb).pid,
+	rcu_read_lock();
+	for (h = s_h; h < NETDEV_HASHENTRIES; h++, s_idx = 0) {
+		idx = 0;
+		head = &net->dev_index_head[h];
+		hlist_for_each_entry_rcu(dev, node, head, index_hlist) {
+			if (idx < s_idx)
+				goto cont;
+			if (idx > s_idx)
+				s_ip_idx = 0;
+			in_dev = __in_dev_get_rcu(dev);
+			if (!in_dev)
+				goto cont;
+
+			for (ifa = in_dev->ifa_list, ip_idx = 0; ifa;
+			     ifa = ifa->ifa_next, ip_idx++) {
+				if (ip_idx < s_ip_idx)
+					continue;
+				if (inet_fill_ifaddr(skb, ifa,
+					     NETLINK_CB(cb->skb).pid,
 					     cb->nlh->nlmsg_seq,
 					     RTM_NEWADDR, NLM_F_MULTI) <= 0)
-				goto done;
-		}
+					goto done;
+			}
 cont:
-		idx++;
+			idx++;
+		}
 	}
 
 done:
-	cb->args[0] = idx;
-	cb->args[1] = ip_idx;
+	rcu_read_unlock();
+	cb->args[0] = h;
+	cb->args[1] = idx;
+	cb->args[2] = ip_idx;
 
 	return skb->len;
 }
Acked-by on v1 1/1
#1 lore raw
Quoted 7 lines · On Thu, 12 Nov 2009 15:11:36 +0100 Eric Dumazet <[email protected]> wrote:

When handling large number of netdevices, inet_dump_ifaddr() is very slow because it has O(N^2) complexity.

Instead of scanning one single list, we can use the NETDEV_HASHENTRIES sub lists of the dev_index hash table, and RCU lookups.

Signed-off-by Eric Dumazet <[email protected]>

You might be able to make RCU critical section smaller by moving it into loop.

Acked-by Stephen Hemminger <[email protected]>
author Acked-by Stephen Hemminger
#2 lore raw
Quoted 6 lines · Stephen Hemminger
Quoted 3 lines · On Thu, 12 Nov 2009 15:11:36 +0100 Eric Dumazet <[email protected]> wrote:

When handling large number of netdevices, inet_dump_ifaddr() is very slow because it has O(N^2) complexity.

Quoted 8 lines
Quoted 4 lines

Instead of scanning one single list, we can use the NETDEV_HASHENTRIES sub lists of the dev_index hash table, and RCU lookups.

Signed-off-by Eric Dumazet <[email protected]>

You might be able to make RCU critical section smaller by moving it into loop.

Indeed. But we dump at most one skb (<= 8192 bytes ?), so rcu_read_lock holding time is small, unless we meet many netdevices without addresses. I wonder if its really common...

Thanks

[PATCH net-next-2.6] ipv4: speedup inet_dump_ifaddr()

When handling large number of netdevices, inet_dump_ifaddr() is very slow because it has O(N2) complexity.

Instead of scanning one single list, we can use the NETDEV_HASHENTRIES sub lists of the dev_index hash table, and RCU lookups.

Signed-off-by Eric Dumazet <[email protected]>
Acked-by Stephen Hemminger <[email protected]>
---
 net/ipv4/devinet.c |   61 ++++++++++++++++++++++++++-----------------
 1 files changed, 38 insertions(+), 23 deletions(-)
net/ipv4/devinet.c +38 -23
index c2045f9..7620382 100644
@@ -1174,39 +1174,54 @@ nla_put_failure:
 static int inet_dump_ifaddr(struct sk_buff *skb, struct netlink_callback *cb)
 {
 	struct net *net = sock_net(skb->sk);
-	int idx, ip_idx;
+	int h, s_h;
+	int idx, s_idx;
+	int ip_idx, s_ip_idx;
 	struct net_device *dev;
 	struct in_device *in_dev;
 	struct in_ifaddr *ifa;
-	int s_ip_idx, s_idx = cb->args[0];
+	struct hlist_head *head;
+	struct hlist_node *node;
 
-	s_ip_idx = ip_idx = cb->args[1];
-	idx = 0;
-	for_each_netdev(net, dev) {
-		if (idx < s_idx)
-			goto cont;
-		if (idx > s_idx)
-			s_ip_idx = 0;
-		in_dev = __in_dev_get_rtnl(dev);
-		if (!in_dev)
-			goto cont;
+	s_h = cb->args[0];
+	s_idx = idx = cb->args[1];
+	s_ip_idx = ip_idx = cb->args[2];
 
-		for (ifa = in_dev->ifa_list, ip_idx = 0; ifa;
-		     ifa = ifa->ifa_next, ip_idx++) {
-			if (ip_idx < s_ip_idx)
-				continue;
-			if (inet_fill_ifaddr(skb, ifa, NETLINK_CB(cb->skb).pid,
+	for (h = s_h; h < NETDEV_HASHENTRIES; h++, s_idx = 0) {
+		idx = 0;
+		head = &net->dev_index_head[h];
+		rcu_read_lock();
+		hlist_for_each_entry_rcu(dev, node, head, index_hlist) {
+			if (idx < s_idx)
+				goto cont;
+			if (idx > s_idx)
+				s_ip_idx = 0;
+			in_dev = __in_dev_get_rcu(dev);
+			if (!in_dev)
+				goto cont;
+
+			for (ifa = in_dev->ifa_list, ip_idx = 0; ifa;
+			     ifa = ifa->ifa_next, ip_idx++) {
+				if (ip_idx < s_ip_idx)
+					continue;
+				if (inet_fill_ifaddr(skb, ifa,
+					     NETLINK_CB(cb->skb).pid,
 					     cb->nlh->nlmsg_seq,
-					     RTM_NEWADDR, NLM_F_MULTI) <= 0)
-				goto done;
-		}
+					     RTM_NEWADDR, NLM_F_MULTI) <= 0) {
+					rcu_read_unlock();
+					goto done;
+				}
+			}
 cont:
-		idx++;
+			idx++;
+		}
+		rcu_read_unlock();
 	}
 
 done:
-	cb->args[0] = idx;
-	cb->args[1] = ip_idx;
+	cb->args[0] = h;
+	cb->args[1] = idx;
+	cb->args[2] = ip_idx;
 
 	return skb->len;
 }
#3 lore raw
Quoted 16 lines · On Thu, 12 Nov 2009 18:44:25 +0100 Eric Dumazet <[email protected]> wrote:
Quoted 6 lines · Stephen Hemminger
Quoted 3 lines · On Thu, 12 Nov 2009 15:11:36 +0100 Eric Dumazet <[email protected]> wrote:

When handling large number of netdevices, inet_dump_ifaddr() is very slow because it has O(N^2) complexity.

Quoted 8 lines
Quoted 4 lines

Instead of scanning one single list, we can use the NETDEV_HASHENTRIES sub lists of the dev_index hash table, and RCU lookups.

Signed-off-by Eric Dumazet <[email protected]>

You might be able to make RCU critical section smaller by moving it into loop.

Quoted 5 lines

Indeed. But we dump at most one skb (<= 8192 bytes ?), so rcu_read_lock holding time is small, unless we meet many netdevices without addresses. I wonder if its really common...

Thanks

One case where that might happen is:
modprobe dummy numdummies=10000

But dummy device should really be added with netlink, not at boot time.

author
#4 lore raw
Quoted 6 lines · Stephen Hemminger

One case where that might happen is:
modprobe dummy numdummies=10000

But dummy device should really be added with netlink, not at boot time.

At least we cannot exceed 32768 dummies :)

maintainer Applied
#5 lore raw
In reply to Eric Dumazet

From: Eric Dumazet <[email protected]>
Date: Thu, 12 Nov 2009 18:44:25 +0100

Quoted 10 lines

[PATCH net-next-2.6] ipv4: speedup inet_dump_ifaddr()

When handling large number of netdevices, inet_dump_ifaddr() is very slow because it has O(N2) complexity.

Instead of scanning one single list, we can use the NETDEV_HASHENTRIES sub lists of the dev_index hash table, and RCU lookups.

Signed-off-by Eric Dumazet <[email protected]>
Acked-by Stephen Hemminger <[email protected]>

Applied.