Testing interface removal speedup patches from Eric Dumazet.

These two patches look good to me. My benchmark has lots of extraneous overhead (process launch per action, primarily), so the numbers bounce around quite a bit on different runs, but it seems consistent that removing vlans is slightly faster, and removing VETH is much faster.

NOTE: for VETH, it's creating and dealing with pairs..so 500 veth means 1000 interfaces in the results below.

The script to generate these results is attached.

System is Atom N270, running Fedora 14. Kernel has HZ 1000, pre-empt, SMP, compiled for Pentium II.

With patches:

Created 500 veth in 17.874695 seconds (0.03574939 per interface). Added IP addresses in 23.306214 seconds (0.046612428 per addr). Deleted 500 veth in 17.023607 seconds. (0.034047214 per interface)

Created 1000 macvlan in 20.005472 seconds (0.020005472 per interface). Added IP addresses in 24.870693 seconds (0.024870693 per addr). Deleted 1000 macvlan in 44.899769 seconds. (0.044899769 per interface)

Created 1000 vlan in 21.343862 seconds (0.021343862 per interface). Added IP addresses in 24.088452 seconds (0.024088452 per addr). Deleted 1000 vlan in 42.623666 seconds. (0.042623666 per interface)

Without:

Created 500 veth in 22.507598 seconds (0.045015196 per interface). Added IP addresses in 28.637194 seconds (0.057274388 per addr). Deleted 500 veth in 34.998733 seconds. (0.069997466 per interface)

Created 1000 macvlan in 27.903848 seconds (0.027903848 per interface). Added IP addresses in 29.725436 seconds (0.029725436 per addr). Deleted 1000 macvlan in 58.552433 seconds. (0.058552433 per interface)

Created 1000 vlan in 33.733169 seconds (0.033733169 per interface). Added IP addresses in 29.755363 seconds (0.029755363 per addr). Deleted 1000 vlan in 54.858484 seconds. (0.054858484 per interface)

--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com

--On 9 May 2011 11:37:37 -0700 Ben Greear <[email protected]> wrote:

With:

Without:

Interesting. On my tests (albeit with CONFIG_HZ=100) I got:

100 pairs 500 pairs
Interface creation 14ms 110ms
Interface deletion 160ms 148ms

So Eric's patches help in the interface create case, even though there is no synchronize_net, sychronize_sched() or rcu_barrier() there.

I had assumed the slow create (which varies by number of pairs) was down to sysfs scalability only (see difference between 14ms and 110ms there).

Out of interest, if you still happen to have the scripts around, how fast is veth creation if you just do 100 pairs?

Alex Bligh

I'm not certain the create case is actually faster. Other runs on the patched kernel showed create to be much closer to the un-patched kernel.

The ratios to create/delete are more consistent it seems.

Created 100 veth in 2.779905 seconds (0.02779905 per interface). Added IP addresses in 2.280084 seconds (0.02280084 per addr). Deleted 100 veth in 3.988818 seconds. (0.03988818 per interface)

Thanks,
Ben

Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com

Hmmm... well you are getting *far* better linearity than me. Creating 500 interfaces is 8 times slower *per interface* than doing 500.

What occurs to me is that your box is faster than one of the ones I tested on, and you use CONFIG_HZ=100 but you get poorer results in absolute terms doing 100 (I see 14ms per interface). This with everything listenting to udev disabled? (so udevd dead, whatever executes your ifup/down scripts dead, unshare -n).

Alex Bligh

I use HZ of 1000, btw.

I killed udev, haldaemon..seemed to be just my stuff running.

I don't see any 'ifup' running with these things dead...

If you want to post your script, I can run it on my machine...

Thanks,
Ben

Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com

Ben,

See below - run (on ubuntu) by
service udev stop
unshare -n bash
./ifseq -n 500

The unshare seems sufficient to stop any remaining udev listeners getting wind of interface creation/deletion.

Alex Bligh


#!/bin/bash

# Usage:
#   ifaceseq [options]
#
# Options:
#   -n NUM : use NUM interfaces
#   -t TYPE : use TYPE of interfaces (supported: veth, vlan)

numifs=10
itype=veth

while getopts n:t: flag; do
   case ${flag} in
	n) numifs=${OPTARG} ;;
	t) itype=${OPTARG} ;;
   esac
done

shift $((OPTIND-1))

createifs ()
{
   echo `date` creating $numifs interfaces
   case ${itype} in
	vlan)
	    for i in `seq 1 $numifs` ; do
		ip link add link eth0 name vlan${i} type vlan id ${i}
	    done
	    ;;
	*)
	    for i in `seq 1 $numifs` ; do
		ip link add testa${i} type veth peer name testb${i}
	    done
   esac
   echo `date` done
}

deleteifs ()
{
   echo `date` deleting $numifs interfaces
   case ${itype} in
	vlan)
	    for i in `seq 1 $numifs` ; do
		ip link delete dev vlan${i}
	    done
	    ;;
	*)
	    for i in `seq 1 $numifs` ; do
		ip link delete testa${i}
	    done
   esac
   echo `date` done
}

time createifs;
time deleteifs;

You were not adding the 'up' flag
on creation...and my script was. I changed my script to not do up and it reports similar numbers to yours...

Also, deleting with IPs added is likely to be a lot more work than deleting w/out, as your script does.

Created 500 veth in 12.954143 seconds (0.025908286 per interface). Added IP addresses in 28.35679 seconds (0.05671358 per addr). Deleted 500 veth in 21.503877 seconds. (0.043007754 per interface)

Created 100 veth in 1.06471 seconds (0.0106471 per interface). Added IP addresses in 2.006418 seconds (0.02006418 per addr). Deleted 100 veth in 2.983303 seconds. (0.02983303 per interface)

From your script:

[root@lec2010-ath9k-1 lanforge]# /mnt/b32/greearb/tmp/testifs.bash -n 500 Mon May 9 13:53:31 PDT 2011 creating 500 interfaces
Mon May 9 13:53:44 PDT 2011 done

real 0m12.756s
user 0m0.732s
sys 0m11.943s
Mon May 9 13:53:44 PDT 2011 deleting 500 interfaces Mon May 9 13:54:02 PDT 2011 done

real 0m18.071s
user 0m1.002s
sys 0m8.376s
[root@lec2010-ath9k-1 lanforge]# /mnt/b32/greearb/tmp/testifs.bash -n 100 Mon May 9 13:54:33 PDT 2011 creating 100 interfaces
Mon May 9 13:54:34 PDT 2011 done

real 0m1.048s
user 0m0.092s
sys 0m0.943s
Mon May 9 13:54:34 PDT 2011 deleting 100 interfaces Mon May 9 13:54:36 PDT 2011 done

real 0m2.219s
user 0m0.134s
sys 0m0.794s

Thanks,
Ben

Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com