performance
Hi,
I'm testing netfilter performance here on PIII 1.2GHz based systems. With default kernel configuration, netfilter is able to process 85,000 pps with 125 rules (all rules matching).
Note the application is just counting. There is no transmitting/forwarding.
Also note the nics are e100.
So my simple question are there any tips in increasing the performance? Hmm actually the performance seems optimal? is it only taking 9 instructions per match? 1.2*10^9/(85000*1500) = 9
thanks,
Pádraig.
Quoted 11 lines · [email protected] wrote:
Hi,
I'm testing netfilter performance here on PIII 1.2GHz based systems. With default kernel configuration, netfilter is able to process 85,000 pps with 125 rules (all rules matching).
Note the application is just counting. There is no transmitting/forwarding.
Quoted 7 lines
Also note the nics are e100.
So my simple question are there any tips in increasing the performance? Hmm actually the performance seems optimal? is it only taking 9 instructions per match? 1.2*10^9/(85000*1500) = 9
I knew that couldn't be right.
That was tested on a dual 1.2GHz,
so that should be approx:
2*10^9/(85000*125) = 188 instructions per match.
I guess that's pretty optimal? The best I could hope for after that would be to increase the rx packet buffer space so as to handle higher spikes than this.
cheers,
Pádraig.
Monday, June 9, 2003, 7:16:35 PM, you wrote:
Quoted 11 lines · Pdc> [email protected] wrote:
Quoted 11 lines
Hi,
I'm testing netfilter performance here on PIII 1.2GHz based systems. With default kernel configuration, netfilter is able to process 85,000 pps with 125 rules (all rules matching).
Note the application is just counting. There is no transmitting/forwarding.
Quoted 7 lines
Quoted 7 lines
Also note the nics are e100.
So my simple question are there any tips in increasing the performance? Hmm actually the performance seems optimal? is it only taking 9 instructions per match? 1.2*10^9/(85000*1500) = 9
Pdc> I knew that couldn't be right.
Pdc> That was tested on a dual 1.2GHz,
Pdc> so that should be approx:
Pdc> 2*10^9/(85000*125) = 188 instructions per match.
I am afraid that is not instructions per match but ticks per match? An instruction can take more than one tick (clock cycle).
P.Krumins
Quoted 24 lines · Peteris Krumins
Monday, June 9, 2003, 7:16:35 PM, you wrote:
Quoted 11 lines · Pdc> [email protected] wrote:
Quoted 11 lines
Hi,
I'm testing netfilter performance here on PIII 1.2GHz based systems. With default kernel configuration, netfilter is able to process 85,000 pps with 125 rules (all rules matching).
Note the application is just counting. There is no transmitting/forwarding.
Quoted 7 lines
Quoted 7 lines
Also note the nics are e100.
So my simple question are there any tips in increasing the performance? Hmm actually the performance seems optimal? is it only taking 9 instructions per match? 1.2*10^9/(85000*1500) = 9
Quoted 8 lines
Pdc> I knew that couldn't be right.
Pdc> That was tested on a dual 1.2GHz,
Pdc> so that should be approx:
Pdc> 2*10^9/(85000*125) = 188 instructions per match.I am afraid that is not instructions per match but ticks per match? An instruction can take more than one tick (clock cycle).
yes true. I was making assumptions.
The following suggests the average cycles:instruction ratio is 1.68 I think:
http://www.cs.berkeley.edu/~pattrsn/252S01/Lec18-dynamic3.pdf
Looking again at the system it's actually 2 x 1.4GHz So the instructions per match are about:
((2.8*10^9)/1.68)/(85000*125) = 156
With the overhead associated with SMP/kernel/... this suggests a figure closer to 120? I'm very impressed.
So to scale I would have to organise the rules into chains that could be bypassed. I wonder is there any projects to do this automatically? hmm..
Pádraig.
Quoted 5 lines · On Tue, Jun 10, 2003 at 09:49:54AM +0100, [email protected] wrote:
yes true. I was making assumptions.
The following suggests the average cycles:instruction ratio is 1.68 I think:
http://www.cs.berkeley.edu/~pattrsn/252S01/Lec18-dynamic3.pdf
Quoted 7 lines
Looking again at the system it's actually 2 x 1.4GHz So the instructions per match are about:
((2.8*10^9)/1.68)/(85000*125) = 156With the overhead associated with SMP/kernel/... this suggests a figure closer to 120? I'm very impressed.
I don't know what kind of weird calculation that would be. First of all, why the hack are you adding up the clock rates of your CPU's? Do you have any idea how SMP systems work?
And then, why do you multiply the pps rate with the size of the packets? do you think we process every byte of a packet individually?
Let's make a simple calculation for the UP case:
average clock ticks per processed packet:
1.4*10^9/85000 = 16470
In order to get any idea about how those clock ticks are distributed among the various pars of kernel networking, you need to use some means of profiling.
I advise you to dig into profiling, and exploit all general kernel networking optimizations (like NAPI, ...) before starting to think about optimizing iptables.
Quoted 3 lines
So to scale I would have to organise the rules into chains that could be bypassed. I wonder is there any projects to do this automatically? hmm..
no.
Quoted 1 line
Pádraig.
Signature · 5 lines
- Harald Welte <[email protected]> http://www.netfilter.org/ ============================================================================ "Fragmentation is like classful addressing -- an interesting early architectural error that shows how much experimentation was going on while IP was being designed." -- Paul Vixie
Quoted 3 lines · Harald Welte
In order to get any idea about how those clock ticks are distributed among the various pars of kernel networking, you need to use some means of profiling.
iptables is in modules so I can't profile it at the moment. But this is informative. This is with 160 match any rules in the mangle::PREROUTING chain and then the packets are just dropped after. Packet rate again is 85Kpps.
14754 total 0.0096
11421 default_idle 142.7625 (71% !!)
537 handle_IRQ_event 3.3563
341 eth_type_trans 1.7760
298 ip_rcv 0.4233
238 do_gettimeofday 1.8594
224 netif_rx 0.5185
217 add_timer_randomness 0.9042
196 skb_release_data 1.3611
180 batch_entropy_store 1.0227
162 alloc_skb 0.3375
157 process_backlog 0.5164
137 kfree 0.7784
126 __kmem_cache_alloc 0.3937
109 netif_receive_skb 0.2004
85 nf_hook_slow 0.1771
66 kmalloc 0.8250
61 kfree_skbmem 0.4766
50 __constant_c_and_count_memset 0.3125
36 __kfree_skb 0.0978
28 nf_iterate 0.1750
16 get_sample_stats 0.1250
15 add_entropy_words 0.0852
14 ip_promisc_rcv_finish 0.2917
10 kmem_cache_free 0.0625
8 add_interrupt_randomness 0.1250
6 __generic_copy_to_user 0.0625
4 schedule 0.0030
2 net_rx_action 0.0057
2 do_softirq 0.0089The main question is why all the idle time?
Note userspace is locked out at this packet rate.
system specs:
dual 1.4GHz PIII
e1000 nic
kernel 2.4.20
Pádraig.