Skip to contentSkip to conversation

netfilter performance dependent on arch

DISCUSSION Marek Kierdelewicz 3 participants 5 messages Feb 7, 2012

lore
Share & tools
lore
#0 lore raw

Hi,

Can anyone point me to some performance comparison of netfilter on i686 and x86_64? I have a few linux routers doing a lot of firewalling and QoS. Currently those routers use i686 arch on 64-bit hardware. Would I notice any performance gain after moving to 64-bit kernel?

Next question. On some routers I don't need statefull firewall at all and I have NOTRACT as a default rule in raw netfilter table. What is the expected performance gain if I would fully disable conntrack instead of using NOTRACK target? What would be the best approach to do it on debian squeeze distribution kernel (nf_conntrack compiled as a module)? Is blacklisting nf_conntrack module enough (to be safe in case of accidental addition of statefull rule)?

best regards,
Marek Kierdelewicz

#1 lore raw
Quoted 6 lines · Marek Kierdelewicz

Hi,

Can anyone point me to some performance comparison of netfilter on i686 and x86_64? I have a few linux routers doing a lot of firewalling and QoS. Currently those routers use i686 arch on 64-bit hardware. Would I notice any performance gain after moving to 64-bit kernel?

Apart from the obvious "Try it on your workload and see." suggestion, you could I suppose make the assumption that a netfilter workload looks similar to one or more components of something like SPECint_rate_base2006 or SPECint_base2006 and mine the results on www.spec.org for some comparisons between 32 and 64 bit compiles on the same chip(s). Probably want something with lots of control flow (term?) rather than loops and such. A high-level description of the integer components can be found at http://www.spec.org/cpu2006/CINT2006/ . One of my first guesses would be 400.perlbench and 403.gcc but definitely peruse the descriptions of all of them.

rick jones

#2 lore raw
Quoted 9 lines
Quoted 5 lines

Can anyone point me to some performance comparison of netfilter on i686 and x86_64? I have a few linux routers doing a lot of firewalling and QoS. Currently those routers use i686 arch on 64-bit hardware. Would I notice any performance gain after moving to 64-bit kernel?

Apart from the obvious "Try it on your workload and see." suggestion, you could I suppose make the assumption that a netfilter workload looks similar to one or more components of something like

Thanks for the response. Anyway moments ago I found answer to the first question on vyatta forum here:
http://www.vyatta.org/forum/viewtopic.php?t=823&sid=3cec85f2cf7866ca26040ec491415ee1

shemminger "The researchers at Uppsla University who are measuring 10G routing performance found that 64bit kernel is slower than 32bit kernel for routing. Most likely the increased code size caused a larger number of CPU cache misses. 64 bit would be an advantage if you are trying to run lots of services."

DaveRoberts "This result is actually not too surprising. Networking code is typically very cache sensitive. This gets worse with 64-bit. If you're running code that uses data structures with lots of pointers (which networking code often has) and you don't needed the expanded address space, then you're wasting half your d-cache with 32 upper bits of zeros. While the additional registers are a benefit, they aren't enough to make up for the increased cache miss rate.

But as you say, without knowing the compiler, optimizer, and instrumenting the heck out of the code with all the hardware analysis counters, it's difficult to say what's really going on. All that is on our list of things to do over time. The main takeaway is that it's definitely not a slam dunk one way or another."

Best regards,
Marek Kierdelewicz

#3 lore raw
Quoted 6 lines · On Tue, 7 Feb 2012 19:54:11 +0100 Marek Kierdelewicz <[email protected]> wrote:
Quoted 5 lines
Quoted 5 lines

Can anyone point me to some performance comparison of netfilter on i686 and x86_64? I have a few linux routers doing a lot of firewalling and QoS. Currently those routers use i686 arch on 64-bit hardware. Would I notice any performance gain after moving to 64-bit kernel?

Quoted 7 lines
Quoted 3 lines

Apart from the obvious "Try it on your workload and see." suggestion, you could I suppose make the assumption that a netfilter workload looks similar to one or more components of something like

Thanks for the response. Anyway moments ago I found answer to the first question on vyatta forum here:
http://www.vyatta.org/forum/viewtopic.php?t=823&sid=3cec85f2cf7866ca26040ec491415ee1

That information is out of date.
The bulk of the slow down in 64 bit was due to the fact that IOMMU (CONFIG_DMAR) was enabled. The IOMMU requires programming DMA registers on each packet which explains the extra overhead. After disabling DMA remapping, the 64 bit kernel is faster because the compiler has more registers and can generate better code.

#4 lore raw
Quoted 6 lines

That information is out of date.
The bulk of the slow down in 64 bit was due to the fact that IOMMU (CONFIG_DMAR) was enabled. The IOMMU requires programming DMA registers on each packet which explains the extra overhead. After disabling DMA remapping, the 64 bit kernel is faster because the compiler has more registers and can generate better code.

Thanks a lot. That's very valuable info.

Best regards,
Marek Kierdelewicz