Skip to contentSkip to conversation

big problem with HTB/CBQ and CPU for more than 1.700

LARTC Pablo Fernandes Yahoo 2 participants 2 messages May 28, 2007 – May 29, 2007

lore
Share & tools
lore
#0 lore raw

Hey,

I have some links that you could be interested on them:

http://www.geocities.com/asimshankar/notes/linux-networking-code.txt

http://oss.sgi.com/archives/netdev/2004-06/msg00162.html

http://www.spec.org/osg/web99/results/res2003q3/web99-20030818-00245.html

http://www.intel.com/support/network/sb/cs-009209.htm

All of them talks about the interrupts.

So, what do you think should i do with my e1000? What do you think could be the best board for sites as 8.000 customers? My problem is exact these lots of interruptions.

Thank you in advance!

Regards

Quoted 5 lines

I have classes configured by protocol..

My NIC is the e1000 too (default parameters), but I'm using off-board cards

(with scalable i/o support - good for multi->core setups). The main problem I had was the number of interruptions (~ 10.000/s)...

Quoted 1 line

With IRQ balance activated, each NIC got used with a specific processor

core, and the use of CPU on these cores was always >100%.

Quoted 3 lines

I've solved this problem in my setup (2 Quad-Cores), deactivating the IRQ

balance, that caused the interruptions to be >processed by all the

Quoted 5 lines

8 cores.

Are you doing p2p control on this server? This is usually what takes more

CPU usage..

_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

#1 lore raw

Re: [LARTC] Re: big problem with HTB/CBQ and CPU for more than

Quoted 3 lines

So, what do you think should i do with my e1000? What do you think could be the best board for sites as 8.000 customers? My problem is exact these lots of interruptions.

Plug as many network interfaces (e1000) as cpu cores you have. E1000 multiport nics have separate irq assigned to each "port", so having 2 x Quad-Core Xeon and 2 x 4-port e1000 would allow you to configure static affinity of each port to one core:
http://bcr2.uwaterloo.ca/~brecht/servers/apic/SMP-affinity.txt

Sometimes symmetric usage of network interfaces (for symmetric core usage) is the problem. I think you can achieve it by plugging all 8 ports to managed switch and configuring some form of aggregation. The best would be src/dst IP EtherChannel or something similar. For some deployments (where router sees all the clients on OSI layer 2) src/dst MAC EtherChannel would suffice. On linux side you would have to configure bonding:
http://linux-net.osdl.org/index.php/Bonding

pozdrawiam,
Marek Kierdelewicz
KoBa ISP
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc