Skip to contentSkip to conversation

tc limit

LARTC Martin Schiøtz 2 participants 4 messages Jan 19, 2005 – Jan 20, 2005

lore
Share & tools
lore
#0 lore raw

Hi

I have been playing with tc. I'm using htb and sfq. I tried to setup shaping of every ip address within a /20.
I found that there must be an upper limit of about 2096 on the number of rules (classes, qdiscs or filters). I was testing a /20 which gives 4096 classes, qdiscs and filters. I'm using hashkey to optimize processing of frames.

When executing the script below - it stops when adding rules for "ip 10.0.7.253" with this error:

handle 1:4097
handle 1:4099
handle 1:4101
RTNETLINK answers: File exists
We have an error talking to the kernel
handle 1:4103
RTNETLINK answers: File exists
We have an error talking to the kernel
handle 1:4105
...
...
etc.

Is there a limit with tc or I'm I doing something wrong ? (I'm using iproute2-2.6.9-041019)

This is the script:
#!/bin/sh

/sbin/tc qdisc add dev eth1 root handle 1: htb
/sbin/tc class add dev eth1 parent 1: classid 1:3 htb rate 34mbit ceil 34mbit burst 0kbit cburst 0kbit
/sbin/tc filter add dev eth1 parent 1:3 protocol ip u32
/sbin/tc filter add dev eth1 parent 1:3 handle 3: protocol ip u32 divisor 256 /sbin/tc filter add dev eth1 protocol ip parent 1: u32 match ip dst 10.0.0.0/20 hashkey mask 0x000000ff at 16 link 3:

echo handle 1:13
/sbin/tc class add dev eth1 parent 1:3 classid 1:13 htb rate 512kbit ceil 512kbit burst 0k cburst 0k
/sbin/tc qdisc add dev eth1 parent 1:13 handle 13: sfq perturb 4 /sbin/tc filter add dev eth1 protocol ip parent 1:3 u32 ht 3:1: match ip dst 10.0.0.1 flowid 1:13

echo handle 1:15
/sbin/tc class add dev eth1 parent 1:3 classid 1:15 htb rate 512kbit ceil 512kbit burst 0k cburst 0k
/sbin/tc qdisc add dev eth1 parent 1:15 handle 15: sfq perturb 4 /sbin/tc filter add dev eth1 protocol ip parent 1:3 u32 ht 3:2: match ip dst 10.0.0.2 flowid 1:15
...
...
echo handle 1:4101
/sbin/tc class add dev eth1 parent 1:3 classid 1:4101 htb rate 512kbit ceil 512kbit burst 0k cburst 0k
/sbin/tc qdisc add dev eth1 parent 1:4101 handle 4101: sfq perturb 4 /sbin/tc filter add dev eth1 protocol ip parent 1:3 u32 ht 3:fd: match ip dst 10.0.7.253 flowid 1:4101
...
...
echo handle 1:8201
/sbin/tc class add dev eth1 parent 1:3 classid 1:8201 htb rate 512kbit ceil 512kbit burst 0k cburst 0k
/sbin/tc qdisc add dev eth1 parent 1:8201 handle 8201: sfq perturb 4 /sbin/tc filter add dev eth1 protocol ip parent 1:3 u32 ht 3:ff: match ip dst 10.0.15.255 flowid 1:8201
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#1 lore raw
Quoted 2 lines · Martin Schiøtz

Hi

Hello.

Quoted 6 lines

I have been playing with tc. I'm using htb and sfq. I tried to setup shaping of every ip address within a /20.
I found that there must be an upper limit of about 2096 on the number of rules (classes, qdiscs or filters). I was testing a /20 which gives 4096 classes, qdiscs and filters. I'm using hashkey to optimize processing of frames.

Filters.

Quoted 1 line

Is there a limit with tc or I'm I doing something wrong ?

Yes, sort of. Yes.

Quoted 5 lines

This is the script:
#!/bin/sh

/sbin/tc filter add dev eth1 parent 1:3 protocol ip u32

'prio #' must be included in all 'filter' lines where "#" is not zero and is always the same; the "zero prio" has a limit and there is a limit on '#' - but no limit on how many times '#' can be (e.g.) 5

So, e.g.:
tc filter add dev eth1 parent 1: protocol ip prio 5 \
u32 match ip src $IP flowid 1:40

will run at least 64K times without complaint, but this will fail after ~2K:
tc filter add dev eth1 parent 1: protocol ip prio 0 \
u32 match ip src $IP flowid 1:40

Quoted 3 lines

/sbin/tc filter add dev eth1 parent 1:3 handle 3: protocol ip u32 divisor 256 /sbin/tc filter add dev eth1 protocol ip parent 1: u32 match ip dst 10.0.0.0/20 hashkey mask 0x000000ff at 16 link 3:

google LARTC please document

gypsy
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#2 lore raw
In reply to Martin Schiøtz

Thanks, with "prio #" on all filters - all rules can be added. :-)

Just curious - Is there a reason for the ~2K limit on filters with "prio 0"?

Martin

Quoted 38 lines · gypsy
Quoted 2 lines · Martin Schiøtz

Hi

Hello.

Quoted 6 lines

I have been playing with tc. I'm using htb and sfq. I tried to setup shaping of every ip address within a /20.
I found that there must be an upper limit of about 2096 on the number of rules (classes, qdiscs or filters). I was testing a /20 which gives 4096 classes, qdiscs and filters. I'm using hashkey to optimize processing of frames.

Filters.

Quoted 1 line

Is there a limit with tc or I'm I doing something wrong ?

Yes, sort of. Yes.

Quoted 5 lines

This is the script:
#!/bin/sh

/sbin/tc filter add dev eth1 parent 1:3 protocol ip u32

'prio #' must be included in all 'filter' lines where "#" is not zero and is always the same; the "zero prio" has a limit and there is a limit on '#' - but no limit on how many times '#' can be (e.g.) 5

So, e.g.:
tc filter add dev eth1 parent 1: protocol ip prio 5 \
u32 match ip src $IP flowid 1:40

will run at least 64K times without complaint, but this will fail after ~2K:
tc filter add dev eth1 parent 1: protocol ip prio 0 \
u32 match ip src $IP flowid 1:40

Quoted 8 lines
Quoted 3 lines

/sbin/tc filter add dev eth1 parent 1:3 handle 3: protocol ip u32 divisor 256 /sbin/tc filter add dev eth1 protocol ip parent 1: u32 match ip dst 10.0.0.0/20 hashkey mask 0x000000ff at 16 link 3:

google LARTC please document

gypsy

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#3 lore raw
In reply to Martin Schiøtz
Quoted 3 lines · Martin Schiøtz

Thanks, with "prio #" on all filters - all rules can be added. :-)

You're welcome.

Quoted 3 lines

Just curious - Is there a reason for the ~2K limit on filters with "prio 0"?

Martin

No reason I'm aware of. Probably some OOPS! in the code that creates filters. Since I've never bothered to learn C, I've also never bothered to read the source code. So I'd never spot a programming mistake no matter how obvious, and I suspect that this one is very subtle.

The real crime here is that the docs do not tell you that there is this trap that "everyone" with a lot of users falls into.

Quoted 39 lines · gypsy
Quoted 38 lines
Quoted 2 lines · Martin Schiøtz

Hi

Hello.

Quoted 6 lines

I have been playing with tc. I'm using htb and sfq. I tried to setup shaping of every ip address within a /20.
I found that there must be an upper limit of about 2096 on the number of rules (classes, qdiscs or filters). I was testing a /20 which gives 4096 classes, qdiscs and filters. I'm using hashkey to optimize processing of frames.

Filters.

Quoted 1 line

Is there a limit with tc or I'm I doing something wrong ?

Yes, sort of. Yes.

Quoted 5 lines

This is the script:
#!/bin/sh

/sbin/tc filter add dev eth1 parent 1:3 protocol ip u32

'prio #' must be included in all 'filter' lines where "#" is not zero and is always the same; the "zero prio" has a limit and there is a limit on '#' - but no limit on how many times '#' can be (e.g.) 5

So, e.g.:
tc filter add dev eth1 parent 1: protocol ip prio 5 \
u32 match ip src $IP flowid 1:40

will run at least 64K times without complaint, but this will fail after ~2K:
tc filter add dev eth1 parent 1: protocol ip prio 0 \
u32 match ip src $IP flowid 1:40

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

/sbin/tc filter add dev eth1 parent 1:3 handle 3: protocol ip u32 divisor 256 /sbin/tc filter add dev eth1 protocol ip parent 1: u32 match ip dst 10.0.0.0/20 hashkey mask 0x000000ff at 16 link 3:

google LARTC please document

gypsy

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/