Skip to contentSkip to conversation

can't get cbq to traffic shape

LARTC Trevor Warren 2 participants 4 messages Jun 11, 2003 – Jun 12, 2003

lore
Share & tools
lore
#0 lore raw

Hello Folks,

For a long time i have tried to get Traffic Shaping work for my 500
Customer setup, but never succeded. I know i have made mistakes which i ain't able to point out.

I have made a heirarchical setup where in the incoming-outgoing
bandwidth is split into 10 different subnets and i have filters to divert traffic into these 10 different pipes. Then in each of these pipes i then have all the clients under those subnets.

Below at the end i have attached the CBQ errors thrown when i
initialise the TC pipes.

Any help and pointers on the same will be appreciated.

Trevor

#####################################
#Deleting and setting up
#Root qdisc's
#####################################
tc qdisc del dev eth0 root
tc qdisc add dev eth0 root handle 1:0 cbq bandwidth 100Mbit avpkt 1000 cell 8

#######################################
#Setting up top 10 qdiscs
#for initial packet classification
#######################################
tc class add dev eth0 parent 1:0 classid 1:1 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.10.0/24 classid 1:1

tc class add dev eth0 parent 1:0 classid 1:2 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.11.0/24 classid 1:2

tc class add dev eth0 parent 1:0 classid 1:3 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.12.0/24 classid 1:3

tc class add dev eth0 parent 1:0 classid 1:4 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.13.0/24 classid 1:4

tc class add dev eth0 parent 1:0 classid 1:5 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.14.0/24 classid 1:5

tc class add dev eth0 parent 1:0 classid 1:6 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.15.0/24 classid 1:6

tc class add dev eth0 parent 1:0 classid 1:7 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.16.0/24 classid 1:7

tc class add dev eth0 parent 1:0 classid 1:8 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.17.0/24 classid 1:8

tc class add dev eth0 parent 1:0 classid 1:9 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.18.0/24 classid 1:9

tc class add dev eth0 parent 1:0 classid 1:10 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.19.0/24 classid 1:10

tc class add dev eth0 parent 1:0 classid 1:11 cbq bandwidth 10Mbit rate 1Mbit allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.20.0/24 classid 1:11

#################################
# Start of Actual Filters
#################################

tc class add dev eth0 parent 1:1 classid 1:12 cbq bandwidth 10Mbit rate 32Kbit prio 4 allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip prio 4 u32 match ip src 192.168.10.1/0 classid 1:12

# -- -- --

tc class add dev eth0 parent 1:1 classid 1:13 cbq bandwidth 10Mbit rate 32Kbit prio 4 allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip prio 4 u32 match ip src 192.168.10.2/0 classid 1:13

# -- -- --

tc class add dev eth0 parent 1:1 classid 1:14 cbq bandwidth 10Mbit rate 32Kbit prio 4 allot 1514 cell 8 maxburst 20 avpkt 1000
tc filter add dev eth0 parent 1:0 protocol ip prio 4 u32 match ip src 192.168.10.3/0 classid 1:14

#######################
# CBQ Dump
#######################

[root@tuxmania tc_server_code]# dmesg
Q: class 00010152 has bad quantum=0, repaired.
CBQ: class 00010143 has bad quantum=0, repaired.
CBQ: class 00010134 has bad quantum=0, repaired.
CBQ: class 00010125 has bad quantum=0, repaired.
CBQ: class 00010116 has bad quantum=0, repaired.
CBQ: class 00010107 has bad quantum=0, repaired.
CBQ: class 00010071 has bad quantum=0, repaired.
CBQ: class 00010060 has bad quantum=0, repaired.

Signature · 9 lines
( >-    GNU/LINUX, It's all about CHOICE      -< )
/~\    __  [email protected]  __   /~\
|  \) /  Pre Sales Consultant - Red Hat     \ (/ |
|_|_  \    9820349221(M) | 22881326(O)      / _|_|
       \___________________________________/

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#1 lore raw
Quoted 11 lines · Trevor Warren

Hello Folks,

For a long time i have tried to get Traffic Shaping work for my 500
Customer setup, but never succeded. I know i have made mistakes which i ain't able to point out.

I have made a heirarchical setup where in the incoming-outgoing
bandwidth is split into 10 different subnets and i have filters to divert traffic into these 10 different pipes. Then in each of these pipes i then have all the clients under those subnets.

Quoted 4 lines

Below at the end i have attached the CBQ errors thrown when i
initialise the TC pipes.

Any help and pointers on the same will be appreciated.

The bandwidth _MUST_ be your NIC bandwidth. So it's 100Mbit for a 100mbit NIC. You combined 100Mbit and 10Mbit.
You also have to provide a weight paramter. Take weight = rate / 10. I think this will remove the quantum errors.

Some general quesions :
What's you link bandwidth you want to shape? It's best if you add 1 bounded class to the root qdisc wih rate = link bandwidth and attach all other classes to that class (and not to the root qdisc).
Is this script used on a firewall? And is eth0 connected to the internet? Then you have a filter problem. You filter on the src address, but all connections are natted so the real src address is not known.

Stef

Signature · 8 lines
[email protected]
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#2 lore raw
In reply to Trevor Warren

Hello Stef,

Thanks a million for the information. Am sending you the modified tc
script that i worked on. This in essence have 2 filters for the incoming and outgoing packets to 192.168.*.*.

#!/bin/bash

#####################################
#Deleting and setting up
#Root qdisc's
#####################################
tc qdisc del dev eth0 root
tc qdisc add dev eth0 root handle 1:0 cbq bandwidth 10Mbit avpkt 1000 cell 8

#######################################
#Setting up top 10 qdiscs
#for initial packet classification
#######################################
tc class add dev eth0 parent 1:0 classid 1:1 cbq bandwidth 10Mbit rate 200Kbit allot 1514 cell 8 maxburst 20 avpkt 1000 weight 20Kbit bounded tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.10.0/24 classid 1:1
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip dst 192.168.10.0/24 classid 1:1

tc class add dev eth0 parent 1:0 classid 1:2 cbq bandwidth 10Mbit rate 200Kbit allot 1514 cell 8 maxburst 20 avpkt 1000 weight 20Kbit bounded tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.11.0/24 classid 1:2
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip dst 192.168.11.0/24 classid 1:2

tc class add dev eth0 parent 1:0 classid 1:3 cbq bandwidth 10Mbit rate 200Kbit allot 1514 cell 8 maxburst 20 avpkt 1000 weight 20Kbit bounded tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.12.0/24 classid 1:3
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip dst 192.168.12.0/24 classid 1:3

:
:
:
:

####################################
#Actual Filters
####################################

Quoted 30 lines · Stef Coene
Quoted 11 lines · Trevor Warren

Hello Folks,

For a long time i have tried to get Traffic Shaping work for my 500
Customer setup, but never succeded. I know i have made mistakes which i ain't able to point out.

I have made a heirarchical setup where in the incoming-outgoing
bandwidth is split into 10 different subnets and i have filters to divert traffic into these 10 different pipes. Then in each of these pipes i then have all the clients under those subnets.

Quoted 4 lines

Below at the end i have attached the CBQ errors thrown when i
initialise the TC pipes.

Any help and pointers on the same will be appreciated.

The bandwidth _MUST_ be your NIC bandwidth. So it's 100Mbit for a 100mbit NIC. You combined 100Mbit and 10Mbit.
You also have to provide a weight paramter. Take weight = rate / 10. I think this will remove the quantum errors.

Some general quesions :
What's you link bandwidth you want to shape? It's best if you add 1 bounded class to the root qdisc wih rate = link bandwidth and attach all other classes to that class (and not to the root qdisc).
Is this script used on a firewall? And is eth0 connected to the internet? Then you have a filter problem. You filter on the src address, but all connections are natted so the real src address is not known.

Stef

Signature · 9 lines
( >-    GNU/LINUX, It's all about CHOICE      -< )
/~\    __  [email protected]  __   /~\
|  \) /  Pre Sales Consultant - Red Hat     \ (/ |
|_|_  \    9820349221(M) | 22881326(O)      / _|_|
       \___________________________________/

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#3 lore raw
In reply to Trevor Warren

Hello Stef,

Thanks a million for the information but things still dont work. Am sending you the modified tc script that i worked on. This in essence has 2 filters for the incoming
and outgoing packets to 192.168.*.*.

Each location has a 3 server setup.

192.168.0.100 : Billing server (eth0)
x.x.x.x : External IP (eth1)

Other servers :-
192.168.0.101 : Cache server for Network 1
192.168.0.102 : Cache server for Network 2

On the gateway these scripts don't work at all(In terms of Traffic Shaping). Clients are still able
to download more than 40Kbit as suggested in the scripts.

Eth0 is on an internal LAN while eth1, eth2 are external ip addresses. The rule sets in tc have been made such as to capture traffic and put them between the classes.

This apart, Natting will take place only on the external ethernet adapter right??. Since the Internal
NIC eth0 doesn't have.any Natted conditions it should be easier for it to shape traffic. That apart, one last but important issue, i am re-directing all web traffic from this gateway load balanced to the other 2 caching boxes. Can this be an issue??

Awaiting your inputs on the same. Please help me resolve this. You efforts on the same will be appreciated.

#!/bin/bash

#####################################
#Deleting and setting up
#Root qdisc's
#####################################
tc qdisc del dev eth0 root
tc qdisc add dev eth0 root handle 1:0 cbq bandwidth 10Mbit avpkt 1000 cell 8

#######################################
#Setting up top 10 qdiscs
#for initial packet classification
#######################################
tc class add dev eth0 parent 1:0 classid 1:1 cbq bandwidth 10Mbit rate 200Kbit allot 1514 cell 8 maxburst 20 avpkt 1000 weight 20Kbit bounded tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.10.0/24 classid 1:1
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip dst 192.168.10.0/24 classid 1:1

tc class add dev eth0 parent 1:0 classid 1:2 cbq bandwidth 10Mbit rate 200Kbit allot 1514 cell 8 maxburst 20 avpkt 1000 weight 20Kbit bounded tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.11.0/24 classid 1:2
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip dst 192.168.11.0/24 classid 1:2

tc class add dev eth0 parent 1:0 classid 1:3 cbq bandwidth 10Mbit rate 200Kbit allot 1514 cell 8 maxburst 20 avpkt 1000 weight 20Kbit bounded tc filter add dev eth0 parent 1:0 protocol ip u32 match ip src 192.168.12.0/24 classid 1:3
tc filter add dev eth0 parent 1:0 protocol ip u32 match ip dst 192.168.12.0/24 classid 1:3

:
:
:
:

####################################
#Actual Filters
####################################

#################################
# Start of Actual Filters
#################################

tc class add dev eth0 parent 1:1 classid 1:13 cbq bandwidth 10Mbit rate 40Kbit weight 4Kbit prio 4 allot 1514 cell 8 maxburst 20 avpkt 1000 bounded tc filter add dev eth0 parent 1:1 protocol ip u32 match ip src 192.168.10.1/32 classid 1:13
tc filter add dev eth0 parent 1:1 protocol ip u32 match ip dst 192.168.10.1/32 classid 1:13

tc class add dev eth0 parent 1:1 classid 1:14 cbq bandwidth 10Mbit rate 40Kbit weight 4Kbit prio 4 allot 1514 cell 8 maxburst 20 avpkt 1000 bounded tc filter add dev eth0 parent 1:1 protocol ip u32 match ip src 192.168.10.2/32 classid 1:14
tc filter add dev eth0 parent 1:1 protocol ip u32 match ip dst 192.168.10.2/32 classid 1:14
:
:
:
:
:
:

Trevor

Quoted 30 lines · Stef Coene
Quoted 11 lines · Trevor Warren

Hello Folks,

For a long time i have tried to get Traffic Shaping work for my 500
Customer setup, but never succeded. I know i have made mistakes which i ain't able to point out.

I have made a heirarchical setup where in the incoming-outgoing
bandwidth is split into 10 different subnets and i have filters to divert traffic into these 10 different pipes. Then in each of these pipes i then have all the clients under those subnets.

Quoted 4 lines

Below at the end i have attached the CBQ errors thrown when i
initialise the TC pipes.

Any help and pointers on the same will be appreciated.

The bandwidth _MUST_ be your NIC bandwidth. So it's 100Mbit for a 100mbit NIC. You combined 100Mbit and 10Mbit.
You also have to provide a weight paramter. Take weight = rate / 10. I think this will remove the quantum errors.

Some general quesions :
What's you link bandwidth you want to shape? It's best if you add 1 bounded class to the root qdisc wih rate = link bandwidth and attach all other classes to that class (and not to the root qdisc).
Is this script used on a firewall? And is eth0 connected to the internet? Then you have a filter problem. You filter on the src address, but all connections are natted so the real src address is not known.

Stef

Signature · 9 lines
( >-    GNU/LINUX, It's all about CHOICE      -< )
/~\    __  [email protected]  __   /~\
|  \) /  Pre Sales Consultant - Red Hat     \ (/ |
|_|_  \    9820349221(M) | 22881326(O)      / _|_|
       \___________________________________/

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/