tc and priority

Hi all,
I'm studying traffic shaping and I need some informations.
I studied HTB and I had no problem to limit bandwidth (with and without borrowing). Then I would undestand how assign different priorities to
packets: I test qdisc PRIO and prio paramenter in CBQ (and HTB) but without success.

with CBQ, I create 2 two classes with different priority and I filter traffic to send different traffic (source port) to a particular queue. Then I run 2 connections having different priority and I see that a connection stalls and the other uses all bandwidth then it succeeds the opposite.

The same behaviour I see using PRIO qdisc.

Instead, I would get that che higher priority connection uses all bandwidth until its queue is full (so che lower priority connection waits until the other finish).
What's wrong? Suggestions?

Thanks,
Fabio

Hi Fabio,

You should do something like this:

tc qdisc add dev eth0 root handle 1: prio bands 3 tc qdisc add dev eth0 parent 1:1 handle 2 sfq perturb 10 tc qdisc add dev eth0 parent 1:2 handle 3 sfq perturb 10 tc qdisc add dev eth0 parent 1:3 handle 4 sfq perturb 10 tc filter add dev eth0 protocol ip parent 1: prio 1 <your criteria_high> flowid 1:1
tc filter add dev eth0 protocol ip parent 1: prio 2 <your criteria_middle> flowid 1:2
tc filter add dev eth0 protocol ip parent 1: prio 3 <your criteria_low> flowid 1:3

That works in my system.

Thanks for your reply.
I have just testing your script and I have some questions...

Quoted 4 lines · Anatoly Muliarski

Hi Fabio,

You should do something like this:

Quoted 11 lines

tc qdisc add dev eth0 root handle 1: prio bands 3 tc qdisc add dev eth0 parent 1:1 handle 2 sfq perturb 10 tc qdisc add dev eth0 parent 1:2 handle 3 sfq perturb 10 tc qdisc add dev eth0 parent 1:3 handle 4 sfq perturb 10 tc filter add dev eth0 protocol ip parent 1: prio 1 <your criteria_high> flowid 1:1
tc filter add dev eth0 protocol ip parent 1: prio 2 <your criteria_middle> flowid 1:2
tc filter add dev eth0 protocol ip parent 1: prio 3 <your criteria_low> flowid 1:3

why do you set a priority on filters? In my tests I always have been used packet marking instead of "u32 match" method without prio parameter. It is the same with priority?

Quoted 2 lines

That works in my system.

With your script I get the same result as mine: 2 parallel connections (with different priority), one uses all bandwitdh and the other stalls alternatively.

perhaps there is a timeout mechanims that forse sending queued packets although lower priority?

Quoted 4 lines

With your script I get the same result as mine: 2 parallel connections (with different priority), one uses all bandwitdh and the other stalls alternatively.

perhaps there is a timeout mechanims that forse sending queued packets although lower priority?

This is exactly how PRIO works. Higher priority classes get dequeued first, so if there is something to dequeue forever then the lower priority classes would never dequeue.

Hi!

Quoted 4 lines

Quoted 4 lines

With your script I get the same result as mine: 2 parallel connections (with different priority), one uses all bandwitdh and the other stalls alternatively.

Quoted 9 lines

Quoted 3 lines

perhaps there is a timeout mechanims that forse sending queued packets although lower priority?

This is exactly how PRIO works. Higher priority classes get dequeued first, so if there is something to dequeue forever then the lower priority classes would never dequeue.

but it's not behaviour shown by test. And I don't no why. Ex.
connection A -> higher priority
connection B -> lower priority

results:
time | connection A | connection B
1 sec 32 KB/s 0 KB/s
10 sec 18 KB/s 15 KB/s

20 sec           2 KB/s           31 KB/s
30 sec           0 KB/s           32 KB/s
40 sec         15 KB/s           16 KB/s
50 sec         25 KB/s             7 KB/s
60 sec         33 KB/s             0 KB/s

... .... ....
and so on

do you understand? I don't know why...

suggestions are welcome.

If you mean why connB get traffic - then it's probably connA doesn't *always* fill its bandwide which makes it possible for connB to send.

Try to fullfil connA with several mass downloading connections

Hi,

Quoted 5 lines

Different priorities are actual for filters with the same parent, lower ones are looking first.

ok

Quoted 6 lines

Check output:
tc -s filter show dev eth0
and
tc -s qdisc show dev eth0

# tc -s -d qdisc show dev eth0
qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 7021292 bytes 4684 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
qdisc sfq 2: parent 1:1 limit 128p quantum 1514b flows 128/1024 perturb
10sec
Sent 2997906 bytes 2003 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
qdisc sfq 3: parent 1:2 limit 128p quantum 1514b flows 128/1024 perturb
10sec
Sent 720 bytes 8 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
qdisc sfq 4: parent 1:3 limit 128p quantum 1514b flows 128/1024 perturb
10sec
Sent 4022666 bytes 2673 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0

# tc -s -d class show dev eth0
class prio 1:1 parent 1: leaf 2:
Sent 2997906 bytes 2003 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:2 parent 1: leaf 3:
Sent 6082 bytes 49 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
class prio 1:3 parent 1: leaf 4:
Sent 4022666 bytes 2673 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0

you see, class with higher priority sent less packets than class 1:3 (with lower priority). This is the proof that something is wrong, or I fall in error?

class 1:2 receive packets from a ssh connection, used for test.

Thanks,
Fabio

I need additional info:
iptables -S
iptables -S -t mangle
and
tc -s filter show dev eth0.

There must be something wrong in your packet marking scheme. What is your test environment?

Quoted 6 lines

OK.
The starnge thing is:  match 00500000/ffff0000 at 20. I suppose you try to match on dst_ip - there  must be 'at 16'!

what does it mean?

this is my filter code:
tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip sport 80 0xffff flowid 1:1
tc filter add dev eth0 protocol ip parent 1: prio 3 u32 match ip sport 81 0xffff flowid 1:3

and packets are queued in the right queue. If you prefer I can filter using iptables mangle, I think it's the same.

Fabio

Quoted 9 lines

That's warmer :)
Try the next:
tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip protocol 6 0xFF ip sport 80 0xFFFF flowid 1:1
tc filter add dev eth0 protocol ip parent 1: prio 2 u32 match ip protocol 6 0xFF ip sport 81 0xFFFF flowid 1:3
tc filter add dev eth0 protocol ip parent 1: prio 3 u32 match u8 0 0 flowid 1:2

Hi,

Quoted 11 lines

Quoted 11 lines

That's warmer :)
Try the next:
tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip protocol 6 0xFF ip sport 80 0xFFFF flowid 1:1
tc filter add dev eth0 protocol ip parent 1: prio 2 u32 match ip protocol 6 0xFF ip sport 81 0xFFFF flowid 1:3
tc filter add dev eth0 protocol ip parent 1: prio 3 u32 match u8 0 0 flowid 1:2

I try to use use it but I get this error:
What is "ip"?
Usage: ...

What's wrong?

Very thanks,
Fabio

Sorry, there was my error.
The correct version:
tc filter add dev eth0 protocol ip parent 1: prio 1 u32 classid 1:1 match ip protocol 6 0xFF match ip sport 80 0xFFFF
tc filter add dev eth0 protocol ip parent 1: prio 2 u32 classid 1:3 match ip protocol 6 0xFF ip sport 81 0xFFFF
tc filter add dev eth0 protocol ip parent 1: prio 3 u32 match u8 0 0 flowid 1:2

Hi,
I test this script:
#!/bin/bash

tc qdisc del dev eth0 root
iptables -P INPUT ACCEPT

tc qdisc add dev eth0 root handle 1: prio bands 3
tc qdisc add dev eth0 parent 1:1 handle 2 sfq perturb 10
tc qdisc add dev eth0 parent 1:2 handle 3 sfq perturb 10
tc qdisc add dev eth0 parent 1:3 handle 4 sfq perturb 10
#tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip sport 80 0xffff flowid 1:1
##tc filter add dev eth0 protocol ip parent 1: prio 2 <your criteria_middle> flowid 1:2
#tc filter add dev eth0 protocol ip parent 1: prio 3 u32 match ip sport 81 0xffff flowid 1:3
tc filter add dev eth0 protocol ip parent 1: prio 1 u32 classid 1:1 match ip protocol 6 0xFF match ip sport 80 0xFFFF
tc filter add dev eth0 protocol ip parent 1: prio 2 u32 classid 1:3 match ip protocol 6 0xFF match ip sport 81 0xFFFF
tc filter add dev eth0 protocol ip parent 1: prio 3 u32 match u8 0 0 flowid 1:2

I run 4 high priority processes (classid 1:1) and 1 low priority one (classid 1:3). These are the result:
high proc1: 19.5K/s
high proc2: 5.71K/s
high proc3: 738B/s
high proc4: 12.8K/s

low proc: 5.8 K/s

I don't undestand: low process didn't stall and a high process used a datarate << than low process.

Help, help, help!

Fabio

( client --- INTERNET--- router --- shaper1 --- server )