tc shown rate larger than ceil (was "Weird rate in HTB")
An earlier exchange about someone seeing the rate larger than the ceiling is posted below. Andy explained the reason for the "above ceiling" rate in Daniel's output . . . but I just saw an example that doesn't fit.
Quoted 1 line
Quoted 1 line
tc output >>
class htb 1:14 parent 1:1 leaf 14: prio 1 quantum 3072 rate 256000bit ceil 282000bit burst 1820b/8 mpu 0b overhead 0b cburst 1851b/8 mpu 0b overhead 0b level 0
Sent 2639448 bytes 1128 pkt (dropped 0, overlimits 0 requeues 0) rate 325360bit 17pps backlog 0b 25p requeues 0
lended: 981 borrowed: 122 giants: 1155
tokens: -48548 ctokens: -56127
<< tc outpu <<
I see a 43360 difference, where the rate is more than the ceiling . . . of which only 952bits are accounted for in the following exchange. I have actually seen the rate be as much as 80408bits off . . . at only 19pps.
Is there something else I am missing?
Stone
Daniel Harold L. wrote On Tuesday 03 July 2007 22:50:
Quoted 5 lines
Quoted 5 lines
Dear all,
First, sorry for my bad English ..
To night one of my client is the victim of UDP attack from
internet. It's tons
Quoted 1 line
Quoted 1 line
of UDP packets from internet with destination to port 80. But
when I look at
Quoted 1 line
Quoted 1 line
class of that victim client, the actual class rate is over than
configured
Quoted 3 lines
Quoted 3 lines
rate class.
Below is my screen capture. You can see at class 1:913 which have
actual rate
Quoted 1 line
Quoted 1 line
105136bit while configured with ceil at 96000bit. Also it's
parent class
Quoted 1 line
Quoted 1 line
(1:91) which have actual rate 107680bit while configured with
ceil at
Quoted 3 lines
Quoted 3 lines
96000bit.
Is this normal? Or I have miss something in my script. Sometimes
ago I found
Quoted 1 line
Quoted 1 line
this situation but I forgot to capture the screen and the traffic
is UDP too
Quoted 4 lines
Quoted 1 line
(maybe from torrent-like client)
Yes it is normal!
Quoted 8 lines
The rate tables that tc use normally have an 8 byte steps, so it is possible for up to a 56bit/s error per packet and you have 300 pps.
There was a small patch submitted for tc to make the error fall on the underrate rather than overrate side, but I think it got lost in the middle of the long ATM overhead patch thread on netdev.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 5 lines · Stonie Cooper
An earlier exchange about someone seeing the rate larger than the ceiling is posted below. Andy explained the reason for the "above ceiling" rate in Daniel's output . . . but I just saw an example that doesn't fit.
Quoted 7 lines
Quoted 1 line
Quoted 1 line
tc output >>
class htb 1:14 parent 1:1 leaf 14: prio 1 quantum 3072 rate 256000bit ceil 282000bit burst 1820b/8 mpu 0b overhead 0b cburst 1851b/8 mpu 0b overhead 0b level 0
Sent 2639448 bytes 1128 pkt (dropped 0, overlimits 0 requeues 0) rate 325360bit 17pps backlog 0b 25p requeues 0
lended: 981 borrowed: 122 giants: 1155
It's because you have giants - possibly because your nic does tcp segmentation offload so locally generated traffic goes through htb as big chunks before it gets segmented down to the nic's mtu.
You can check/turn that off with ethtool -k or you could use htb's mtu parameter for each rate (I'm not sure if you need to do ceils aswell) which makes the granularity of the rate table bigger so it can handle the larger mtu.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Andy - Thanks!
I took the ethtool path, and turned off tcp segmentation on that nic; I was unsure how to set the HTB MTU - I use shorewall on a Gentoo system.
It has definitely made a difference. The highest rate I see is 282312bits on a line with a ceil of 282000bits - and that is with 24pps, which, according to your previous email . . . would be right in line with what it should be.
Giants have fallen off, but are not completely eliminated. Is this something I should be concerned with (having giants)? Would setting the HTB MTU be more elegant? I have avoided creating my own tc script, and have been using shorewall's internals . . . but if utilizing the HTB MTU setting is "better", I will dive in and try to write a script that does the same thing as shorewall.
Stone
Quoted 17 lines · Andy Furniss
Quoted 11 lines · Stonie Cooper
An earlier exchange about someone seeing the rate larger than the ceiling is posted below. Andy explained the reason for the "above ceiling" rate in Daniel's output . . . but I just saw an example that doesn't fit.
Quoted 1 line
Quoted 1 line
tc output >>
class htb 1:14 parent 1:1 leaf 14: prio 1 quantum 3072 rate 256000bit ceil 282000bit burst 1820b/8 mpu 0b overhead 0b cburst 1851b/8 mpu 0b overhead 0b level 0
Sent 2639448 bytes 1128 pkt (dropped 0, overlimits 0 requeues 0) rate 325360bit 17pps backlog 0b 25p requeues 0
lended: 981 borrowed: 122 giants: 1155It's because you have giants - possibly because your nic does tcp segmentation offload so locally generated traffic goes through htb as big chunks before it gets segmented down to the nic's mtu.
Quoted 6 lines
You can check/turn that off with ethtool -k or you could use htb's mtu parameter for each rate (I'm not sure if you need to do ceils aswell) which makes the granularity of the rate table bigger so it can handle the larger mtu.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 4 lines · Stonie Cooper
Andy - Thanks!
I took the ethtool path, and turned off tcp segmentation on that nic; I was unsure how to set the HTB MTU - I use shorewall on a Gentoo system.
I think you would need to find each line with a rate in the script and add mtu X - I don't know what X would be, though.
Quoted 5 lines
It has definitely made a difference. The highest rate I see is 282312bits on a line with a ceil of 282000bits - and that is with 24pps, which, according to your previous email . . . would be right in line with what it should be.
If you are shaping for a wan then I would turn off tso rather than use mtu. It should be better for latency as a big chunk of data is going to take more time to be transmitted and hurt interactive traffic. I can't imagine it being very good to effectively drop multiple segments at once either - but then shorewall may not setup with queues short enough to get drops.
Quoted 3 lines
Giants have fallen off, but are not completely eliminated. Is this something I should be concerned with (having giants)?
As long as there aren't many I wouldn't care - I don't know why you still see them, though. I haven't got any nics that do tso so can't test.
Would setting the
Quoted 4 lines
HTB MTU be more elegant? I have avoided creating my own tc script, and have been using shorewall's internals . . . but if utilizing the HTB MTU setting is "better", I will dive in and try to write a script that does the same thing as shorewall.
You would also loose some accuracy in the rate lookup tables if you used mtu 16,32,64 etc. bytes depending on how big an mtu you needed to use.
If you have a slow dsl link and really care about latency or not having to back off from it's rate, then there are other ways to tweak things. They involve patching and recompiling kernel/tc.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Andy - I spoke too soon; there were evidently just a few giants left in the queue when I did the change. After an hour, all giants ceased. Thanks again - it works much better now.
Stone
Quoted 38 lines · Andy Furniss
Quoted 4 lines · Stonie Cooper
Andy - Thanks!
I took the ethtool path, and turned off tcp segmentation on that nic; I was unsure how to set the HTB MTU - I use shorewall on a Gentoo system.I think you would need to find each line with a rate in the script and add mtu X - I don't know what X would be, though.
Quoted 4 lines
It has definitely made a difference. The highest rate I see is 282312bits on a line with a ceil of 282000bits - and that is with 24pps, which, according to your previous email . . . would be right in line with what it should be.
If you are shaping for a wan then I would turn off tso rather than use mtu. It should be better for latency as a big chunk of data is going to take more time to be transmitted and hurt interactive traffic. I can't imagine it being very good to effectively drop multiple segments at once either - but then shorewall may not setup with queues short enough to get drops.
Quoted 2 lines
Giants have fallen off, but are not completely eliminated. Is this something I should be concerned with (having giants)?
As long as there aren't many I wouldn't care - I don't know why you still see them, though. I haven't got any nics that do tso so can't test.
Would setting the
Quoted 4 lines
HTB MTU be more elegant? I have avoided creating my own tc script, and have been using shorewall's internals . . . but if utilizing the HTB MTU setting is "better", I will dive in and try to write a script that does the same thing as shorewall.
You would also loose some accuracy in the rate lookup tables if you used mtu 16,32,64 etc. bytes depending on how big an mtu you needed to use.
Quoted 5 lines
If you have a slow dsl link and really care about latency or not having to back off from it's rate, then there are other ways to tweak things. They involve patching and recompiling kernel/tc.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc