Extremely inaccurate results using either TBF or HTB
Hi everyone,
I am getting extremely inaccurate results from my setup:
- RedHat 8.0 on a Compaq ProLiant 1600, dual PII/450, Intel Dual 100 NIC
- I've tried both SMP and non-SMP kernels.
- I'm using the updated tc from the HTB home page.
- I'm using the HTB that comes with the RH8 kernel.
Here's what's happening. If I, for instance:
tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit
15Kb
Then my download rate goes at 360,000 bytes/s, which, by my calculations, is 2.77mbit. I am using Win2k perfmon to test the speed. The line is flat and the speed is very consistent (consistently *wrong*, unfortunately...)
Immediately after, I do:
tc qdisc change dev eth2 root tbf rate 1.8mbit buffer 20Kb/8 limit
15Kb
My download speed is then 243,000 bytes/s, or 1.85mbit. Again, very flat, consistent line.
I have played with the parameters on both TBF and HTB in many different configurations. I always seem to have a large jump between 1.8-1.9. I can't find any number between those two that will actually produce a 2mbit limit. I know 1.85 is close to 2.0, but is this the expected margin of error? (Then again, 2.77 isn't close to 1.9 at all, so I hope not!)
Has anyone else experienced anything like this?
Thank you very much for any help you can provide!
Dan Farino
Sr. System Engineer
Stamps.com, Inc.
Santa Monica, CA
Quoted 27 lines
I am getting extremely inaccurate results from my setup:
- RedHat 8.0 on a Compaq ProLiant 1600, dual PII/450, Intel Dual 100 NIC
- I've tried both SMP and non-SMP kernels.
- I'm using the updated tc from the HTB home page.
- I'm using the HTB that comes with the RH8 kernel.Here's what's happening. If I, for instance:
tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit
15KbThen my download rate goes at 360,000 bytes/s, which, by my calculations, is 2.77mbit. I am using Win2k perfmon to test the speed. The line is flat and the speed is very consistent (consistently *wrong*, unfortunately...)
Immediately after, I do:
tc qdisc change dev eth2 root tbf rate 1.8mbit buffer 20Kb/8 limit
15KbMy download speed is then 243,000 bytes/s, or 1.85mbit. Again, very flat, consistent line.
Quoted 8 lines
I have played with the parameters on both TBF and HTB in many different configurations. I always seem to have a large jump between 1.8-1.9. I can't find any number between those two that will actually produce a 2mbit limit. I know 1.85 is close to 2.0, but is this the expected margin of error? (Then again, 2.77 isn't close to 1.9 at all, so I hope not!)
Has anyone else experienced anything like this?
I did some tests with htb and tbf :
http://www.docum.org/stef.coene/qos/tests/tbf/
http://www.docum.org/stef.coene/qos/tests/htb/index.html
And the accuracy was very good.
I did the tests on a debian system with a custom kernel and tc command.
How did you measure the throughput? I use the iptables counters so I know exactly what passes the box. If you download a file from the internet with a ftp client, there can be overhead due to retransmission, control packets, .... So the speed you measure with the ftp client may be lower then the real connection speed.
Stef
Signature · 8 lines
[email protected] "Using Linux as bandwidth manager" http://www.docum.org/ #lartc @ irc.oftc.net _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi Stef,
Thanks for the reply! However, I think the gap is way too large to be explained by overhead, etc.
Here are the results from some other tests. I started with:
tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit 15Kb
Here are the average transfer rates measured over approx 20 seconds (again according to PerfMon) as I change the "1.9mbit" to other values while performing a large download:
Mbit Kb/s
----- ------
2.5 360
2.0 360
1.9 360
1.8 240
1.5 240
1.4 240
1.3 180
1.2 180
The lines really are that flat. There is nothing resembling the graphs on your site. Mine has the low-res, stair-step effect that you can see by graphing the above numbers.
I can't seem to find any "mbit" number that will actually produce a cap at exactly 2mbit.
I will try a both different type NIC and Debian at some point today.
Thanks for your help!
Dan Farino
Sr. System Engineer
Stamps.com, Inc.
Santa Monica, CA
-----Original Message-----
From: Stef Coene [mailto:[email protected]]
Sent: Tuesday, November 12, 2002 8:09 AM
To: Dan Farino; [email protected]
Subject: Re: [LARTC] Extremely inaccurate results using either TBF or HTB
Quoted 4 lines
I am getting extremely inaccurate results from my setup:
- RedHat 8.0 on a Compaq ProLiant 1600, dual PII/450, Intel Dual 100
NIC
Quoted 4 lines
- I've tried both SMP and non-SMP kernels.
- I'm using the updated tc from the HTB home page.
- I'm using the HTB that comes with the RH8 kernel.
Quoted 8 lines
Here's what's happening. If I, for instance:
tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit
15KbThen my download rate goes at 360,000 bytes/s, which, by my calculations, is 2.77mbit. I am using Win2k perfmon to test the speed. The line is flat and the speed is very consistent (consistently
*wrong*,
Quoted 5 lines
unfortunately...)
Immediately after, I do:
tc qdisc change dev eth2 root tbf rate 1.8mbit buffer 20Kb/8
limit
Quoted 7 lines
15Kb
My download speed is then 243,000 bytes/s, or 1.85mbit. Again, very flat, consistent line.
I have played with the parameters on both TBF and HTB in many
different
Quoted 4 lines
configurations. I always seem to have a large jump between 1.8-1.9. I can't find any number between those two that will actually produce a 2mbit limit. I know 1.85 is close to 2.0, but is this the expected margin of error? (Then again, 2.77 isn't close to 1.9 at all, so I
hope
Quoted 3 lines
not!)
Has anyone else experienced anything like this?
I did some tests with htb and tbf :
http://www.docum.org/stef.coene/qos/tests/tbf/
http://www.docum.org/stef.coene/qos/tests/htb/index.html
And the accuracy was very good.
I did the tests on a debian system with a custom kernel and tc command.
How did you measure the throughput? I use the iptables counters so I know
exactly what passes the box. If you download a file from the internet with a
ftp client, there can be overhead due to retransmission, control packets,
.... So the speed you measure with the ftp client may be lower then the real
connection speed.
Stef
Signature · 8 lines
[email protected] "Using Linux as bandwidth manager" http://www.docum.org/ #lartc @ irc.oftc.net _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 23 lines · Dan Farino
Hi Stef,
Thanks for the reply! However, I think the gap is way too large to be explained by overhead, etc.
Here are the results from some other tests. I started with:
tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit 15KbHere are the average transfer rates measured over approx 20 seconds (again according to PerfMon) as I change the "1.9mbit" to other values while performing a large download:
Mbit Kb/s
----- ------
2.5 360
2.0 360
1.9 360
1.8 240
1.5 240
1.4 240
1.3 180
1.2 180
Quoted 8 lines
The lines really are that flat. There is nothing resembling the graphs on your site. Mine has the low-res, stair-step effect that you can see by graphing the above numbers.
I can't seem to find any "mbit" number that will actually produce a cap at exactly 2mbit.
I will try a both different type NIC and Debian at some point today.
What NIC are you using ?
Quoted 1 line
Thanks for your help!
No problem. I can send you my scripts that I use the record the bandwidth usage. Some of them can be found on www.docum.org. Basically, I create a iptables chain for each traffic stream I want to monitor. I record the counters each second and calculates the bandwdith.
I have some scripts that executes a htb script, starts the needed flows, calculates the bandwidth, if the bandwidth is stable : record the bandwidth, restart the proces with a different setting.
After some time, you have it for each rate so you can create a nice graph.
Stef
Signature · 8 lines
[email protected] "Using Linux as bandwidth manager" http://www.docum.org/ #lartc @ irc.oftc.net _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
This worked miracles for me:
change /usr/src/linux/include/net/pkt_sched.h PSCHED_CLOCK_SOURCE to PSCHED_CPU if you have a cpu with timestamp counter (TSC) that will give you Mhz timer granularity.
(don't forget to recompile of course)
Jannes Faber
----- Original Message -----
From: Dan Farino
To: [email protected]
Sent: Tuesday, November 12, 2002 9:57 AM
Subject: [LARTC] Extremely inaccurate results using either TBF or HTB
Hi everyone,
I am getting extremely inaccurate results from my setup:
- RedHat 8.0 on a Compaq ProLiant 1600, dual PII/450, Intel Dual 100 NIC
- I've tried both SMP and non-SMP kernels.
- I'm using the updated tc from the HTB home page.
- I'm using the HTB that comes with the RH8 kernel.
Here's what's happening. If I, for instance:
tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit 15Kb
Then my download rate goes at 360,000 bytes/s, which, by my calculations, is 2.77mbit. I am using Win2k perfmon to test the speed. The line is flat and the speed is very consistent (consistently *wrong*, unfortunately.)
Immediately after, I do:
tc qdisc change dev eth2 root tbf rate 1.8mbit buffer 20Kb/8 limit 15Kb
My download speed is then 243,000 bytes/s, or 1.85mbit. Again, very flat, consistent line.
I have played with the parameters on both TBF and HTB in many different configurations. I always seem to have a large jump between 1.8-1.9. I can't find any number between those two that will actually produce a 2mbit limit. I know 1.85 is close to 2.0, but is this the expected margin of error? (Then again, 2.77 isn't close to 1.9 at all, so I hope not!)
Has anyone else experienced anything like this?
Thank you very much for any help you can provide!
Dan Farino
Sr. System Engineer
Stamps.com, Inc.
Santa Monica, CA
Jannes,
That worked great! Everything is dead-on accurate now.
Thank you very much for your help!
Dan Farino
Sr. System Engineer
Stamps.com, Inc.
Santa Monica, CA
-----Original Message-----
From: sufcrusher [mailto:[email protected]]
Sent: Tuesday, November 12, 2002 3:11 PM
To: Dan Farino; [email protected]
Subject: Re: [LARTC] Extremely inaccurate results using either TBF or HTB
This worked miracles for me:
change /usr/src/linux/include/net/pkt_sched.h PSCHED_CLOCK_SOURCE to PSCHED_CPU if you have a cpu with timestamp counter (TSC) that will give you Mhz timer granularity.
(don't forget to recompile of course)
Jannes Faber
----- Original Message ----- From: Dan Farino <mailto:[email protected]> To: [email protected] Sent: Tuesday, November 12, 2002 9:57 AM Subject: [LARTC] Extremely inaccurate results using either TBF
or HTB
Hi everyone,
I am getting extremely inaccurate results from my setup:
- RedHat 8.0 on a Compaq ProLiant 1600, dual PII/450, Intel Dual
100 NIC
- I've tried both SMP and non-SMP kernels.
- I'm using the updated tc from the HTB home page.
- I'm using the HTB that comes with the RH8 kernel.
Here's what's happening. If I, for instance:
tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8
limit 15Kb
Then my download rate goes at 360,000 bytes/s, which, by my
calculations, is 2.77mbit. I am using Win2k perfmon to test the speed. The line is flat and the speed is very consistent (consistently *wrong*, unfortunately...)
Immediately after, I do:
tc qdisc change dev eth2 root tbf rate 1.8mbit buffer
20Kb/8 limit 15Kb
My download speed is then 243,000 bytes/s, or 1.85mbit. Again,
very flat, consistent line.
I have played with the parameters on both TBF and HTB in many
different configurations. I always seem to have a large jump between 1.8-1.9. I can't find any number between those two that will actually produce a 2mbit limit. I know 1.85 is close to 2.0, but is this the expected margin of error? (Then again, 2.77 isn't close to 1.9 at all, so I hope not!)
Has anyone else experienced anything like this?
Thank you very much for any help you can provide!
Dan Farino
Sr. System Engineer
Stamps.com, Inc.
Santa Monica, CA
Stef,
I solved my problem by changing PSCHED_CLOCK_SOURCE to PSCHED_CPU in /usr/src/linux/include/net/pkt_sched.h
My graph is now a nice smooth diagonal line as I increase the bandwidth in increments as I did before.
Thanks for you help on this one. You site has been a tremendous help in getting me up and running.
Take care,
Dan Farino
Sr. System Engineer
Stamps.com, Inc.
Santa Monica, CA
-----Original Message-----
From: Stef Coene [mailto:[email protected]]
Sent: Tuesday, November 12, 2002 10:07 AM
To: Dan Farino; [email protected]
Subject: Re: [LARTC] Extremely inaccurate results using either TBF or HTB
Quoted 23 lines · Dan Farino
Hi Stef,
Thanks for the reply! However, I think the gap is way too large to be explained by overhead, etc.
Here are the results from some other tests. I started with:
tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit 15KbHere are the average transfer rates measured over approx 20 seconds (again according to PerfMon) as I change the "1.9mbit" to other values while performing a large download:
Mbit Kb/s
----- ------
2.5 360
2.0 360
1.9 360
1.8 240
1.5 240
1.4 240
1.3 180
1.2 180
Quoted 5 lines
The lines really are that flat. There is nothing resembling the graphs on your site. Mine has the low-res, stair-step effect that you can see by graphing the above numbers.
I can't seem to find any "mbit" number that will actually produce a
cap
Quoted 3 lines
at exactly 2mbit.
I will try a both different type NIC and Debian at some point today.
What NIC are you using ?
Quoted 1 line
Thanks for your help!
No problem. I can send you my scripts that I use the record the bandwidth
usage. Some of them can be found on www.docum.org. Basically, I create a
iptables chain for each traffic stream I want to monitor. I record the counters each second and calculates the bandwdith.
I have some scripts that executes a htb script, starts the needed flows,
calculates the bandwidth, if the bandwidth is stable : record the bandwidth,
restart the proces with a different setting.
After some time, you have it for each rate so you can create a nice graph.
Stef
Signature · 8 lines
[email protected] "Using Linux as bandwidth manager" http://www.docum.org/ #lartc @ irc.oftc.net _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 5 lines · sufcrusher
This worked miracles for me:
change
/usr/src/linux/include/net/pkt_sched.hPSCHED_CLOCK_SOURCEtoPSCHED_CPUif you have a cpu with timestamp counter (TSC) that will give you Mhz timer granularity.
Is this only needed if you have multiple CPU's ? Or is your setup a 1 CPU setup?
Stef
Signature · 8 lines
[email protected] "Using Linux as bandwidth manager" http://www.docum.org/ #lartc @ irc.oftc.net _______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/