urgent question about tcng!
Hello List,
I'm new to QoS/tcng/HTB and friends, so please forgive me if my question might be silly...
After having read lots of HowTo documents I'm totally confused...
The Challenge:
=======
I'll have to deploy several "mirror" download servers (Linux) which must be able to handle a huge number of HTTP download requests (about 10k to 20k unicast requests per server within 120 min.) which will hit my servers . The mirror servers will be connected with 100MBit to the Internet backbone. Each "client" will download between 7MB up to 30MB per request. The download mirror servers will be directly attached to the Internet (no FW or router between, which could handle QoS for me). Of course will these download mirrors be specially hardened :-)
The Idea:
====I can not handle all the traffic at once without QoS, so I thought it should be possible to establish a traffic control mechanism to guarantee bandwith for each client download (e.g. 256kbps CIR up to 512kbps ceil). This should allow me, to "serve" all clients within 120 min, before a new download wave will hit my servers. If the available bandwith for HTTP traffic is used, clients shall not be able to start a download, until some bandwith is freed by another client who finished a download.
Note: I would rather go with "tcng" than with the old "tc".
The Problem:
======
While playing with tcng and HTB I managed to shape traffic (to limit the total bandwith for HTTP traffic to e.g. 50Mbps) but I did not manage to write a tcng configuration which guarantees a certain bandwith per client request. I must be able to assign (and guarantee) a defined bandwith to a client request, to make the downloads more predictable. My experiments so far only allowed me to limit e.g. HTTP traffic, which then was equally distributed to the number of clients, but this led to the usual "traffic jam" when too many clients initiated a simultanes download.
Can anybody please assist me, or does anybody have a working "tcng" config file to do the job. Additional readings/comments/links (beside those listed in the LDB/Linux Traffic Control Howtos are also welcome.)
Many thanks in advance and
best regards from Austria
Thomas
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 2 lines · Thomas Mandl
Hello List,
I'm new to QoS/tcng/HTB and friends, so please forgive me if my question
might be silly...
Quoted 6 lines
After having read lots of HowTo documents I'm totally confused...
The Challenge:
=======
I'll have to deploy several "mirror" download servers (Linux) which must be able to handle a huge number of HTTP download requests (about 10k to 20k
unicast requests per server within 120 min.) which will hit my servers . The
Quoted 1 line
mirror servers will be connected with 100MBit to the Internet backbone.
Each
Quoted 1 line
"client" will download between 7MB up to 30MB per request. The download
mirror servers will be directly attached to the Internet (no FW or router between, which could handle QoS for me). Of course will these download mirrors be specially hardened :-)
Quoted 3 lines
The Idea:
====> I can not handle all the traffic at once without QoS, so I thought it
should
Quoted 1 line
be possible to establish a traffic control mechanism to guarantee
bandwith for each client download (e.g. 256kbps CIR up to 512kbps ceil). This should
Quoted 2 lines
allow me, to "serve" all clients within 120 min, before a new download wave will hit my servers. If the available bandwith for HTTP traffic is used,
clients shall not be able to start a download, until some bandwith is freed
Quoted 7 lines
by another client who finished a download.
Note: I would rather go with "tcng" than with the old "tc".
The Problem:
======
While playing with tcng and HTB I managed to shape traffic (to limit the
total bandwith for HTTP traffic to e.g. 50Mbps) but I did not manage to write a tcng configuration which guarantees a certain bandwith per client request. I must be able to assign (and guarantee) a defined bandwith to a client request, to make the downloads more predictable. My experiments so far only allowed me to limit e.g. HTTP traffic, which then was equally distributed to the number of clients, but this led to the usual "traffic jam" when too many clients initiated a simultanes download.
Quoted 2 lines
Can anybody please assist me, or does anybody have a working "tcng"
config file to do the job. Additional readings/comments/links (beside those listed
Quoted 7 lines
in the LDB/Linux Traffic Control Howtos are also welcome.)
Many thanks in advance and
best regards from AustriaThomas
I think you might want to check Apache's mod_bandwidth / mod_throttle instead of QoS.
Regards,
Sylvain
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
On Wednesday 27 April 2005 12:17, Sylvain BERTRAND wrote:
<snip>
Quoted 2 lines
I think you might want to check Apache's
mod_bandwidth/mod_throttleinstead of QoS.
There's also bw_mod if you're running Apache2, which for me is working fine so far.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 38 lines · Thomas Mandl
Hello List,
I'm new to QoS/tcng/HTB and friends, so please forgive me if my question might be silly...
After having read lots of HowTo documents I'm totally confused...The Challenge:
=======
I'll have to deploy several "mirror" download servers (Linux) which must be able to handle a huge number of HTTP download requests (about 10k to 20k unicast requests per server within 120 min.) which will hit my servers . The mirror servers will be connected with 100MBit to the Internet backbone. Each "client" will download between 7MB up to 30MB per request. The download mirror servers will be directly attached to the Internet (no FW or router between, which could handle QoS for me). Of course will these download mirrors be specially hardened :-)The Idea:
====> I can not handle all the traffic at once without QoS, so I thought it should be possible to establish a traffic control mechanism to guarantee bandwith for each client download (e.g. 256kbps CIR up to 512kbps ceil). This should allow me, to "serve" all clients within 120 min, before a new download wave will hit my servers. If the available bandwith for HTTP traffic is used, clients shall not be able to start a download, until some bandwith is freed by another client who finished a download.Note: I would rather go with "tcng" than with the old "tc".
The Problem:
======
While playing with tcng and HTB I managed to shape traffic (to limit the total bandwith for HTTP traffic to e.g. 50Mbps) but I did not manage to write a tcng configuration which guarantees a certain bandwith per client request. I must be able to assign (and guarantee) a defined bandwith to a client request, to make the downloads more predictable. My experiments so far only allowed me to limit e.g. HTTP traffic, which then was equally distributed to the number of clients, but this led to the usual "traffic jam" when too many clients initiated a simultanes download.
Quoted 3 lines
Can anybody please assist me, or does anybody have a working "tcng" config file to do the job. Additional readings/comments/links (beside those listed in the LDB/Linux Traffic Control Howtos are also welcome.)
Wang Jian posted a perflow queue recently -
http://mailman.ds9a.nl/pipermail/lartc/2005q2/015381.html
It is the sort of thing you need if you can't do it with apache as recommended.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Hello Everybody,
I forgot to mention, that my HTTP traffic will not be handled by an apache web server, and the HTTP protocol can be replaced in the near future by a different protocol. Thus apache's mod_bandwith isn't an option for me right now.
I'm not sure how Wang Jian's post can help me. I thought the existing Linux 2.6.11 kernel and tcng already can do what I need? All I have to do is write a proper tcng config file...
But the proper tcng file is the challenge...
regards
Thomas
-----Ursprüngliche Nachricht-----
Von: Andy Furniss [mailto:[email protected]]
Gesendet: Mittwoch, 27. April 2005 23:23
An: [email protected]
Cc: LARTC
Betreff: Re: [LARTC] urgent question about tcng!
Quoted 15 lines · Thomas Mandl
Hello List,
I'm new to QoS/tcng/HTB and friends, so please forgive me if my question might be silly...
After having read lots of HowTo documents I'm totally confused...The Challenge:
=======
I'll have to deploy several "mirror" download servers (Linux) which must be able to handle a huge number of HTTP download requests (about 10k to 20k unicast requests per server within 120 min.) which will hit my servers . The mirror servers will be connected with 100MBit to the Internet backbone. Each "client" will download between 7MB up to 30MB per request. The download mirror servers will be directly attached to the Internet (no FW or router between, which could handle QoS for me). Of course will these download mirrors be specially hardened :-)
[...snip...]
Quoted 5 lines
Can anybody please assist me, or does anybody have a working "tcng" config file to do the job. Additional readings/comments/links (beside those listed in the LDB/Linux Traffic Control Howtos are also welcome.)
Wang Jian posted a perflow queue recently -
http://mailman.ds9a.nl/pipermail/lartc/2005q2/015381.html
It is the sort of thing you need if you can't do it with apache as recommended.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 7 lines · Thomas Mandl
Hello Everybody,
I forgot to mention, that my HTTP traffic will not be handled by an apache web server, and the HTTP protocol can be replaced in the near future by a different protocol. Thus apache's
mod_bandwithisn't an option for me right now.
Quoted 5 lines
I'm not sure how Wang Jian's post can help me. I thought the existing Linux 2.6.11 kernel and tcng already can do what I need? All I have to do is write a proper tcng config file...
But the proper tcng file is the challenge...
I don't know tcng, but the reason I suggested perflow is that you want each flow to have a ceil - unless you make a class and rule to match each flow I can't see how you can do this. Also iptables could limit the number of connections - tc can't, perflow can.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 5 lines · Andy Furniss
I don't know tcng, but the reason I suggested perflow is that you want each flow to have a ceil - unless you make a class and rule to match each flow I can't see how you can do this. Also iptables could limit the number of connections - tc can't, perflow can.
Forgot to say you can also use iptables to limit rate per connection.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Sorry for the stupid question, but how would I rate limit connections using iptables?
regards
Thomas
-----Ursprüngliche Nachricht-----
Von: Andy Furniss [mailto:[email protected]]
Gesendet: Dienstag, 03. Mai 2005 16:43
An: Andy Furniss
Cc: [email protected]; 'LARTC'
Betreff: Re: AW: [LARTC] urgent question about tcng!
Quoted 5 lines · Andy Furniss
I don't know tcng, but the reason I suggested perflow is that you want each flow to have a ceil - unless you make a class and rule to match each flow I can't see how you can do this. Also iptables could limit the number of connections - tc can't, perflow can.
Forgot to say you can also use iptables to limit rate per connection.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Quoted 3 lines · Thomas Mandl
Sorry for the stupid question, but how would I rate limit connections using iptables?
I was thinking of the patch-o-matic-ng patches connlimit and connrate, though I've never used either and they may or may not be expensive for many connections compared to perflow.
Connrate lets you mark packets if they are above limits - you could then drop those later in a filter table or with a TC filter/queue.
Quoted 18 lines
regards
Thomas
-----Ursprüngliche Nachricht-----
Von: Andy Furniss [mailto:[email protected]]
Gesendet: Dienstag, 03. Mai 2005 16:43
An: Andy Furniss
Cc: [email protected]; 'LARTC'
Betreff: Re: AW: [LARTC] urgent question about tcng!Quoted 4 lines · Andy Furniss
I don't know tcng, but the reason I suggested perflow is that you want each flow to have a ceil - unless you make a class and rule to match each flow I can't see how you can do this. Also iptables could limit the number of connections - tc can't, perflow can.
Quoted 5 lines
Forgot to say you can also use iptables to limit rate per connection.
Andy.
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc