Skip to contentSkip to conversation

Comments on my set-up please

LARTC ambitious_llama 1 participant 1 message Apr 29, 2003

lore
Share & tools
lore
#0 lore raw

Hi,

A few days ago I asked you all for input on whether my P200 could handle the tasks of NAT'ing, controlling, and firewalling. The answer from a number of you was "yes".

Below is current set-up that I pieced together after skimming the LARTC how-to.

I'll start by describing my situation and objectives, then I'll get into the script and state how the lines contribute to the objective.

SITUATION:
---------

- Firewall/Router/Gateway: P200, 48MB RAM, RedHat 9 -- shares internet with all other machines through a 1.5Mbit/192Kbit cable connection (mine)
- 4 WinXP Pro desktops (2 at neighbours and 2 are mine)
- 1 RedHat 9 Samba server. Runs a UT2003 server that is accessible by password from internet and an Apache server for UT2003 map downloads (mine)

OBJECTIVES:
----------

First and foremost I want to ensure that the UT2003 server/players and Microsoft Gamvoice have guaranteed bandwidth regardless of what is happening elsewhere.

Second, I want keep the neighbour's use of Kaazaa to flood my upstream and downstream bandwidth.

Third, I would like to shape my traffic to give interactive traffic priority.

WHAT I'VE DONE SO FAR:
----------------------

It's not elegant, but so far it's seems to be making a difference. Your advice is welcome.

#################################################################
#!/bin/bash

# delete old settings
#tc qdisc del dev eth0 root
#tc qdisc del dev eth1 root

/root/shaping/stop

############## HTB DOWNSTREAM #################

# create qdisc - eth0 is my internal nic
tc qdisc add dev eth0 root handle 1: htb default 50

# -- 100MBit NIC
tc class add dev eth0 parent 1: classid 1:1 htb rate 75000kbit ceil 100000kbit

# neighbour's house - Class for neighbour's traffic is limited to 1/2 max downstream so they always get consistent performance tc class add dev eth0 parent 1:1 classid 1:10 htb rate 730kbit ceil 750kbit prio 1

# our house - Class for our traffic; limit is higher to allow for
variability
tc class add dev eth0 parent 1:1 classid 1:20 htb rate 730kbit ceil
1500kbit prio 1
# for lan only traffic between admin and firewall computer (ssh) --
class to ensure that my ssh sessions are not crippled
tc class add dev eth0 parent 1:1 classid 1:30 htb rate 5000kbit ceil
15000kbit prio 2
# guaranteed pipe for voice and ut2k3 -- class for GameVoice and UT2K3
tc class add dev eth0 parent 1:1 classid 1:40 htb rate 40kbit ceil
80kbit prio 0
# everything else -- Anything that doesn't fit into either class. Don't
know what it would be, but it doesn't get much bandwidth.
tc class add dev eth0 parent 1:1 classid 1:50 htb rate 5kbit ceil 10kbit
prio 3
# sfq -- Implement SFQ for all classes
tc qdisc add dev eth0 parent 1:10 handle 110: sfq perturb 10
tc qdisc add dev eth0 parent 1:20 handle 120: sfq perturb 10
tc qdisc add dev eth0 parent 1:40 handle 140: sfq perturb 10
tc qdisc add dev eth0 parent 1:50 handle 150: sfq perturb 10

# Mark Packets

# Neighbour's Packets by IP
tc filter add dev eth0 parent 1: protocol ip prio 100 u32 match ip \
dst 192.168.0.7 flowid 1:10
tc filter add dev eth0 parent 1: protocol ip prio 100 u32 match ip \
dst 192.168.0.8 flowid 1:10

# UT2K3 and GameVoice by ports
tc filter add dev eth0 parent 1: protocol ip prio 100 u32 match ip \
dport 9110 0xffff flowid 1:40
tc filter add dev eth0 parent 1: protocol ip prio 100 u32 match ip \
dport 7777 0xffff flowid 1:40

# ssh by port
tc filter add dev eth0 parent 1: protocol ip prio 100 u32 match ip \
sport 22 0xffff flowid 1:30

# My computers (not all exist) by IP
tc filter add dev eth0 parent 1: protocol ip prio 100 u32 match ip \
dst 192.168.0.2 flowid 1:20
tc filter add dev eth0 parent 1: protocol ip prio 100 u32 match ip \
dst 192.168.0.3 flowid 1:20
tc filter add dev eth0 parent 1: protocol ip prio 100 u32 match ip \
dst 192.168.0.4 flowid 1:20
tc filter add dev eth0 parent 1: protocol ip prio 100 u32 match ip \
dst 192.168.0.5 flowid 1:20
tc filter add dev eth0 parent 1: protocol ip prio 100 u32 match ip \
dst 192.168.0.6 flowid 1:20

############ HTB UPSTREAM #########################

# create qdisc - eth1 is external nic on 192kbit upstream connnection
tc qdisc add dev eth1 root handle 2: htb default 50
# limits upstream
tc class add dev eth1 parent 2: classid 2:2 htb rate 180kbit ceil
180kbit
# guaranteed pipe for voice and ut2k3
tc class add dev eth1 parent 2:2 classid 2:40 htb rate 40kbit ceil
80kbit prio 1
# everything else
tc class add dev eth1 parent 2:2 classid 2:50 htb rate 140 ceil 180kbit
prio 0
# sfq - on all classes
tc qdisc add dev eth1 parent 2:40 handle 240: sfq perturb 10
tc qdisc add dev eth1 parent 2:50 handle 250: sfq perturb 10

# Mark Packets

# ut2k3 and voice
tc filter add dev eth1 parent 2: protocol ip prio 100 u32 match ip \
dport 59110 0xffff flowid 2:40
tc filter add dev eth1 parent 2: protocol ip prio 100 u32 match ip \
sport 7777 0xffff flowid 2:40 
# not sure how to separate my upstream from neighbour's upstream

######################################################################

Any ideas, suggestions would be appreciated, but please be explicit. I'm very new at this and don't understand much of the theory yet.

Thanks!

N

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of [email protected]
Sent: April 29, 2003 12:38 AM
To: [email protected]
Subject: LARTC digest, Vol 1 #1146 - 10 msgs

Send LARTC mailing list submissions to
[email protected]

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.ds9a.nl/mailman/listinfo/lartc
or, via email, send a message with subject or body 'help' to
[email protected]

You can reach the person managing the list at
[email protected]

When replying, please edit your Subject line so it is more specific than "Re: Contents of LARTC digest..."

Today's Topics:

1. tcng version 9e (Werner Almesberger)
2. Re: problem: Error: Qdisc "htb" is classless. (Stef Coene)
3. Re: question about tc and ip aliasing (Stef Coene)
4. Re: Eureka! (Stef Coene)
5. Re: Eureka! (=?iso-8859-1?Q?John_Bäckstrand?=)
6. proxy arp problem (Don Cohen)
7. Re: Eureka! (Ricardo Jorge da Fonseca Marques Ferreira)
8. Re: Eureka! (Ricardo Jorge da Fonseca Marques Ferreira)
9. PCI IRQ conflict (Muhammad Al-Gharribeh)
10. PCI IRQ conflict (Muhammad Al-Gharribeh)

--__--__--

Message: 1
Date: Mon, 28 Apr 2003 15:42:44 -0300
From: Werner Almesberger <[email protected]>
To: [email protected]
Subject: [LARTC] tcng version 9e

... is on SourceForge:

http://tcng.sourceforge.net/dist/tcng-9e.tar.gz
md5sum 22db4d60a78ff8338a8cd73995c78a2c

(See also http://tcng.sourceforge.net/)

Another maintenance release:

- fixes a regression in version 8w (tcsim_filter output change broke tcsim_plot -t delay; thanks to Dimitry Ketov for spotting this)
- a few scripts in examples/ used UDP_HDR instead of IP_HDR (this error was probably introduced in version 8y)
- minor compatibility fixes for RedHat 8.0

The complete list of changes is below.

- Werner

----------------------------------- CHANGES
-----------------------------------

Version 9e (28-APR-2003)
------------------------

- updated tests/tcng-6u to accept captialization of cpp 3.2
- put back packet ID in tcsim_filter output, which was removed in tcng-8x
because I had forgotten that it was used for delay calculation in
tcsim_plot
(fixed by Dimitry Ketov; updated tests/basic, tests/tcng-8x,
tests/tcsattpro,
tests/tcsattpsv, and tests/tcsattset)
- doc/tcsim.inc: updated tcsim_filter output format description accordingly
- examples/dsmark+policing, examples/ef-prio, and examples/gred used UDP_HDR
instead of IP_HDR
- "make rpm" and "make tcsim-rpms" now use "rpmbuild" instead of "rpm" if
available

--

________________________________________________________________________
_
/ Werner Almesberger, Buenos Aires, Argentina
[email protected] / /_http://www.almesberger.net/___________________________________________ _/

--__--__--

Message: 2
From: Stef Coene <[email protected]>
Organization: None
To: Ana Carolina Alonso <[email protected]>,
[email protected]
Subject: Re: [LARTC] problem: Error: Qdisc "htb" is classless.
Date: Mon, 28 Apr 2003 21:19:31 +0200

Quoted 31 lines · Ana Carolina Alonso

Hello, I have a problem, I want to configure the htb qdisc, my configuration scripts loke like:

tc qdisc add dev eth0 root handle 1: htb

tc class add dev eth0 parent 1: classid 1:1 htb rate 6mbit burst 15k tc class add dev eth0 parent 1:1 classid 1:10 htb rate 5mbit burst 15k tc class add dev eth0 parent 1:1 classid 1:20 htb rate 3mbit burst 15k tc class add dev eth0 parent 1:1 classid 1:30 htb rate 1kbit burst 15k

tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev eth0 parent 1:30 handle 30: sfq perturb 10

tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip sport 80 0xffff flowid 1:10 tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip sport 25 0xffff flowid 1:20

When I execute my script I get:

RTNETLINK answers: Invalid argument
Error: Qdisc "htb" is classless.
Error: Qdisc "htb" is classless.
Error: Qdisc "htb" is classless.
Error: Qdisc "htb" is classless.
RTNETLINK answers: No such file or directory
RTNETLINK answers: No such file or directory
RTNETLINK answers: No such file or directory
RTNETLINK answers: Invalid argument
RTNETLINK answers: Invalid argument

Quoted 1 line

What is wrong?.

Do you have htb support in the kernel and a tc binary with htb support ?

Stef

--

[email protected]
"Using Linux as bandwidth manager"
http://www.docum.org/
#lartc @ irc.oftc.net

--__--__--

Message: 3
From: Stef Coene <[email protected]>
Organization: None
To: Szymon Miotk <[email protected]>, [email protected]
Subject: Re: [LARTC] question about tc and ip aliasing
Date: Mon, 28 Apr 2003 21:26:43 +0200

Quoted 12 lines · Szymon Miotk
Quoted 11 lines · Enric Ramos Mas

I have implemented a traffic control / advanced router server using iproute2 and tc (using htb). For all my outgoing tc policies, all it's ok (all the traffic goes out using eth0, and therefore I'm able to catch it using the corresponfing tc filter).

However, the incoming traffic has to be treated in eth1, which has several virtual ifaces (eth1:0, eth1:1, eth1:2 and so on). Even I have introduced all the tc rules correctly, the kernel is not matching any filter rule and there is no way to match any destination into any queue discipline.

Quoted 8 lines
Quoted 1 line

Anyone knows some way to implement that ?

I'm also very interested about this question as I have similiar link configuration. One question about classes on eth0: do leaf rates (1:10
- 1:??) sum up to 3256kbit (parent rate)?
I have to split 1040 kbit for 700 users and HTB manual advices
a) that children rates should sum up to parent rate
b) that the rate should not be less than 4kbit

This is the first time I hear about this rule. You can have a problem with
quantum if you have a low rate. But you can overrule quantum when you add a
htb class (rule : quantum > MTU).

Quoted 3 lines

I expect that I should go by the b) rule, as used rates will never sum up over 1040kbit (there will never be 700 users using this link at the same time). Am I right?

Yes. The problem is that the parent rate is not respected. So if each class
is sending, it will exceed the parent rate. This can be "bad" because you
send more data then your router/modem can handle so it's possible that you
your shaping will not be that perfect.

Stef

--

[email protected]
"Using Linux as bandwidth manager"
http://www.docum.org/
#lartc @ irc.oftc.net

--__--__--

Message: 4
From: Stef Coene <[email protected]>
Organization: None
To: Ricardo Jorge da Fonseca Marques Ferreira <[email protected]>,

        <[email protected]>

Subject: Re: [LARTC] Eureka!
Date: Mon, 28 Apr 2003 21:28:31 +0200

Quoted 10 lines · Ricardo Jorge da Fonseca Marques Ferreira

I finally figured out what in emule was ruining my network.

I tried everything and nothing worked. I was always getting timeouts. Until i went into the extended settings in emule and changed connections per 5 seconds to 5 from the default of 20. Miracle. It now works great.

So apparently lots of connection attempts in a few seconds is an effective way of starving all other machines behind a linux gateway even if you're using SFQ.

Mhh. I wonder how this can be. Some tcp connection specialists out here? Is it so that after a long time the connections are fair distributed or is
there always a starvation of the other connections?

Stef

--

[email protected]
"Using Linux as bandwidth manager"
http://www.docum.org/
#lartc @ irc.oftc.net

--__--__--

Message: 5
From: =?iso-8859-1?Q?John_Bäckstrand?= <[email protected]>
To: "Stef Coene" <[email protected]>,
"Ricardo Jorge da Fonseca Marques Ferreira" <[email protected]>, <[email protected]>
Subject: Re: [LARTC] Eureka!
Date: Mon, 28 Apr 2003 22:06:23 +0200

Quoted 1 line

On Monday 28 April 2003 00:20, Ricardo Jorge da

Fonseca Marques Ferreira

Quoted 2 lines
Quoted 1 line · wrote:

I finally figured out what in emule was ruining my

network.

Quoted 2 lines
Quoted 2 lines

I tried everything and nothing worked. I was always

getting timeouts. Until

Quoted 1 line
Quoted 1 line

i went into the extended settings in emule and

changed connections per 5

Quoted 1 line
Quoted 1 line

seconds to 5 from the default of 20. Miracle. It

now works great.

Quoted 2 lines
Quoted 2 lines

So apparently lots of connection attempts in a few

seconds is an effective

Quoted 1 line
Quoted 1 line

way of starving all other machines behind a linux

gateway even if you're

Quoted 2 lines
Quoted 1 line

using SFQ.

Mhh. I wonder how this can be. Some tcp connection

specialists out here?

Quoted 1 line

Is it so that after a long time the connections are

fair distributed or is

Quoted 3 lines

there always a starvation of the other connections?

Stef

It could be because SFQ only equalizes flows. If you open lots of connections then, it means every connection will get worse. If this is the problem, this will not happen with ESFQ, afaik.

---
John Bäckstrand

--__--__--

Message: 6
From: [email protected] (Don Cohen)
Date: Mon, 28 Apr 2003 13:05:31 -0700
To: [email protected]
Subject: [LARTC] proxy arp problem

Proxy arp seems to be the right thing for answering arp queries but it doesn't solve the problem of decaching old cache entries. Here's the example:

router --- hub --- host

             |
          firewall --

The gateway for host is router. First I use the host to communicate with the outside world, so it and the router have each other in their arp caches.

Now I move the host down to the other side of the firewall, which is doing arp proxy. In order to do that, of course, I have to add a route to the firewall. At this point the host and router can not talk to each other because each is trying to use the mac address in its arp cache. Ideally I'd like to solve this problem without having to log in to either the router or host. One way, of course, is to wait for the arp caches to time out. I thought this was ok, cause I thought these were very short lived, but in the case I currently face that timeout on the router is 4 hours (!!) (Is this common? Or even reasonable? Anyone know what values are in common use?)

I notice rfc1122 (p22) suggests timeouts on the order of a minute, which is what I expected.

If the host has a shorter timeout then here's a possible solution: I *suspect* that when a machine sends an arp request to the router: from [senderMAC] to [BroadcastMAC] who has [routerIP] tell [senderIP] the router stores in its arp cache the association [senderIP]=[senderMAC]

Of course, the client machine now sends that packet to the firewall. The firewall sees this, stores [senderIP]=[senderMAC] in its own cache, and replies [routerIP] is at [MAC of firewall interface connected to client] What I want to add is that the firewall should also send out the interface connected to the router: from [MAC of firewall interface connected to router] to [BroadcastMAC] who has [routerIP] tell [clientIP] This will then elicit a reply from the router, but in fact the firewall can ignore that since it will send its own arp request to the router.

Even better would be some mechanism that, when we add the route to the firewall could figure out who on either end should be sent such packets. Perhaps it could remember all of the arp requests that it had seen (these are, after all, sent to broadcast, so it wouldn't need to be in promiscuous mode) it could see from the new route (1) which ip addresses had been used in that range, (2) which ip addresses those addresses had tried to talk to directly.

I'll be interested in any other ideas, and even more interested in anything that has already been implemented to solve this problem.

--__--__--

Message: 7
From: Ricardo Jorge da Fonseca Marques Ferreira <[email protected]>
To: Stef Coene <[email protected]>, <[email protected]>
Subject: Re: [LARTC] Eureka!
Date: Mon, 28 Apr 2003 21:30:41 +0100

Quoted 11 lines · Stef Coene

On Monday 28 April 2003 00:20, Ricardo Jorge da Fonseca Marques Ferreira

Quoted 7 lines · wrote:

I finally figured out what in emule was ruining my network.

I tried everything and nothing worked. I was always getting timeouts. Until i went into the extended settings in emule and changed connections per 5 seconds to 5 from the default of 20. Miracle. It now works great.

Quoted 7 lines
Quoted 3 lines

So apparently lots of connection attempts in a few seconds is an effective way of starving all other machines behind a linux gateway even if you're using SFQ.

Mhh. I wonder how this can be. Some tcp connection specialists out here? Is it so that after a long time the connections are fair distributed or is there always a starvation of the other connections?

In my experience the connections never get fairly distributed. I can reproduce
this just increasing the number of connections per 5 seconds to 20 again. The
effect is imediate.

I get connection timeouts in HTTP (more apparent, i'm sure other protocols
suffer too) and have to reload a series of times before the connection "goes"
and even then it then fails for some other elements in the page.

--__--__--

Message: 8
From: Ricardo Jorge da Fonseca Marques Ferreira <[email protected]>
To: John =?iso-8859-1?q?Bäckstrand?= <[email protected]>,
"Stef Coene" <[email protected]>, <[email protected]>
Subject: Re: [LARTC] Eureka!
Date: Mon, 28 Apr 2003 22:18:55 +0100

Quoted 4 lines · John Bäckstrand

It could be because SFQ only equalizes flows. If you open lots of connections then, it means every connection will get worse. If this is the problem, this will not happen with ESFQ, afaik.

Hmm, is there any info on the web about ESFQ & can it be used with tcng whi= ch is what i'm using now to generate my tc scripts ?

--__--__--

Message: 9
From: Muhammad Al-Gharribeh <[email protected]>
Date: Mon, 28 Apr 2003 20:20:30 -0400 (EDT)
To: <[email protected]>
Subject: [LARTC] PCI IRQ conflict

Actually, I have NetVista 4 IBM personal Computers with Linux OS Red Hat 8, my research requires installing an extra three Ethernet PCI cards on each machine and I've installed those cards on my boxes. But I have a problem that all the new NIC cards are sharing the same IRQ Number. I have been searching to solve this problem but unfortunately I coudn't find anything useful . Could you please help me to solve this problem? Thanks so much in advance for your reply and I appreciate your help. Regards Al-Gharribeh Muhammad

--__--__--

Message: 10
From: Muhammad Al-Gharribeh <[email protected]>
Date: Mon, 28 Apr 2003 20:20:42 -0400 (EDT)
To: <[email protected]>
Subject: [LARTC] PCI IRQ conflict

Actually, I have NetVista 4 IBM personal Computers with Linux OS Red Hat 8, my research requires installing an extra three Ethernet PCI cards on each machine and I've installed those cards on my boxes. But I have a problem that all the new NIC cards are sharing the same IRQ Number. I have been searching to solve this problem but unfortunately I coudn't find anything useful . Could you please help me to solve this problem? Thanks so much in advance for your reply and I appreciate your help. Regards Al-Gharribeh Muhammad

--__--__--

_______________________________________________
LARTC mailing list
[email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc

End of LARTC Digest

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/