Skip to contentSkip to conversation

multipath routing problem [Shorter version] - Help

LARTC Vincent Jaussaud 3 participants 15 messages Oct 25, 2002 – Oct 29, 2002

lore
Share & tools
lore
#0 lore raw
Quoted 8 lines · Arthur van Leeuwen
Quoted 5 lines · Vincent Jaussaud

When only one gateway is used to reach remote networks, everything is working just fine. (Whatever gateway we choose to use) Whenever we attempt to activate multipath routing over both gateways, then SSH don't work anymore. We can ping, traceroute, telnet, ... but not SSH nor FTP (PASV).

Quoted 5 lines

ssh tends to play with TOS fields (and rightly so). Routing is keyed to the *triple* (src, dst, tos), something that most people (including me) normally forget. However, in this particular case that may be the reason for your ssh's breaking.

Hmm... that's really interesting. Thanks for the pointer. I remember now that I've read something regarding SSH & TOS field some days ago. If I'm right, it use the Minimum Delay TOS value.

Now, how am I suppose to deal with this TOS issue ? What TOS value should do the trick ?
I'm using a 2.2 kernel with ipchains.

Quoted 3 lines

The reason for FTP breaking possibly has to do with packets for the control connection going out the one gateway and for the data going out the other... but this is speculation on my part.

That sounds wise. However, routes are suppose to be cached using the src IP field as well (If I'm not mistaken), so that every packets coming from a particular IP are likely to take the same route than the others. Am I wrong ?

A BIG Thanks for your reply :-)
Cheers,
Vincent.

Quoted 7 lines

Doei, Arthur.

Signature · 3 lines
  /\    / |      [email protected]      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching
Signature · 10 lines
Vincent Jaussaud
Kelkoo.com Security Manager 
email: [email protected]

"The UNIX philosophy is to design small tools that do one thing, and do
it well."

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#1 lore raw

Hello,

Quoted 11 lines · Vincent Jaussaud
Quoted 5 lines

ssh tends to play with TOS fields (and rightly so). Routing is keyed to the *triple* (src, dst, tos), something that most people (including me) normally forget. However, in this particular case that may be the reason for your ssh's breaking.

Hmm... that's really interesting. Thanks for the pointer. I remember now that I've read something regarding SSH & TOS field some days ago. If I'm right, it use the Minimum Delay TOS value.

Now, how am I suppose to deal with this TOS issue ? What TOS value should do the trick ?

In theory, you should not reach multipath route for
traffic that is already NAT-ed. May be you have to fix your routes. The TOS field plays in the input routing performed on forwarding and traffic between two public IP addresses can select different nexthop if the TOS is different or if the routing cache is somehow flushed (on route/address add/del, expiration).

Quoted 10 lines

I'm using a 2.2 kernel with ipchains.

Quoted 3 lines

The reason for FTP breaking possibly has to do with packets for the control connection going out the one gateway and for the data going out the other... but this is speculation on my part.

That sounds wise. However, routes are suppose to be cached using the src IP field as well (If I'm not mistaken), so that every packets coming from a particular IP are likely to take the same route than the others. Am I wrong ?

Yes, TOS is a routing key just like SADDR and DADDR.
By using multipath route between 2 IP addresses you agree that the packets can _safely_ choose any of the paths. When using two or more ISPs you simply can't do this if the ISPs have source spoofing disabled. In such cases only the traffic that is NAT-ed from your box has the right to use the multipath route. This is a key requirement for the patches you are using. Once the NAT connections are established they don't hit multipath route.

Quoted 3 lines

A BIG Thanks for your reply :-)
Cheers,
Vincent.

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#2 lore raw
In reply to Vincent Jaussaud
Quoted 2 lines · Julian Anastasov

Hello,

Hi julian !

Quoted 15 lines · Vincent Jaussaud
Quoted 6 lines
Quoted 6 lines

ssh tends to play with TOS fields (and rightly so). Routing is keyed to the *triple* (src, dst, tos), something that most people (including me) normally forget. However, in this particular case that may be the reason for your ssh's breaking.

Quoted 6 lines

Hmm... that's really interesting. Thanks for the pointer. I remember now that I've read something regarding SSH & TOS field some days ago. If I'm right, it use the Minimum Delay TOS value.

Now, how am I suppose to deal with this TOS issue ? What TOS value should do the trick ?

Quoted 6 lines

In theory, you should not reach multipath route for
traffic that is already NAT-ed. May be you have to fix your routes. The TOS field plays in the input routing performed on forwarding and traffic between two public IP addresses can select different nexthop if the TOS is different or if the routing cache is somehow flushed (on route/address add/del, expiration).

But traffic is NAT-ed after multipath routing occurs !
Eg, the box which do multipath routing do not NAT traffic; traffic get NAT-ed when leaving the gateways:

LAN --> FW w/ multipath-routing

	   |		|
	  Gateway1  Gateway2
	   | (NAT)	| (NAT) 
	   |		|
	-------------------- Remote Network

Packets reach the Remote Network using one of the Gateway NAT-ed IP, so that when packets come back they should use the proper return path. Am I wrong ?

Quoted 12 lines
Quoted 11 lines

I'm using a 2.2 kernel with ipchains.

Quoted 3 lines

The reason for FTP breaking possibly has to do with packets for the control connection going out the one gateway and for the data going out the other... but this is speculation on my part.

That sounds wise. However, routes are suppose to be cached using the src IP field as well (If I'm not mistaken), so that every packets coming from a particular IP are likely to take the same route than the others. Am I wrong ?

Quoted 10 lines

Yes, TOS is a routing key just like SADDR and DADDR.
By using multipath route between 2 IP addresses you agree that the packets can _safely_ choose any of the paths. When using two or more ISPs you simply can't do this if the ISPs have source spoofing disabled. In such cases only the traffic that is NAT-ed from your box has the right to use the multipath route. This is a key requirement for the patches you are using. Once the NAT connections are established they don't hit multipath route.

Hmmm... Then this is where the problem is. So, if I understand correctly, packets coming from a single TCP connections will use any of the multipath route _if_ they are not NAT-ed ? Isn't the routing cache suppose to ensure that every packets coming from a single connection use the same path ?

Now, If I understand the whole topic, in order to fix my problem, I need to:

- NAT everything on the FW itself
- or disable NAT on both gateways, and ensure that routing is done
properly

Am I right ? I'm becomming a bit lost, here :-\

Many Thanks for your time.
Vincent.

Quoted 12 lines
Quoted 3 lines

A BIG Thanks for your reply :-)
Cheers,
Vincent.

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Signature · 10 lines
Vincent Jaussaud
Kelkoo.com Security Manager 
email: [email protected]

"The UNIX philosophy is to design small tools that do one thing, and do
it well."

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#3 lore raw
In reply to Vincent Jaussaud

Hello,

Quoted 11 lines · Vincent Jaussaud

But traffic is NAT-ed after multipath routing occurs !
Eg, the box which do multipath routing do not NAT traffic; traffic get NAT-ed when leaving the gateways:

LAN --> FW w/ multipath-routing

	   |		|
	  Gateway1  Gateway2
	   | (NAT)	| (NAT)
	   |		|
	-------------------- Remote Network
Quoted 3 lines

Packets reach the Remote Network using one of the Gateway NAT-ed IP, so that when packets come back they should use the proper return path. Am I wrong ?

Now I see, then the TOS is a big problem for you. May
be your problem will be solved if TOS is not a routing key but it does not sound as a thing that is easy to fix in kernel.

Quoted 3 lines

Hmmm... Then this is where the problem is. So, if I understand correctly, packets coming from a single TCP connections will use any of the multipath route _if_ they are not NAT-ed ? Isn't the routing cache

	Yes, traffic A->B with TOS=XXX will use one path, traffic

A->B with TOS=YYY can use different path.

Quoted 2 lines

suppose to ensure that every packets coming from a single connection use the same path ?

The routing does not cache connections but route resolutions keyed
by saddr, daddr, tos, etc. Even there are no TCP/UDP ports (yet?).

Quoted 4 lines

Now, If I understand the whole topic, in order to fix my problem, I need to:

- NAT everything on the FW itself

This is a solution

Quoted 2 lines

- or disable NAT on both gateways, and ensure that routing is done
properly

Not sure what you mean. But you can run multipath routes
on the both gateways. Then the internal hosts can select any of the gateways as default (or to use alternative default gateways). Another solution is your apps not to change the TOS, it can be changed at the border gateways (if useful at all).

Quoted 4 lines

Am I right ? I'm becomming a bit lost, here :-\

Many Thanks for your time.
Vincent.

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#4 lore raw
In reply to Vincent Jaussaud
Quoted 24 lines · Julian Anastasov

Hello,

Quoted 11 lines · Vincent Jaussaud

But traffic is NAT-ed after multipath routing occurs !
Eg, the box which do multipath routing do not NAT traffic; traffic get NAT-ed when leaving the gateways:

LAN --> FW w/ multipath-routing

	   |		|
	  Gateway1  Gateway2
	   | (NAT)	| (NAT)
	   |		|
	-------------------- Remote Network
Quoted 3 lines

Packets reach the Remote Network using one of the Gateway NAT-ed IP, so that when packets come back they should use the proper return path. Am I wrong ?

Now I see, then the TOS is a big problem for you. May
be your problem will be solved if TOS is not a routing key but it does not sound as a thing that is easy to fix in kernel.

Quoted 7 lines
Quoted 3 lines

Hmmm... Then this is where the problem is. So, if I understand correctly, packets coming from a single TCP connections will use any of the multipath route _if_ they are not NAT-ed ? Isn't the routing cache

	Yes, traffic A->B with TOS=XXX will use one path, traffic

A->B with TOS=YYY can use different path.

Ok, I understand a lot of things now, thanks to you. However, I don't get why, in the same SSH session, TOS may differ from one packet to another. Using tcpdump, it seems that TOS value change right after the authentication has been successfully made.

Quoted 6 lines
Quoted 2 lines

suppose to ensure that every packets coming from a single connection use the same path ?

The routing does not cache connections but route resolutions keyed
by saddr, daddr, tos, etc. Even there are no TCP/UDP ports (yet?).

Ok.

Quoted 6 lines
Quoted 4 lines

Now, If I understand the whole topic, in order to fix my problem, I need to:

- NAT everything on the FW itself

This is a solution

Yes, but unfortunately, and regarding my current networks architecture, this will require massive changes in the firewall ACLs / routing rules. Currently, this is not an option.

Quoted 7 lines
Quoted 3 lines

- or disable NAT on both gateways, and ensure that routing is done
properly

Not sure what you mean. But you can run multipath routes
on the both gateways. Then the internal hosts can select any of the gateways as default (or to use alternative default gateways).

Each gateway only have one route to the remote network. So I don't see where the multipath routing will send packets to, except maybe to the other gateway. But this may become a big problem, in case all tunnels to the remote networks are down; this will create a routing loop.

Quoted 3 lines

Another solution is your apps not to change the TOS, it can be changed at the border gateways (if useful at all).

Actually, and thanks to you, now I know where the problem is. Considering that doing NAT on the firewall itself is not possible, and that disabling NAT on both gateways will quickly become a big headache, because of the current networks setup, I think of adding another layout of NAT on the final gateway.

Eg, something like:

LAN --> FW w/ multipath-routing

 	   |		|
 	  Gateway1  Gateway2
 	   | (NAT)	| (NAT)
 	   |		|
        Remote Gateway (other side)
	   	| (NAT)
		|

-------------------- Remote Network

This'll ensure that all packets arriving to the remove network comes from the same IP address, whatever path they used.

What do you think of such setup ? Is it likely to work ?

Thanks again.
Regards,
Vincent.

Quoted 13 lines
Quoted 4 lines

Am I right ? I'm becomming a bit lost, here :-\

Many Thanks for your time.
Vincent.

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Signature · 10 lines
Vincent Jaussaud
Kelkoo.com Security Manager 
email: [email protected]

"The UNIX philosophy is to design small tools that do one thing, and do
it well."

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#5 lore raw
In reply to Vincent Jaussaud
Quoted 15 lines · Julian Anastasov

Hello,

Quoted 11 lines · Vincent Jaussaud

But traffic is NAT-ed after multipath routing occurs !
Eg, the box which do multipath routing do not NAT traffic; traffic get NAT-ed when leaving the gateways:

LAN --> FW w/ multipath-routing

	   |		|
	  Gateway1  Gateway2
	   | (NAT)	| (NAT)
	   |		|
	-------------------- Remote Network
Quoted 7 lines
Quoted 3 lines

Packets reach the Remote Network using one of the Gateway NAT-ed IP, so that when packets come back they should use the proper return path. Am I wrong ?

Now I see, then the TOS is a big problem for you. May
be your problem will be solved if TOS is not a routing key but it does not sound as a thing that is easy to fix in kernel.

Actually, you can simply play whack-a-mole with the TOS value, using ipchains (or iptables), killing all TOS values present on the packets. Ofcourse, this is not very *nice*, but it'll work.

Doei, Arthur.

Signature · 7 lines
  /\    / |      [email protected]      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#6 lore raw
In reply to Vincent Jaussaud
Quoted 3 lines · Vincent Jaussaud

However, I don't get why, in the same SSH session, TOS may differ from one packet to another. Using tcpdump, it seems that TOS value change right after the authentication has been successfully made.

Shit... you figured that one out *quite* a bit faster than I did at the time... took me two weeks.

What openssh does is first authenticate, then set the TOS value depending on whether you're doing interactive communications (ssh) or bulk transfer (scp). One could see this as a way of minimizing information leakage...

Oh, and yes, it does what you deduced. I finally got that from reading the sources...

Doei, Arthur.

Signature · 7 lines
  /\    / |      [email protected]      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#7 lore raw
In reply to Vincent Jaussaud
Quoted 9 lines · Arthur van Leeuwen
Quoted 3 lines · Vincent Jaussaud

However, I don't get why, in the same SSH session, TOS may differ from one packet to another. Using tcpdump, it seems that TOS value change right after the authentication has been successfully made.

Shit... you figured that one out *quite* a bit faster than I did at the time... took me two weeks.

:-)

Quoted 4 lines

What openssh does is first authenticate, then set the TOS value depending on whether you're doing interactive communications (ssh) or bulk transfer (scp). One could see this as a way of minimizing information leakage...

OK, now I know why openssh is changing it's TOS !. Thanks. :-)

Quoted 2 lines

Oh, and yes, it does what you deduced. I finally got that from reading the sources...

I could mangle the TOS field as you suggested, but I don't like this, since packets *should* be able to find their way out, whatever path they use to come back.

The thing I don't understand, is that even by NAT'ing everything, everywhere, my connections still break.

I've tried to NAT on the firewall everything coming from a test IP, just to see how it goes. No luck.

I even tried NAT'ing on the firewall, then on the gateways, then on the final router, in the other network. Still no luck ! This is non sense !

There has to be something wrong, somewhere.

Thanks for your reply.
Regards,
Vincent.

Quoted 11 lines

Doei, Arthur.

Signature · 7 lines
  /\    / |      [email protected]      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Signature · 10 lines
Vincent Jaussaud
Kelkoo.com Security Manager 
email: [email protected]

"The UNIX philosophy is to design small tools that do one thing, and do
it well."

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#8 lore raw
In reply to Vincent Jaussaud

Hello,

Quoted 7 lines · Arthur van Leeuwen
Quoted 3 lines

Now I see, then the TOS is a big problem for you. May
be your problem will be solved if TOS is not a routing key but it does not sound as a thing that is easy to fix in kernel.

Actually, you can simply play whack-a-mole with the TOS value, using ipchains (or iptables), killing all TOS values present on the packets. Ofcourse, this is not very *nice*, but it'll work.

This is a good idea. Vincent, may be you can play with
ipchains -t AND XOR in the input chain to see what happens. Just make sure you don't touch bits 0, 1, 5, 6, 7. It seems the routing uses only bits 2, 3 and 4 for routing key (if I'm not overlooking something). This is for kernel 2.4. For kernel 2.2 it seems bit 1 is also included in the routing key.

2.4 mask 0x1C, inverted 0xE3
2.2 mask 0x1E, inverted 0xE1

	So, for 2.2 may be:

ipchains -I input -d 0.0.0.0/0 22 -t 0xE3 0x00

What are the TOS values used during the SSH session?

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#9 lore raw
In reply to Vincent Jaussaud
Quoted 20 lines · Julian Anastasov

Hello,

Quoted 7 lines · Arthur van Leeuwen
Quoted 3 lines

Now I see, then the TOS is a big problem for you. May
be your problem will be solved if TOS is not a routing key but it does not sound as a thing that is easy to fix in kernel.

Actually, you can simply play whack-a-mole with the TOS value, using ipchains (or iptables), killing all TOS values present on the packets. Ofcourse, this is not very *nice*, but it'll work.

This is a good idea. Vincent, may be you can play with
ipchains -t AND XOR in the input chain to see what happens. Just make sure you don't touch bits 0, 1, 5, 6, 7. It seems the routing uses only bits 2, 3 and 4 for routing key (if I'm not overlooking something). This is for kernel 2.4. For kernel 2.2 it seems bit 1 is also included in the routing key.

Quoted 6 lines

2.4 mask 0x1C, inverted 0xE3
2.2 mask 0x1E, inverted 0xE1

	So, for 2.2 may be:

ipchains -I input -d 0.0.0.0/0 22 -t 0xE3 0x00

Just tried. Now SSH connections don't break anymore !!! :) Thanks ! Am I suppose to do this on both side, or doing this on the firewall itself is enough ?

Quoted 2 lines

What are the TOS values used during the SSH session?

Right after authentication, TOS value is set to 0x10

20:53:46.515566 192.168.0.2.ssh > 172.1.1.3.2418: R 4008315859:4008315859(0) win 0 [tos 0x10]

The only problem with this, is that I will need to do this trick for any applications changing it's TOS during the session. It seems that FTP behaves exactly the same way as SSH, regarding the TOS field.

Do you guys know if many applications do this ? Or is this just particular to SSH & FTP ?

Anyway, I really would like to understand why it doesn't work when doing NAT.

A big thanks to both of you. I've learned a lot today :)

Thanks again.
Regards,
Vincent.

Quoted 5 lines

Regards

--
Julian Anastasov <[email protected]>

Signature · 10 lines
Vincent Jaussaud
Kelkoo.com Security Manager 
email: [email protected]

"The UNIX philosophy is to design small tools that do one thing, and do
it well."

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#10 lore raw
In reply to Vincent Jaussaud

Hello,

Quoted 9 lines · Vincent Jaussaud
Quoted 6 lines

2.4 mask 0x1C, inverted 0xE3
2.2 mask 0x1E, inverted 0xE1

	So, for 2.2 may be:

ipchains -I input -d 0.0.0.0/0 22 -t 0xE3 0x00

Just tried. Now SSH connections don't break anymore !!! :) Thanks ! Am I suppose to do this on both side, or doing this on the firewall itself is enough ?

I now see that my example ipchains command is wrong,
use 0xE1 for 2.2 as the above table.

Quoted 3 lines

The only problem with this, is that I will need to do this trick for any applications changing it's TOS during the session. It seems that FTP behaves exactly the same way as SSH, regarding the TOS field.

It seems you can safely alter the TOS for all packets
entering your box/site.

Quoted 2 lines

Do you guys know if many applications do this ? Or is this just particular to SSH & FTP ?

The TOS is usually used for routing between routers in your
site, then the border gateways can assign different priorities based on the TOS values, for traffic control purposes.

Quoted 2 lines

Anyway, I really would like to understand why it doesn't work when doing NAT.

May be you can hunt it with tcpdump. I assume your are
using the patches because the plain kernel has the same problem for NAT.

Quoted 5 lines

A big thanks to both of you. I've learned a lot today :)

Thanks again.
Regards,
Vincent.

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#11 lore raw
In reply to Vincent Jaussaud
Quoted 4 lines

It seems you can safely alter the TOS for all packets
entering your box/site.

Ok, I'll dig into this tip, and see how it goes. If I can't figure out this NAT problem, I'll do this.

Quoted 4 lines

May be you can hunt it with tcpdump. I assume your are
using the patches because the plain kernel has the same problem for NAT.

Yes, I am running your patch. Kernel is 2.2.22 with routes-2.2.20-7.diff patch applied. (I'm sure of this, otherwise dead gateway detection will simply not work.)

My question is, if we ensure that EVERY packets, whatever path they use to arrive, finally pass through a single peer doing NAT, is this suppose to work around my TOS problem ?

Eg, end services will only see packets coming from the last NAT address, which is single whatever path packets used to arrive.

Something like:

LAN --> Multipath Firewall

	  |		|
	 GW1		GW2
	  |		|
	-------------------
		|
	      Gateway
	       (NAT)
		|
	    --------- Remote Network

What about the rp_filter kernel value ? Could it be a problem in such setup ?

Thanks again.
Vincent.

Quoted 14 lines
Quoted 5 lines

A big thanks to both of you. I've learned a lot today :)

Thanks again.
Regards,
Vincent.

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Signature · 10 lines
Vincent Jaussaud
Kelkoo.com Security Manager 
email: [email protected]

"The UNIX philosophy is to design small tools that do one thing, and do
it well."

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#12 lore raw
In reply to Vincent Jaussaud

Hello,

Quoted 3 lines · Vincent Jaussaud

My question is, if we ensure that EVERY packets, whatever path they use to arrive, finally pass through a single peer doing NAT, is this suppose to work around my TOS problem ?

Sounds correct. The requirement is each packet from one
connection to be NAT-ed only from one NAT router and to same masquerade address and port. The routing cache can not guarantee that. It can be done only from the patched masquerade.

Quoted 2 lines

What about the rp_filter kernel value ? Could it be a problem in such setup ?

The patches are designed to work with rp_filter enabled.
You can safely use it, it is changed to work only with the defined paths.

Quoted 2 lines

Thanks again.
Vincent.

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#13 lore raw
In reply to Vincent Jaussaud
Quoted 5 lines · Julian Anastasov

Hello,

On 28 Oct 2002, Vincent Jaussaud wrote:

Quoted 9 lines
Quoted 3 lines

My question is, if we ensure that EVERY packets, whatever path they use to arrive, finally pass through a single peer doing NAT, is this suppose to work around my TOS problem ?

Sounds correct. The requirement is each packet from one
connection to be NAT-ed only from one NAT router and to same masquerade address and port. The routing cache can not guarantee that. It can be done only from the patched masquerade.

Hmmm.. then that's why it doesn't work.. final gateway doing NAT isn't patched, only the first one is.

I think I'll have to drop the idea of using both gateways simultaneously.
Now, If I only want do to fail-over (eg; only one gateway used at the same time, other one used only in case the first one breaks.)

I was thinking about using the metric value for this.

Let's say:

ip route add table dual-gw proto static 192.168.0.0/24 via GW1 dev eth1 metric 1
ip route add table dual-gw proto static 192.168.0.0/24 via GW2 dev eth1 metric 2

I assume the kernel will always use the best route, that is the one with best metric. So that all packets will use the same route.
If GW1 breaks, patched kernel should mark first route as dead, and force all further packets to use GW2 instead.

Is this suppose to work ? Or can we use different metric value inside a multipath route, like:

ip route add table dual-gw proto static 192.168.0.0/24 nexthop via GW1 dev eth1 metric 1 nexthop via GW2 dev eth1 metric 2

?

Anyway, the more I think about this setup, the more I think I should use a clustering solution instead. Maybe a cluster of gateway with one VIP is much more appropriate for what I want to build. I'll use multipath routing for ISP redundency then :)

Thanks to both of you, I've learn a lot during the last past few days, this was one of my main concern too.

Thanks again.
Cheers,
Vincent.

Quoted 18 lines
Quoted 2 lines

What about the rp_filter kernel value ? Could it be a problem in such setup ?

The patches are designed to work with rp_filter enabled.
You can safely use it, it is changed to work only with the defined paths.

Quoted 2 lines

Thanks again.
Vincent.

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Signature · 10 lines
Vincent Jaussaud
Kelkoo.com Security Manager 
email: [email protected]

"The UNIX philosophy is to design small tools that do one thing, and do
it well."

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#14 lore raw
In reply to Vincent Jaussaud

Hello,

Quoted 8 lines · Vincent Jaussaud

I was thinking about using the metric value for this.

Let's say:

ip route add table dual-gw proto static 192.168.0.0/24 via GW1 dev eth1 metric 1
ip route add table dual-gw proto static 192.168.0.0/24 via GW2 dev eth1 metric 2

No, the metrics work only if the routes disappear.
This usually happens when device goes down (for example, ppp). For gateways reachable via ARP it can't work.

You need to define alternative routes (ip route append), see
my docs about alt routes that use same metric.

Quoted 4 lines

I assume the kernel will always use the best route, that is the one with best metric. So that all packets will use the same route.
If GW1 breaks, patched kernel should mark first route as dead, and force all further packets to use GW2 instead.

No, dead gateway detection currently works for routes with
same metric. But even then the detection is passive and needs help from user space. Without such checks you can expect almost random results. OTOH, you can run your own checks and to keep only the alive routes.

Regards

--
Julian Anastasov <[email protected]>

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/