Skip to contentSkip to conversation

problems with priority routing + masq + ntp. (fwd)

LARTC tc lewis [email protected] 1 participant 1 message Nov 16, 2000

lore
Share & tools
lore

tc lewis [email protected]

author
#0 lore raw

<PRE>this doesn't involve 2.4, but i just found that you guys have a mailing list too, so maybe some of you might have ideas of what i'm doing wrong here...

-tcl.

---------- Forwarded message ----------
Date: Wed, 15 Nov 2000 21:16:04 -0500 (EST)
From: tc lewis &lt;<A HREF="mailto:[email protected]">[email protected]</A>&gt;
To: <A HREF="mailto:[email protected]">[email protected]</A>
Subject: [ot] problems with priority routing + masq + ntp.

not much of an lvs issue again, but hey, you guys are smart.

i've got a machine that i'm using as a mail server and name server behind lvs using dr. that machine also needs to get to the outside world, but it doesn't have a real ip, so it goes through a separate masq server for that. all works well, but not now that i'm trying to make it an ntp server also. here's the setup:

jfk: lvs
lga: masq
phl: real server in question.

- - - - -

jfk: 192.168.1.2 on eth1. a few real ips on eth2 for incoming service requests which get directed to misc real servers. gateway 64.211.224.161 via eth2:

[<A HREF="mailto:root@jfk">root@jfk</A> /root]# /sbin/route -n Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
64.211.224.166  0.0.0.0         255.255.255.255 UH    0      0        0 eth2
192.168.1.2     0.0.0.0         255.255.255.255 UH    0      0        0 eth1
64.211.224.165  0.0.0.0         255.255.255.255 UH    0      0        0 eth2
64.211.224.162  0.0.0.0         255.255.255.255 UH    0      0        0 eth2
64.211.224.163  0.0.0.0         255.255.255.255 UH    0      0        0 eth2
192.168.0.2     0.0.0.0         255.255.255.255 UH    0      0        0 eth2
64.211.224.160  0.0.0.0         255.255.255.240 U     0      0        0 eth2
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth2
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
0.0.0.0         64.211.224.161  0.0.0.0         UG    0      0        0 eth2

[<A HREF="mailto:root@jfk">root@jfk</A> /root]# /sbin/ipchains -L -n ; /sbin/ipvsadm -L -n Chain input (policy ACCEPT):
Chain forward (policy DENY):
Chain output (policy ACCEPT):
IP Virtual Server version 1.0.0-beta1 (size@96)
Prot LocalAddress:Port Scheduler Flags
-&gt; RemoteAddress:Port Forward Weight ActiveConn InActConn
UDP 64.211.224.162:53 lc
-&gt; 192.168.1.11:53 Route 1 0 0
UDP 64.211.224.163:53 lc
-&gt; 192.168.1.12:53 Route 1 0 0
TCP 64.211.224.166:22 lc
-&gt; 192.168.1.21:22 Route 1 1 0
TCP 64.211.224.166:25 lc
-&gt; 192.168.1.21:25 Route 1 0 0
TCP 64.211.224.165:80 lc persistent 360

  -&gt; 192.168.1.101:80            Route   1      0          0         
  -&gt; 192.168.1.102:80            Route   1      0          0         

[<A HREF="mailto:root@jfk">root@jfk</A> /root]# cat /etc/sysctl.conf # Disables packet forwarding
net.ipv4.ip_forward = 1
# Enables source route verification net.ipv4.conf.all.rp_filter = 1
# Disables automatic defragmentation (needed for masquerading, LVS) net.ipv4.ip_always_defrag = 0
# Disables the magic-sysrq key
kernel.sysrq = 1

# -tcl.
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.eth1.send_redirects = 0
net.ipv4.conf.eth2.send_redirects = 0
#

- - - - -

lga: 192.168.1.1 on eth1. a few real ips on eth2 for outgoing masquerading. gateway 64.211.224.161 on eth2.

[<A HREF="mailto:root@lga">root@lga</A> /root]# /sbin/route -n Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.1     0.0.0.0         255.255.255.255 UH    0      0        0 eth1
64.211.224.167  0.0.0.0         255.255.255.255 UH    0      0        0 eth2
64.211.224.164  0.0.0.0         255.255.255.255 UH    0      0        0 eth2
192.168.0.1     0.0.0.0         255.255.255.255 UH    0      0        0 eth2
64.211.224.160  0.0.0.0         255.255.255.240 U     0      0        0 eth2
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth2
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
0.0.0.0         64.211.224.161  0.0.0.0         UG    0      0        0 eth2
[<A HREF="mailto:root@lga">root@lga</A> /root]# /sbin/ipchains -L -n ; /sbin/ipvsadm -L -n
Chain input (policy ACCEPT):
Chain forward (policy DENY):
target     prot opt     source                destination           ports
MASQ       all  ------  192.168.1.41         0.0.0.0/0             n/a
MASQ       all  ------  192.168.1.21         0.0.0.0/0             n/a

Chain output (policy ACCEPT):
[<A HREF="mailto:root@lga">root@lga</A> /root]# cat /etc/sysctl.conf # Disables packet forwarding
net.ipv4.ip_forward = 1
# Enables source route verification net.ipv4.conf.all.rp_filter = 1
# Disables automatic defragmentation (needed for masquerading, LVS) net.ipv4.ip_always_defrag = 0
# Disables the magic-sysrq key
kernel.sysrq = 1

# -tcl.
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.eth1.send_redirects = 0
net.ipv4.conf.eth2.send_redirects = 0
#

- - - - -

phl: 192.168.1.21 on eth0. other 192.168.1 addresses on eth0 also. lots of other ips on other interfaces for internal network stuff. no gateway in the routing table -- that's handled with iproute2 so that requests via lvs go back through the 64.211.224.161 gateway directly, but connections originating on phl go through 192.168.1.1 for masquerading. this config works for incoming dns and mail requests, and works for outgoing traffic such as nameservice requests, outgoing mail, pings, and other normal outgoing traffic.

[<A HREF="mailto:root@phl">root@phl</A> /root]# /sbin/route -n Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.2.10    0.0.0.0         255.255.255.255 UH    0      0        0 eth1
192.168.2.13    0.0.0.0         255.255.255.255 UH    0      0        0 eth1
192.168.1.21    0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.3.21    0.0.0.0         255.255.255.255 UH    0      0        0 eth2
64.211.224.162  0.0.0.0         255.255.255.255 UH    0      0        0 lo
64.211.224.163  0.0.0.0         255.255.255.255 UH    0      0        0 lo
192.168.2.14    0.0.0.0         255.255.255.255 UH    0      0        0 eth1
192.168.1.11    0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.1.10    0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.3.10    0.0.0.0         255.255.255.255 UH    0      0        0 eth2
192.168.1.13    0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.3.13    0.0.0.0         255.255.255.255 UH    0      0        0 eth2
192.168.2.21    0.0.0.0         255.255.255.255 UH    0      0        0 eth1
192.168.1.12    0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.1.14    0.0.0.0         255.255.255.255 UH    0      0        0 eth0
192.168.3.14    0.0.0.0         255.255.255.255 UH    0      0        0 eth2
64.211.224.160  0.0.0.0         255.255.255.240 U     0      0        0 eth0
192.168.3.0     0.0.0.0         255.255.255.0   U     0      0        0 eth2
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
[<A HREF="mailto:root@phl">root@phl</A> /root]# /sbin/ip rule show
0:      from all lookup local 
32766:  from all lookup main 
32767:  from all lookup 253 
33000:  from 192.168.1.0/24 lookup 100 
34000:  from all lookup 200 

[<A HREF="mailto:root@phl">root@phl</A> /root]# /sbin/ip route show table 100 default via 192.168.1.1 dev eth0
[<A HREF="mailto:root@phl">root@phl</A> /root]# /sbin/ip route show table 200 default via 64.211.224.161 dev eth0
[<A HREF="mailto:root@phl">root@phl</A> /root]# cat /etc/sysctl.conf # Disables packet forwarding
net.ipv4.ip_forward = 1
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Disables automatic defragmentation (needed for masquerading, LVS) net.ipv4.ip_always_defrag = 0
# Disables the magic-sysrq key
kernel.sysrq = 1

# -tcl.
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.eth0.send_redirects = 0
net.ipv4.conf.all.hidden = 1
net.ipv4.conf.lo.hidden = 1
#

- - - - -

so that's the setup. here's what happens when i try, for example, an ntpdate:

[<A HREF="mailto:root@phl">root@phl</A> /root]# /usr/sbin/ntpdate ntp.nasa.gov 15 Nov 18:10:25 ntpdate[1256]: no server suitable for synchronization found

tcpdump on phl:

18:10:21.231494   lo &gt; 127.0.0.1.1025 &gt; 127.0.0.1.domain: 14459+ A? ntp.nasa.gov. (30)
18:10:21.231494   lo &lt; 127.0.0.1.1025 &gt; 127.0.0.1.domain: 14459+ A? ntp.nasa.gov. (30)
18:10:21.231666   lo &gt; 127.0.0.1.domain &gt; 127.0.0.1.1025: 14459 2/0/0 CNAME nsipo.nasa.gov., A 143.232.55.13 (66)
18:10:21.231666   lo &lt; 127.0.0.1.domain &gt; 127.0.0.1.1025: 14459 2/0/0 CNAME nsipo.nasa.gov., A 143.232.55.13 (66)

18:10:21.331314 eth0 &gt; 192.168.1.21.ntp &gt; 143.232.55.13.ntp: v3 client strat 0 poll 4 prec -6 18:10:22.331286 eth0 &gt; 192.168.1.21.ntp &gt; 143.232.55.13.ntp: v3 client strat 0 poll 4 prec -6 18:10:23.331282 eth0 &gt; 192.168.1.21.ntp &gt; 143.232.55.13.ntp: v3 client strat 0 poll 4 prec -6 18:10:24.331281 eth0 &gt; 192.168.1.21.ntp &gt; 143.232.55.13.ntp: v3 client strat 0 poll 4 prec -6

tcpdump on lga and jfk reveal no traffic.

it should be getting to lga at least tho:

[<A HREF="mailto:root@phl">root@phl</A> /root]# /sbin/ip route get from 192.168.1.21 to 143.232.55.13 143.232.55.13 from 192.168.1.21 via 192.168.1.1 dev eth0

    cache  mtu 1500 rtt 375ms

but it's not.
i tried an ntpdate from another machine inside my network here on a machine with no advanced routing--just masquerading through 192.168.1.1 (that 192.168.1.41 ip if you refer to lga's ipchains table), and it works as expected:

[<A HREF="mailto:root@ith">root@ith</A> /root]# /usr/sbin/ntpdate ntp.nasa.gov 15 Nov 18:13:51 ntpdate[6907]: adjust time server 143.232.55.13 offset 0.071391 sec

any ideas on what i may be missing here?

-tcl.

</PRE>