Policy routing question
Hi,
I have a testing multihome setup, with the default gateway being one of the links and using policy routing to honor requests for a specific link. Everything works as expected when I request a specific IP to bind to. But if I request a specific interface things fall apart in ways that I can not explain:
default gw (WORKS)
----------
rabbit@Thesaurus:~$ ping -c 1 yahoo.com PING yahoo.com (216.109.112.135) 56(84) bytes of data. 64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 ttlH time2 ms
request IP on same link as default gw (WORKS)
-------------------------------------
rabbit@Thesaurus:~$ ping -I 192.168.9.102 -c 1 yahoo.com PING yahoo.com (66.94.234.13) from 192.168.9.102 : 56(84) bytes of data. 64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1 ttlG time6 ms
request IP on secondary link (WORKS)
----------------------------
rabbit@Thesaurus:~$ ping -I 172.16.0.2 -c 1 yahoo.com PING yahoo.com (216.109.112.135) from 172.16.0.2 : 56(84) bytes of data. 64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 ttlG time6 ms
request interface of default gw link (WORKS)
------------------------------------
rabbit@Thesaurus:~$ ping -I eth1 -c 1 yahoo.com PING yahoo.com (66.94.234.13) from 192.168.9.102 eth1: 56(84) bytes of data. 64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1 ttlG time6 ms
request secondary interface (FAILS)
---------------------------
rabbit@Thesaurus:~$ ping -I eth0 -c 1 yahoo.com PING yahoo.com (216.109.112.135) from 192.168.9.102 eth0: 56(84) bytes of data. From 172.16.0.2 icmp_seq=1 Destination Host Unreachable
I went over the setup again and again, but I can't figure out why the last ping attempt fails. Any pointers are welcome!
Thanks
Peter
Here is the setup:
ip addr
----------
2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:11:09:8d:4f:c1 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.2/24 brd 172.16.0.255 scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:04:e2:80:b4:97 brd ff:ff:ff:ff:ff:ff
inet 192.168.9.102/24 brd 192.168.9.255 scope global eth1ip ro show table all
-----------------------
default via 172.16.0.1 dev eth0 table 10
default via 192.168.9.1 dev eth1 table 20
default via 192.168.9.1 dev eth1 table default 172.16.0.0/24 dev eth0 proto kernel scope link src 172.16.0.2 192.168.9.0/24 dev eth1 proto kernel scope link src 192.168.9.102 broadcast 127.255.255.255 dev lo table local proto kernel scope link
src 127.0.0.1
broadcast 192.168.9.0 dev eth1 table local proto kernel scope link src 192.168.9.102
broadcast 172.16.0.0 dev eth0 table local proto kernel scope link src 172.16.0.2
local 192.168.9.102 dev eth1 table local proto kernel scope host src 192.168.9.102
local 172.16.0.2 dev eth0 table local proto kernel scope host src 172.16.0.2
broadcast 192.168.9.255 dev eth1 table local proto kernel scope link
src 192.168.9.102
broadcast 172.16.0.255 dev eth0 table local proto kernel scope link src 172.16.0.2
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
ip ru
-----
0: from all lookup local 5: from all lookup main 10: from all iif eth0 lookup 10 11: from 172.16.0.0/24 lookup 10 20: from all iif eth1 lookup 20 21: from 192.168.9.0/24 lookup 20 100: from all lookup default
no netfilter rules of any sort (all policies set at ACCEPT)
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
No takers on this question? I investigated further and it seems that this is a specific problem with iputils-ping. It seems that regardless of the supplied interface name, the source IP is chosen to be closest t the default gateway. OTOH my ability to follow C code is next to minimal, and I would really appreciate if someone can confirm this. Here is the part I believe is relevant:
if (source.sin_addr.s_addr = 0) {
socklen_t alen;
struct sockaddr_in dst = whereto;
int probe_fd = socket(AF_INET, SOCK_DGRAM, 0);
if (probe_fd < 0) {
perror("socket");
exit(2);
}
if (device) {
struct ifreq ifr;
memset(&ifr, 0, sizeof(ifr));
strncpy(ifr.ifr_name, device, IFNAMSIZ-1);
if (setsockopt(probe_fd, SOL_SOCKET, SO_BINDTODEVICE, device,
strlen(device)+1) = -1) {
if (IN_MULTICAST(ntohl(dst.sin_addr.s_addr))) {
struct ip_mreqn imr;
if (ioctl(probe_fd, SIOCGIFINDEX, &ifr) < 0) {
fprintf(stderr, "ping: unknown iface %s\n", device);
exit(2);
}
memset(&imr, 0, sizeof(imr));
imr.imr_ifindex = ifr.ifr_ifindex;
if (setsockopt(probe_fd, SOL_IP, IP_MULTICAST_IF, &imr, sizeof(imr))
= -1) {
perror("ping: IP_MULTICAST_IF");
exit(2);
}
}
}
}
if (settos &&
setsockopt(probe_fd, IPPROTO_IP, IP_TOS, (char *)&settos, sizeof(int))
< 0)
perror("Warning: error setting QOS sockopts");
dst.sin_port = htons(1025);
if (nroute)
dst.sin_addr.s_addr = route[0];
if (connect(probe_fd, (struct sockaddr*)&dst, sizeof(dst)) = -1) {
if (errno = EACCES) {
if (broadcast_pings = 0) {
fprintf(stderr, "Do you want to ping broadcast? Then -b\n");
exit(2);
}
fprintf(stderr, "WARNING: pinging broadcast address\n");
if (setsockopt(probe_fd, SOL_SOCKET, SO_BROADCAST,
&broadcast_pings, sizeof(broadcast_pings)) < 0) {
perror ("can't set broadcasting");
exit(2);
}
if (connect(probe_fd, (struct sockaddr*)&dst, sizeof(dst)) = -1) {
perror("connect");
exit(2);
}
} else {
perror("connect");
exit(2);
}
}
alen = sizeof(source);
if (getsockname(probe_fd, (struct sockaddr*)&source, &alen) = -1) {
perror("getsockname");
exit(2);
}
source.sin_port = 0;
close(probe_fd);
} while (0);Quoted 106 lines · Peter Rabbitson
Hi,
I have a testing multihome setup, with the default gateway being one of the links and using policy routing to honor requests for a specific link. Everything works as expected when I request a specific IP to bind to. But if I request a specific interface things fall apart in ways that I can not explain:
default gw (WORKS)
----------rabbit@Thesaurus:~$ ping -c 1 yahoo.com PING yahoo.com (216.109.112.135) 56(84) bytes of data. 64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 ttlH time2 msrequest IP on same link as default gw (WORKS)
-------------------------------------rabbit@Thesaurus:~$ ping -I 192.168.9.102 -c 1 yahoo.com PING yahoo.com (66.94.234.13) from 192.168.9.102 : 56(84) bytes of data. 64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1 ttlG time6 msrequest IP on secondary link (WORKS)
----------------------------rabbit@Thesaurus:~$ ping -I 172.16.0.2 -c 1 yahoo.com PING yahoo.com (216.109.112.135) from 172.16.0.2 : 56(84) bytes of data. 64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 ttlG time6 msrequest interface of default gw link (WORKS)
------------------------------------rabbit@Thesaurus:~$ ping -I eth1 -c 1 yahoo.com PING yahoo.com (66.94.234.13) from 192.168.9.102 eth1: 56(84) bytes of data. 64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1 ttlG time6 msrequest secondary interface (FAILS)
---------------------------rabbit@Thesaurus:~$ ping -I eth0 -c 1 yahoo.com PING yahoo.com (216.109.112.135) from 192.168.9.102 eth0: 56(84) bytes of data. From 172.16.0.2 icmp_seq=1 Destination Host UnreachableI went over the setup again and again, but I can't figure out why the last ping attempt fails. Any pointers are welcome!
Thanks
Peter
Here is the setup:
ip addr
----------
2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdiscpfifo_fastqlen 1000link/ether 00:11:09:8d:4f:c1 brd ff:ff:ff:ff:ff:ff inet 172.16.0.2/24 brd 172.16.0.255 scope global eth0 3: eth1: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:04:e2:80:b4:97 brd ff:ff:ff:ff:ff:ff inet 192.168.9.102/24 brd 192.168.9.255 scope global eth1ip ro show table all
-----------------------
default via 172.16.0.1 dev eth0 table 10
default via 192.168.9.1 dev eth1 table 20
default via 192.168.9.1 dev eth1 table default 172.16.0.0/24 dev eth0 proto kernel scope link src 172.16.0.2 192.168.9.0/24 dev eth1 proto kernel scope link src 192.168.9.102 broadcast 127.255.255.255 dev lo table local proto kernel scope link
src 127.0.0.1
broadcast 192.168.9.0 dev eth1 table local proto kernel scope link src 192.168.9.102
broadcast 172.16.0.0 dev eth0 table local proto kernel scope link src 172.16.0.2
local 192.168.9.102 dev eth1 table local proto kernel scope host src 192.168.9.102
local 172.16.0.2 dev eth0 table local proto kernel scope host src 172.16.0.2
broadcast 192.168.9.255 dev eth1 table local proto kernel scope link
src 192.168.9.102
broadcast 172.16.0.255 dev eth0 table local proto kernel scope link src 172.16.0.2
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1ip ru
-----0: from all lookup local 5: from all lookup main 10: from all iif eth0 lookup 10 11: from 172.16.0.0/24 lookup 10 20: from all iif eth1 lookup 20 21: from 192.168.9.0/24 lookup 20 100: from all lookup default
Quoted 6 lines
no netfilter rules of any sort (all policies set at ACCEPT)
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Older versions of ping does not support interface with I option. It won't give error, but it simply won't work. I had such an issue and was solved with the latest ping tool.
Have you tried using tcpdump to capture packets from interfaces?
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Peter Rabbitson
Sent: Wednesday, August 22, 2007 4:09 PM
To: [email protected]
Subject: Re: [LARTC] Policy routing question
No takers on this question? I investigated further and it seems that this is a specific problem with iputils-ping. It seems that regardless of the supplied interface name, the source IP is chosen to be closest t the default gateway. OTOH my ability to follow C code is next to minimal, and I would really appreciate if someone can confirm this. Here
is the part I believe is relevant:
if (source.sin_addr.s_addr = 0) {
socklen_t alen;
struct sockaddr_in dst = whereto;
int probe_fd = socket(AF_INET, SOCK_DGRAM, 0);
if (probe_fd < 0) {
perror("socket");
exit(2);
}
if (device) {
struct ifreq ifr;
memset(&ifr, 0, sizeof(ifr));
strncpy(ifr.ifr_name, device, IFNAMSIZ-1);
if (setsockopt(probe_fd, SOL_SOCKET, SO_BINDTODEVICE,
device,
strlen(device)+1) = -1) {
if (IN_MULTICAST(ntohl(dst.sin_addr.s_addr))) {
struct ip_mreqn imr;
if (ioctl(probe_fd, SIOCGIFINDEX, &ifr)
< 0) {
fprintf(stderr, "ping: unknown
iface %s\n", device);
exit(2);
}
memset(&imr, 0, sizeof(imr));
imr.imr_ifindex = ifr.ifr_ifindex;
if (setsockopt(probe_fd, SOL_IP,
IP_MULTICAST_IF, &imr, sizeof(imr))
= -1) {
perror("ping: IP_MULTICAST_IF");
exit(2);
}
}
}
}
if (settos &&
setsockopt(probe_fd, IPPROTO_IP, IP_TOS, (char
*)&settos, sizeof(int))
< 0)
perror("Warning: error setting QOS sockopts");
dst.sin_port = htons(1025);
if (nroute)
dst.sin_addr.s_addr = route[0];
if (connect(probe_fd, (struct sockaddr*)&dst, sizeof(dst)) =
-1) {
if (errno = EACCES) {
if (broadcast_pings = 0) {
fprintf(stderr, "Do you want to ping
broadcast? Then -b\n");
exit(2);
}
fprintf(stderr, "WARNING: pinging broadcastaddress\n");
if (setsockopt(probe_fd, SOL_SOCKET,SO_BROADCAST,
&broadcast_pings,
sizeof(broadcast_pings)) < 0) {
perror ("can't set broadcasting");
exit(2);
}
if (connect(probe_fd, (struct sockaddr*)&dst,
sizeof(dst)) = -1) {
perror("connect");
exit(2);
}
} else {
perror("connect");
exit(2);
}
}
alen = sizeof(source);
if (getsockname(probe_fd, (struct sockaddr*)&source, &alen) =
-1) {
perror("getsockname");
exit(2);
}
source.sin_port = 0;
close(probe_fd);
} while (0);Quoted 3 lines · Peter Rabbitson
Hi,
I have a testing multihome setup, with the default gateway being one
of
Quoted 2 lines
the links and using policy routing to honor requests for a specific link. Everything works as expected when I request a specific IP to
bind
Quoted 1 line
to. But if I request a specific interface things fall apart in ways
that
Quoted 10 lines
I can not explain:
default gw (WORKS)
----------rabbit@Thesaurus:~$ ping -c 1 yahoo.com PING yahoo.com (216.109.112.135) 56(84) bytes of data. 64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 ttlH time2 ms
Quoted 4 lines
request IP on same link as default gw (WORKS)
-------------------------------------rabbit@Thesaurus:~$ ping -I 192.168.9.102 -c 1 yahoo.com PING yahoo.com (66.94.234.13) from 192.168.9.102 : 56(84) bytes of
data.
Quoted 1 line
64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1
ttlG
Quoted 6 lines
time6 ms
request IP on secondary link (WORKS)
----------------------------rabbit@Thesaurus:~$ ping -I 172.16.0.2 -c 1 yahoo.com PING yahoo.com (216.109.112.135) from 172.16.0.2 : 56(84) bytes of
data.
Quoted 10 lines
64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 ttlG time6 ms
request interface of default gw link (WORKS)
------------------------------------rabbit@Thesaurus:~$ ping -I eth1 -c 1 yahoo.com PING yahoo.com (66.94.234.13) from 192.168.9.102 eth1: 56(84) bytes ofdata.
64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1
ttlG
Quoted 16 lines
time6 ms
request secondary interface (FAILS)
---------------------------rabbit@Thesaurus:~$ ping -I eth0 -c 1 yahoo.com PING yahoo.com (216.109.112.135) from 192.168.9.102 eth0: 56(84) bytesof data.
From 172.16.0.2 icmp_seq=1 Destination Host UnreachableI went over the setup again and again, but I can't figure out why the last ping attempt fails. Any pointers are welcome!
Thanks
Quoted 8 lines
Peter
Here is the setup:
ip addr
----------
2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdiscpfifo_fastqlen
1000
Quoted 3 lines
link/ether 00:11:09:8d:4f:c1 brd ff:ff:ff:ff:ff:ff inet 172.16.0.2/24 brd 172.16.0.255 scope global eth03: eth1: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc
pfifo_fastqlen
1000
Quoted 4 lines
link/ether 00:04:e2:80:b4:97 brd ff:ff:ff:ff:ff:ff
inet 192.168.9.102/24 brd 192.168.9.255 scope global eth1Quoted 8 lines
ip ro show table all
-----------------------
default via 172.16.0.1 dev eth0 table 10
default via 192.168.9.1 dev eth1 table 20
default via 192.168.9.1 dev eth1 table default 172.16.0.0/24 dev eth0 proto kernel scope link src 172.16.0.2 192.168.9.0/24 dev eth1 proto kernel scope link src 192.168.9.102 broadcast 127.255.255.255 dev lo table local proto kernel scope
link
Quoted 4 lines
src 127.0.0.1
broadcast 192.168.9.0 dev eth1 table local proto kernel scope link src 192.168.9.102
broadcast 172.16.0.0 dev eth0 table local proto kernel scope link
src
Quoted 2 lines
172.16.0.2
local 192.168.9.102 dev eth1 table local proto kernel scope host
src
Quoted 4 lines
192.168.9.102
local 172.16.0.2 dev eth0 table local proto kernel scope host src 172.16.0.2
broadcast 192.168.9.255 dev eth1 table local proto kernel scope
link
Quoted 24 lines
src 192.168.9.102
broadcast 172.16.0.255 dev eth0 table local proto kernel scope linksrc 172.16.0.2
broadcast 127.0.0.0 dev lo table local proto kernel scope link src127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1ip ru
-----0: from all lookup local 5: from all lookup main 10: from all iif eth0 lookup 10 11: from 172.16.0.0/24 lookup 10 20: from all iif eth1 lookup 20 21: from 192.168.9.0/24 lookup 20 100: from all lookup default
Quoted 6 lines
no netfilter rules of any sort (all policies set at ACCEPT)
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
[email protected]
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc