multicast: bug or "feature"

We've been trying to field some questions regarding multicast behavior and one such behavior has stumped us.

I've reproduced the following behavior on 2.6.23.

The application opens 2 sockets. One socket is the receiver and it simply binds to 0.0.0.0:2000 and joins a multicast group on interface eth0 (for the test we used 224.0.1.3). The other socket is the sender. It turns off MULTICAST_LOOP, sets MULTICAST_IF to eth1, and sends a packet to the group that the first socket joined.

We are expecting to receive the data on the receiver socket, but nothing comes back.

Running tcpdump on both interfaces during the test, I see the packet on both interfaces, ie. I see it sent on eth0 and received on eth1 with IP statistics going up appropriately.

Looking at the group memberships, I see the receiving interface as part of the group and IGMP messages were on the wire.

So, before I try to spend time figuring out where the packet went is why, I'd like to know if this is a Linux "feature".

Thanks
-vlad

I'm not clear on your configuration.

Are the sender and receiver running on the same machine? Are you saying eth0 and eth1 are connected on the same link?

                                        +-DLS

Yes and Yes.

I know it's a strange config, but it works with IPv6.

Here is the info off the reproducing system:

# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:17:08:7D:47:18  
          inet addr:10.202.1.23  Bcast:10.202.255.255  Mask:255.255.0.0
          inet6 addr: 2001:1890:1109:a10:217:8ff:fe7d:4718/64 Scope:Global
          inet6 addr: fe80::217:8ff:fe7d:4718/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1094 errors:0 dropped:0 overruns:0 frame:0
          TX packets:669 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:107776 (105.2 KiB)  TX bytes:109874 (107.2 KiB)
          Base address:0x4000 Memory:f9de0000-f9e00000 

eth1 Link encap:Ethernet HWaddr 00:18:FE:7F:49:C8

          inet addr:10.202.1.26  Bcast:10.202.255.255  Mask:255.255.0.0
          inet6 addr: 2001:1890:1109:a10:218:feff:fe7f:49c8/64 Scope:Global
          inet6 addr: fe80::218:feff:fe7f:49c8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:479 errors:0 dropped:0 overruns:0 frame:0
          TX packets:35 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:52236 (51.0 KiB)  TX bytes:4467 (4.3 KiB)
          Interrupt:41 Memory:f6000000-f6012100 

# ip r l
10.202.0.0/16 dev eth0  proto kernel  scope link  src 10.202.1.23 
10.202.0.0/16 dev eth1  proto kernel  scope link  src 10.202.1.26 
default via 10.202.1.1 dev eth0

-vlad

Can you send the contents of /proc/net/igmp and the packet trace, also? And the code?

                                        +-DLS

# cat /proc/net/igmp
Idx     Device    : Count Querier       Group    Users Timer    Reporter
1       lo        :     0      V3
                                010000E0     1 0:00000000               0
2       eth0      :     3      V2
                                010000E0     1 0:00000000               0
3       eth1      :     4      V2
                                030100E0     1 0:00000000               1
                                010000E0     1 0:00000000               0

Source attached. The trace only shows a single udp packet and you can re-create it with the attached small apps.

-vlad

#include <stdio.h>
#include <string.h>
#include <netinet/in.h>

int main(int argc, char **argv)

{
	struct sockaddr_storage dest;
	struct sockaddr_storage src;
	struct sockaddr_in *s = (struct sockaddr_in*) &src;
	struct sockaddr_in *d = (struct sockaddr_in *)&dest;
	int sock;
	char msg[] = "Hello Multicast";
	int off = 0;

	memset(&dest, 0, sizeof(dest));
	memset(&src, 0, sizeof(src));

	if (argc < 3) {
		printf("Usage: <src ip>  <mcast dest>\n");
		return 1;
	}


	d->sin_family = s->sin_family = AF_INET;
	d->sin_port = htons(2000);

	inet_aton(argv[1], &s->sin_addr);
	inet_aton(argv[2], &d->sin_addr);

	sock = socket(AF_INET, SOCK_DGRAM, 0);

	if (setsockopt(sock, IPPROTO_IP, IP_MULTICAST_LOOP, &off,
							sizeof(off))) {
		perror("setsockopt");
		return 1;
	}
	if (setsockopt(sock, IPPROTO_IP, IP_MULTICAST_IF, s, sizeof(*s))) {
		perror("setsockopt");
		return 1;
	}

	sendto(sock, msg, sizeof(msg), 0, (struct sockaddr *)d, sizeof(*d));

	close (socket);
	return 0;
}

#include <stdio.h>
#include <string.h>
#include <netinet/in.h>

int main(int argc, char **argv)

{
	struct sockaddr_in addr;
	int addr_len;
	struct ip_mreq req;
	int sock;
	char msg[256];

	memset(&addr, 0, sizeof(addr));

	if (argc < 3) {
		printf("Usage: <interface ip> <mcast group>\n");
		return 1;
	}

	sock = socket(AF_INET, SOCK_DGRAM, 0);
	addr.sin_family = AF_INET;
	addr.sin_port = htons(2000);
	if (bind(sock, (struct sockaddr *)&addr, sizeof(addr))) {
		perror("bind");
		return 1;
	}

	inet_aton(argv[1], &req.imr_interface);
	inet_aton(argv[2], &req.imr_multiaddr);

	if (setsockopt(sock, IPPROTO_IP, IP_ADD_MEMBERSHIP, &req,
							    sizeof(req))) {
		perror("setsockopt");
		return 1;
	}

	recvfrom(sock, msg, sizeof(msg), 0, (struct sockaddr *)&addr, &addr_len);

	printf("Message recieved: %s", msg);

	close (socket);
	return 0;
}

You're joining the group on interface eth1, which is the sender, right? You need to be a member on eth0 to receive it there. I think your program needs another argument, to specify the receiving interface, which you want to be different from the sending interface.

                                        +-DLS

I may have switched the ordering in the last test I ran, but I always join the group on the interface different from the one I send on.

-vlad

Yes, that was just the typing switch. The send packet was on eth1 (the checksum wasn't finished yet), and the received packet was on eth0.

Well, from what I've see, MULTICAST_LOOP is checked only on output, so that doesn't seem to be the issue.

Also, IPv6 works as expected, i.e. I can receive the data in the app above if I use IPv6 multicast addresses.

-vlad

Ok, so I've traced the failure down to fib_validate_source().

Because the packet we received was sourced from one of our own addresses, we end up finding a RTN_LOCAL route and fail out of that function with -EINVAL.

I can see the reason for this behavior and I think dropping in this case is fine.

Now, to figure out what IPv6 does different and why it works. Seems to me that the two should have the same behavior.

-vlad

IPv6 on Linux uses a per-interface addressing model as opposed to the per-host model used by IPv4.

Cheers,

Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[email protected] wrote on 10/19/2007 04:43:27 AM:

For link-local addresses, yes.

It's really a security feature; the ordinary
case where you'd receive something on an interface that's using one of your source addresses is when someone is spoofing you, has a duplicate address, or maybe an (unintentional) routing loop. All of those are error cases, so dropping a received packet that claims to be sent by you is a reasonable thing to do.
If you're getting link-local source addresses for your
IPv6 multicast packets, that may explain it. The link-local addresses are required to be unique and valid only for that link, so IPv6 should not consider a different interface's link-local address as "local" for a destination address, or a packet using that source address as bogus.
For a global address, v4 and v6 use the same rules--
for a destination you can receive it on any interface for any global address. So, if your source address was a global IPv6 address and it worked, I'd guess IPv6 just isn't checking the source address. I don't know that it's required by RFC for either v4 or v6, though it's probably a good idea.

                                                        +-DLS

I can see this as a good feature for unicast, but starting to doubt it just a little bit for multicast. With IGMPv3/MLDv2 and source filtering, this could be done as a filter on individual sockets.

The problem them becomes IGMPv2 and MLDv1.

Looks like the only time IPv6 does any type of source filtering is when CONFIG_IPV6_MULTIPLE_TABLES is turned on.

I need to turn this on and see if I get the same results or not.

I've reproduced the multicast traffic using both global and link-local addresses (both source and destination were of the same scope in the tests, i.e. either global or link-local).

So, it appears that IPv6 didn't do any source verification for multicast traffic.

-vlad

I don't know why you'd want it to be different for multicasting. If you want to hear your own multicasts, you should use MULTICAST_LOOP; hearing them off the wire indicates all the same bad things -- a forger, a duplicate address or a routing loop. Those aren't any better for multicasting than they are for unicasting, that I can see.

Why would you want that to be delivered, other than (apparently) to test the multicast capability of a pair of interfaces while using only 1 machine?

                                        +-DLS

I don't really, but the customer is complaining. The customer is transitioning from BSD to Linux, but wishes to support both. The behaviors are different. Client applications that assume certain
behavior on BSD, do not work on Linux.

It's just a pain. I was more wondering why v4 and v6 did things different in this case.

-vlad

From looking at the code, it appears that validate
source is failing just because of the rp_filter. Do you have rp_filter set to nonzero?
If so, it may do what you want just by setting that
to 0:

sysctl -w net.ipv4.conf.all.rp_filter=0

                                        +-DLS

Hi David,

rp_filter is set to zero, it's the "if (res.type != RTN_UNICAST)" check in fib_validate_source() that's doing it. If I add a new "accept_local_addr" sysctl to ipv4_devconf to allow RTN_LOCAL here, everything works just fine. I just don't know how palatable that would be to upstream...

-Brian

I don't know about a new knob, but it's the same notion as rp_filter, so why not use rpf for RTN_LOCAL types? Ie, allow RTN_LOCAL and RTN_UNICAST at the top, but check rpf if the devs aren't equal or RTN_LOCAL....

It seems like not a good thing to rely on in the first place, though; usually receiving things from yourself is a bad thing. :-)

                                        +-DLS