255.255.255.255 won't broadcast to multiple NICs

Yeah, that maps with what I've seen.

Apparently, the stack is stricly "one packet in, one packet out", and only concerns itself with where to send that packet. So when a packet can go out more than one interface (as in 255.255.255.255), the broadcast nature of it doesn't actually cause the packet to fork in the stack, it's treated like a load balancing situation instead where one interface is selected and the one packet is delivered. (The multiple delivery aspect is left solely to the ethernet layer. Despite being a broadcast packet, the IP layer doesn't replicate the broadcast on an interface level.)

I personally did not expect this behavior, I expected the packet to go out all the interfaces. And I still think the behavior I expected makes more sense. However, if the behavior that's in the kernel now is documented and what IP stacks the world over are expected to do, I can certainly live with it. :)

It's entirely possible that the other platforms I worked on before never had more than one NIC in a box I actually used. The fact I wasn't suprised by the behavior I saw elsewhere could simply be because I didn't try stuff on a box where the behavior would be suprising.

That said, I still think 255.255.255.255 "should" go out all interfaces (since gateways don't forward broadcast packets anyway, it's not going to flood the network or anything). I suspect the "it works, leave it alone" principle will apply here, but it'd be nice to attract the attention of a networking guru like Alan Cox, David S. Miller, or Donald Becker for a few seconds to at least get shot down on this issue decisively. :)

Unfortunately, I literally can't do that in this instance. (Yes, I've tried.) The client and server I'm writing are in Java. There's two problems here.

1) It's for a heterogeneous render farm. Linux boxes, NT/2000 boxes, and SGI boxes at least. Maybe someday macintoshes too, and who knows what else. (That's why I'm writing it in Java in the first place.)

2) I'm trying to have it configure itself as much as possible automatically. A lot of the people using render farms went to art college and liked it. That's why the broadcast packets (so the clients can find their server and vice versa when everybody boots up, without being told). I don't want them to have to tell it anything, I want them to throw it on the machine and run it (quite possibly running all the nodes from a shared mount) and just have it work. I've managed to avoid any node specific configuration so far, and that makes things MUCH easier for my expected user base.

3) Java sucks in many ways. Today's way is that it never occurred to Sun that a machine might have more than one IP address assigned to it, so InetAddress.getLocalHost() returns exactly one address. Unfortunately, just about EVERY machine has two interfaces defined, the other one being loopback on 127.0.0.1, and natrually the loopback is the one that getLocalHost() returns. (Since it's the one that we pretty much already know the address of anyway, and querying it is therefore useless, that's the one it queries. Thank you Sun.) There is no way to query the current machine's interfaces without resorting to native code.

Bind to a socket to a local port and query that address you say? Nope, too easy. The address returned when I query a socket (rather than a connection) is 0.0.0.0 on any machine with multiple interfaces (even loopback), since the socket is bound to that port on ALL the interfaces. Each incoming or outgoing connection does have a valid "from" IP address, but I have to wait for a connection to come in to get that. (Unless I explicitly specify which IP to bind to when I create the socket, but if I knew that I'd already be there.)

Nope, making my own connection to a port on the same machine just means 127.0.0.1 is talking to 127.0.0.1. Tried it. Didn't work.

Nope, feeding the loopback address to getAllByName() doesn't help either. I tried that too, it just returns a length 1 array containing just the loopback address.

Now you know why I'm resorting to 255.255.255.255. I'm sort of faking things: when the server broadcasts to clients they know who it is, and when they broadcast to it, it knows who THEY are (it says in the UDP datagram header info). And the way I've written it, that's all they really need to know (although when we reply to each other we can each find out the info we don't know: who WE are. But by that point, we no longer need it. :)

I may just document "if you run this on a machine with more than one network card, you have to specify the broadcast addresses on the command line". It's configuration, but the only machine likely to HAVE multiple interfaces is the server (which could be serving multiple subnets in a really BIG render farm), so I suppose it's tolerable...

I'm only passingly familiar with it, does it map well to this problem?

The only data I'm trying to transmit is "where is everybody", or "wake up". The broadcast packets are only needed for clients to find the server on bootup (and vice versa if the server is rebooted). They're also used to wake up clients if they go to sleep because the server has nothing for them to do at the moment, but that second part's a convenience, really. The server could loop through and address them individually instead since it knows where they are by that point.

The actual heavy lifting of data is done by TCP/IP streams. UDP broadcast is just for figuring out where to open the TCP/IP connections to.

Rob

__________________________________________________
Do You Yahoo!?

http://experts.yahoo.com/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

I faced this problem a while ago - in the end I cheated and put this bit of code in a shell script used to start the application

    NET_ADDRESSES=`/sbin/ifconfig   |                         \
                   awk '/^[^   ].*HWaddr/ { HWaddr=$5; next;}
                        /^[^        ]/ { HWaddr=0; }
                        /^[         ].*addr:/ { if (HWaddr != 0) { printf("-Dethaddr.%s=%s -Dnetmask.%s=%s ",substr($2,6),HWaddr,substr($2,6),substr($4,6)); } } END {print;}'`

including ${NET_ADDRESSES} in the java command line sets up a set of defines, one per interface. For example

-Dethaddr.172.16.1.1=00:00:0A:BC:CD:78 -Dnetmask.172.16.1.1=255.255.0.0

which you can use via System.getProperty() and System.getProperties()

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

The source IP address (as returned by getsockname()) is only set when the socket is connected... It follows the same logic: for a multihomed machine, we know which interface will be used only when we know who we'll be talking to...

You could use SIOGIFCONF (from C) to get the address list. I'm not sure is java has the equivalent... Or maybe a very small native method...

Sounds like a good job for multicast... It's fairly simple to use, but:

1) I'm not sure if java gives you access to the required ioctls (there's only five of them).

2) You may need to run mrouted on all your gateways.

Phil.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/