Skip to contentSkip to conversation

Resend: SIOCGIFBRDADDR?

DISCUSSION Alan Robertson 4 participants 8 messages Aug 15, 2002 – Aug 19, 2002

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I sent this earlier, but I got no reply, and can't seem to find it in the archives. If you reply to this, could you please copy both Kevin Dwyer and myself on the reply? Many thanks!

Hi,

Some folks using the linux-ha (heartbeat) software have been running into some network-related problems, and Alan Cox suggested that you folks could help us out.

One of the ways you can configure linux-ha protocols is to send broadcast packets. In this case, our software needs to get the broadcast address from the OS, so we can proceed. We currently use the SIOCGIFBRDADDR ioctl to do this for us. This has worked pretty well for us for quite some time, but recently some users have run into some problems.

It appears that if one configures an interface using ifconfig, then the SIOCGIFBRDADDR ioctl works quite nicely to determine the broadcast address of an interface.

However, if one uses the iproute2 tools to configure the interface, then it appears that the SIOCGIFBRDADDR ioctl returns bogus results on that interface.

Is this supposed to be the case?

Is there a different method we're supposed to use instead of SIOCGIFBRDADDR?

	Thanks!

	-- Alan Robertson
	   [email protected]
	   http://linux-ha.org/
#1 lore raw

Hello!

Quoted 1 line

Is this supposed to be the case?

You forgot to say what kind of "bogus result" is obtained.

Alexey

#2 lore raw

On Thu, 15 Aug 2002, [email protected] grunted something like:

Quoted 1 line

You forgot to say what kind of "bogus result" is obtained.

Since I'm the one who has primarily been chasing this problem, I'll explain.

(cut from my original email...)

Specifically, it uses iproute to add aliases to a device, like eth0 in this example:

ip -f inet addr add 10.5.5.1 dev eth0 scope link
ip -f inet addr add 10.5.5.2 dev eth0 scope link
ip -f inet addr add 10.5.5.3 dev eth0 scope link
ip -f inet addr add 10.5.5.4 dev eth0 scope link
ip -f inet addr add 10.5.5.5 dev eth0 scope link

That's all fine, except that it causes really weird stuff to happen to the output of ifconfig. For one, the inet addr: field will change once in a while if I make a modification with ifconfig. That doesn't affect heartbeat, but the Bcast: field is wrongly reported as 0.0.0.0. Now, I know for a fact that I set it correctly when I bring up the interface, prior to using the ip route commands. Once they've been applied, it goes crazy. So my question: Does anyone know if there's an ip route command to make the Bcast field show up as the correct value, or is there a better way for heartbeat to get the broadcast address?

(and from a followup email...)

# ip addr show eth0 primary scope global

2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100

    link/ether 00:c0:95:e1:bc:64 brd ff:ff:ff:ff:ff:ff
    inet 10.5.5.8/24 brd 10.5.5.255 scope global eth0

That command gives me the correct broadcast address in the brd field.

Correct meaning, the one we're looking for, and would otherwise have gotten if I hadn't used the before-mentioned iproute commands.

Any help is appreciated.

Thanks.

-[ [email protected] devel.pheared.net ]-
-[ Rather be forgotten, than remembered for giving in. ]-
-[ ZZ = g ^ (xb * xa) mod p g = h^{(p-1)/q} mod p ]-

#3 lore raw

Hello!

Quoted 2 lines

crazy. So my question: Does anyone know if there's an ip route command to make the Bcast field show up as the correct value,

Yes, that one which is described in manual (and equivalent to corresponding ifconfig)

ifconfig eth0:tra-ta-ta 10.x.y.z etc.

Quoted 2 lines

or is there a better
way for heartbeat to get the broadcast address?

Well, yes. Initialize ifreq with right address, for instance. Probably, your program has the same problem, which is in net-tools package.

Or this:

Quoted 1 line

That command gives me the correct broadcast address in the brd field.

Alexey

#4 lore raw

On Thu, 15 Aug 2002, [email protected] grunted something like:

Thanks for the response...

Quoted 7 lines
Quoted 2 lines

crazy. So my question: Does anyone know if there's an ip route command to make the Bcast field show up as the correct value,

Yes, that one which is described in manual (and equivalent to corresponding ifconfig)

ifconfig eth0:tra-ta-ta 10.x.y.z etc.

Perhaps I should clarify a bit further.
When we are starting heartbeat, the interface is setup thusly:
(I've simplified this to make it easier to read)

ifconfig eth0 10.5.5.8 broadcast 10.5.5.255 netmask 255.255.255.0 ip -f inet addr add 10.5.5.1 dev eth0 scope link
ip -f inet addr add 10.5.5.2 dev eth0 scope link

After executing those commands, we see the following:

ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:C0:95:E1:BC:64
inet addr:10.5.5.1 Bcast:0.0.0.0 Mask:255.255.255.255 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
<snip>

(Note: the inet addr, Bcast, and Mask fields are all incorrectly reporting
ifconfig's initial settings, which is why heartbeat is also confused.)

2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100

    link/ether 00:c0:95:e1:bc:64 brd ff:ff:ff:ff:ff:ff
    inet 10.5.5.1/32 scope link eth0
    inet 10.5.5.2/32 scope link eth0
    inet 10.5.5.8/24 brd 10.5.5.255 scope global eth0

So we have eth0 setup as 10.5.5.8, and two aliases setup with iproute at 10.5.5.1 and 10.5.5.2. They are not however aliases like eth0:1, eth0:2, etc. But, iproute has setup the kernel to reply to ARPs for those IPs.

So, heartbeat needs to know which interface to use for sending broadcast packets to another node. I've defined eth0 as said interface, and it uses eth0 as the ifreq for the ioctl call. However, the ioctl call returns 0.0.0.0, when in fact I need 10.5.5.255.

Quoted 5 lines
Quoted 2 lines

or is there a better
way for heartbeat to get the broadcast address?

Well, yes. Initialize ifreq with right address, for instance. Probably, your program has the same problem, which is in net-tools package.

See above.. the problem is that using eth0 is the right address.

Quoted 3 lines

Or this:

Quoted 1 line

That command gives me the correct broadcast address in the brd field.

The only problem I see with emulating iproute's commands is that you'll need netlink (is this correct?) so it won't work for everyone until everyone starts adding that to their kernel config. (A possibly reasonable request, if this is indeed the best way to attack the problem. This is partially why we're asking for commentary/help.)

-[ [email protected] devel.pheared.net ]-
-[ Rather be forgotten, than remembered for giving in. ]-
-[ ZZ = g ^ (xb * xa) mod p g = h^{(p-1)/q} mod p ]-

#5 lore raw
Quoted 12 lines · Kevin Dwyer

On Thu, 15 Aug 2002, [email protected] grunted something like:

Thanks for the response...

Quoted 7 lines
Quoted 2 lines

crazy. So my question: Does anyone know if there's an ip route command to make the Bcast field show up as the correct value,

Yes, that one which is described in manual (and equivalent to corresponding ifconfig)

ifconfig eth0:tra-ta-ta 10.x.y.z etc.

Quoted 7 lines

Perhaps I should clarify a bit further.
When we are starting heartbeat, the interface is setup thusly:
(I've simplified this to make it easier to read)

ifconfig eth0 10.5.5.8 broadcast 10.5.5.255 netmask 255.255.255.0 ip -f inet addr add 10.5.5.1 dev eth0 scope link
ip -f inet addr add 10.5.5.2 dev eth0 scope link

Why do you use the 'scope link' statements?

And as far as your original question try the following:

ip -f inet addr add 10.5.5.8/24 dev eth0 brd +
ip -f inet addr add 10.5.5.1 dev eth0 scope link brd 10.5.5.255 ip -f inet addr add 10.5.5.2 dev eth0 scope link brd 10.5.5.255

And then you will have the output of 'ip ad li dev eth0' as:

3: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100

    link/ether 00:c0:9f:0c:5f:db brd ff:ff:ff:ff:ff:ff
    inet 10.5.5.1/32 brd 10.5.5.255 scope link eth0
    inet 10.5.5.2/32 brd 10.5.5.255 scope link eth0
    inet 10.5.5.8/24 brd 10.5.5.255 scope global eth0

And also you will then have 'ifconfig eth0':

eth0 Link encap:Ethernet HWaddr 00:C0:9F:0C:5F:DB
inet addr:10.5.5.2 Bcast:10.5.5.255 Mask:255.255.255.255 UP BROADCAST MULTICAST MTU:1500 Metric:1

Work for you?

[snip]

Quoted 3 lines

-[ [email protected] devel.pheared.net ]-
-[ Rather be forgotten, than remembered for giving in. ]-
-[ ZZ = g ^ (xb * xa) mod p g = h^{(p-1)/q} mod p ]-

And if I am way off base here due to jumping in just ignore me.

--------------------------------------------------
Matthew G. Marsh, President
Paktronix Systems LLC
1506 North 59th Street
Omaha NE 68104
Phone: (402) 932-7250 x101
Email: [email protected]
WWW: http://www.paktronix.com
--------------------------------------------------

#6 lore raw

On Sun, 18 Aug 2002, Matthew G. Marsh grunted something like:

Quoted 5 lines
Quoted 3 lines

ifconfig eth0 10.5.5.8 broadcast 10.5.5.255 netmask 255.255.255.0 ip -f inet addr add 10.5.5.1 dev eth0 scope link
ip -f inet addr add 10.5.5.2 dev eth0 scope link

Why do you use the 'scope link' statements?

Why do I use the 'scope link' argument, or the statement as a whole? Those two statements give the same effect as aliases (more or less) without having to ifconfig eth0:X for each X. My understanding was the 'scope link' part was useful in keeping the effects of the command local to the device. However, I also did not generate these commands on my own:
fwbuilder is doing that. We're trying to get a setup that can be easily managed by other people who may not have the same experience with command-line tools and such.

Quoted 5 lines

And as far as your original question try the following:

ip -f inet addr add 10.5.5.8/24 dev eth0 brd +
ip -f inet addr add 10.5.5.1 dev eth0 scope link brd 10.5.5.255 ip -f inet addr add 10.5.5.2 dev eth0 scope link brd 10.5.5.255

Aha! At the suggestion of the iproute2 documentation, and another person on the linux-ha list, I tried using 'brd +' on the aliased IPs, but apparently we were reading the docs wrong. Setting it explicitly works!

Quoted 7 lines

And also you will then have 'ifconfig eth0':

eth0 Link encap:Ethernet HWaddr 00:C0:9F:0C:5F:DB
inet addr:10.5.5.2 Bcast:10.5.5.255 Mask:255.255.255.255 UP BROADCAST MULTICAST MTU:1500 Metric:1

Work for you?

Exactly what we need. Now the ioctl can pull the broadcast address.

Quoted 1 line

And if I am way off base here due to jumping in just ignore me.

Not at all, thanks for the suggestion. It seems now that I need to try to encourage the fwbuilder folks to specify the broadcast address on each alias. At the very least, I should be able to whip out a patch to do it, and hopefully they'll accept it.

/* [email protected] http://devel.pheared.net/ */
/* Network Security Engineer http://pheared.net/~kevin */
/* Sabotage will set us free. Throw a rock in the machine. */

#7 lore raw
Quoted 11 lines · Kevin Dwyer

On Sun, 18 Aug 2002, Matthew G. Marsh grunted something like:

Quoted 1 line

Why do you use the 'scope link' statements?

Why do I use the 'scope link' argument, or the statement as a whole? Those two statements give the same effect as aliases (more or less) without having to ifconfig eth0:X for each X. My understanding was the 'scope link' part was useful in keeping the effects of the command local to the device. However, I also did not generate these commands on my own:
fwbuilder is doing that. We're trying to get a setup that can be easily managed by other people who may not have the same experience with command-line tools and such.

Ah OK. I was wondering because if you leave off the 'scope link' then you get inheritance of the broadcast for secondary ip addrs.

Quoted 9 lines
Quoted 5 lines

And as far as your original question try the following:

ip -f inet addr add 10.5.5.8/24 dev eth0 brd +
ip -f inet addr add 10.5.5.1 dev eth0 scope link brd 10.5.5.255 ip -f inet addr add 10.5.5.2 dev eth0 scope link brd 10.5.5.255

Aha! At the suggestion of the iproute2 documentation, and another person on the linux-ha list, I tried using 'brd +' on the aliased IPs, but apparently we were reading the docs wrong. Setting it explicitly works!

Yep - that is the 'scope link' override that basically tells the system to ignore any inheritance and treat the address as a special local case. Note that the order in which the addresses are applied is critical. For example if you do:

ip addr add 10.5.5.1/24 dev eth0 scope link brd +

Then you get the appropriate broadcast address for the /24 netmask. BUT then you cannot add in a global address in the same network as in:

ip addr add 10.5.5.8/24 dev eth0 scope global brd 10.5.5.255 RTNETLINK answers: Invalid argument

You have to change the netmask at this point (think scope~=mask). Essentially you cannot have a primary and secondary address on a single device exist with different scope. Think of IPv6 scope and it becomes a little clearer where the addressing conflicts.

Your best bet when in doubt is to always specify the actual broadcast address you require. The 'brd +' is merly a shortcut to specify the broadcast address associated with the specified CIDR mask. In fact you can have way interesting amounts of fun by specifying alternate broadcasts from CIDR masks as in:

ip addr add 10.5.5.1/24 dev eth0 brd 10.5.255.255

Using this on a "Class B" network allow you to "be seen" by other devices but only speak to/from devices within your CIDR scope (hint: look at the output of 'ip ro li tab local') as well as really fsck routers and ARP tables for the network... ;-}

[snip]

Quoted 10 lines
Quoted 1 line

And if I am way off base here due to jumping in just ignore me.

Not at all, thanks for the suggestion. It seems now that I need to try to encourage the fwbuilder folks to specify the broadcast address on each alias. At the very least, I should be able to whip out a patch to do it, and hopefully they'll accept it.

/* [email protected] http://devel.pheared.net/ */
/* Network Security Engineer http://pheared.net/~kevin */
/* Sabotage will set us free. Throw a rock in the machine. */

--------------------------------------------------
Matthew G. Marsh, President
Paktronix Systems LLC
1506 North 59th Street
Omaha NE 68104
Phone: (402) 932-7250 x101
Email: [email protected]
WWW: http://www.paktronix.com
--------------------------------------------------