bonding: move IPv6 support into a separate kernel module

This patch moves the IPv6 bonding code into a separate kernel module called bonding_ipv6 if either bonding or IPv6 are built as modules. If both are built into the kernel then this is as well. Bonding_ipv6.ko registers an "send_unsol_na" function pointer for the unsolicited advertisement function to be called on a failover - the default action is to do nothing. The notifier callbacks are now registered in this module and not in the base bonding module.

Also, have the IPv6 address notifier request that the bonding_ipv6 module be loaded when an IFF_MASTER device is first brought-up. This avoids users from having to do this explicitly with modprobe.

I think I can pretty much guarantee that distro users will be clamoring for a quick and easy way to block ipv6, and it's in our interest to document the recomended way to block it that doesn't cause weird problems with bonding, etc.

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded yet, although I haven't really checked).

From: Jay Vosburgh <[email protected]>
Date: Wed, 25 Feb 2009 14:10:58 -0800

Quoted 4 lines

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded yet, although I haven't really checked).

Correct, that's the problem.

We could create a blocker bitmap. Two sysctls, "block_af" and "unblock_af". You write the AF_foo value for the protocol there and it sets or clears the assosciated bit in the internal blocker bitmap.

Things like sys_socket() et al. key off of this.

Quoted 10 lines · David Miller

From: Jay Vosburgh <[email protected]>
Date: Wed, 25 Feb 2009 14:10:58 -0800

Quoted 4 lines

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded yet, although I haven't really checked).

Correct, that's the problem.

Quoted 5 lines

We could create a blocker bitmap. Two sysctls, "block_af" and "unblock_af". You write the AF_foo value for the protocol there and it sets or clears the assosciated bit in the internal blocker bitmap.

Things like sys_socket() et al. key off of this.

I'm open to suggestions at this point in time, I just don't see how this will solve the bonding problem since it still wouldn't load, right?

Dave - do you feel I need to fix this regression? If not I can try to work on this AF blocker thing. My only other thought if we want to fix this is to have the IPv6 module register these five functions into an ops structure that bonding can call. It doesn't fix SCTP, qeth, etc, but it gets these "blacklist ipv6" configs working again, and gets me out of the crosshairs :)

Quoted 10 lines · Brian Haley

Quoted 9 lines · David Miller

From: Jay Vosburgh <[email protected]>
Date: Wed, 25 Feb 2009 14:10:58 -0800

Quoted 4 lines

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded yet, although I haven't really checked).

Correct, that's the problem.

Quoted 7 lines

Quoted 5 lines

We could create a blocker bitmap. Two sysctls, "block_af" and "unblock_af". You write the AF_foo value for the protocol there and it sets or clears the assosciated bit in the internal blocker bitmap.

Things like sys_socket() et al. key off of this.

I'm open to suggestions at this point in time, I just don't see how this will solve the bonding problem since it still wouldn't load, right?

It would permit users to load ipv6 (thus allowing bonding to
load), but prevent ipv6 from actually doing anything. (because sys_socket, e.g., won't open an ipv6 socket if block_af includes ipv6).

Actually, __sock_create might be the better place to put the
hook for "create a socket"; there would probably need to be a check within the protocol code as well, so that, e.g., ipv6 addrconf won't run if AF_INET6 is disabled.

Quoted 7 lines

Dave - do you feel I need to fix this regression? If not I can try to work on this AF blocker thing. My only other thought if we want to fix this is to have the IPv6 module register these five functions into an ops structure that bonding can call. It doesn't fix SCTP, qeth, etc, but it gets these "blacklist ipv6" configs working again, and gets me out of the crosshairs :)

I think the problem (customers want to disable ipv6 and use
bonding, sctp, qeth, whatever) needs to be fixed. If it's not, I'm sure I'll be getting lots of cards and letters from customers.

I don't think the solution needs to preserve the current
solution (preventing the ipv6 module from loading). Ipv6 being unusable should be sufficient. Except perhaps in an embedded environment, but they're probably in a position to compile their kernel without ipv6.

Another possible resolution is to modify the initscripts in the
distros to perform sysctl -p (read sysctls from /etc/sysctl.conf) after ipv6 is loaded, so that the disable_ipv6 sysctl can be set. That seems like more work, and is limited to ipv6, so I don't see it as being better than a "kernel shut off AF_xxx" type of solution.

	-J

---
-Jay Vosburgh, IBM Linux Technology Center, [email protected]

Quoted 20 lines · Jay Vosburgh

Quoted 10 lines · Brian Haley

Quoted 9 lines · David Miller

From: Jay Vosburgh <[email protected]>
Date: Wed, 25 Feb 2009 14:10:58 -0800

Quoted 4 lines

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded yet, although I haven't really checked).

Correct, that's the problem.

Quoted 7 lines

Quoted 5 lines

We could create a blocker bitmap. Two sysctls, "block_af" and "unblock_af". You write the AF_foo value for the protocol there and it sets or clears the assosciated bit in the internal blocker bitmap.

Things like sys_socket() et al. key off of this.

I'm open to suggestions at this point in time, I just don't see how this will solve the bonding problem since it still wouldn't load, right?

Quoted 8 lines

It would permit users to load ipv6 (thus allowing bonding to
load), but prevent ipv6 from actually doing anything. (because sys_socket, e.g., won't open an ipv6 socket if block_af includes ipv6).

Actually, __sock_create might be the better place to put the
hook for "create a socket"; there would probably need to be a check within the protocol code as well, so that, e.g., ipv6 addrconf won't run if AF_INET6 is disabled.

But addrconf_init doesn't care about AF_INET6 sockets...

Additionally, why is it absolutely necessary to block AF_INET6 sockets. I never understood that requirement?

I can see people blocking IPv6 from loading because the module automatically configures IPv6 addresses and thus opens another communication channel that may not be monitored/controlled. AF_INET6 sockets, on the other hand, are simply relegated to IPv4 protocol, when there are no IPv6 addresses.

Quoted 7 lines

Quoted 6 lines

Dave - do you feel I need to fix this regression? If not I can try to work on this AF blocker thing. My only other thought if we want to fix this is to have the IPv6 module register these five functions into an ops structure that bonding can call. It doesn't fix SCTP, qeth, etc, but it gets these "blacklist ipv6" configs working again, and gets me out of the crosshairs :)

Quoted 8 lines

I think the problem (customers want to disable ipv6 and use
bonding, sctp, qeth, whatever) needs to be fixed. If it's not, I'm sure I'll be getting lots of cards and letters from customers.

I don't think the solution needs to preserve the current
solution (preventing the ipv6 module from loading). Ipv6 being unusable should be sufficient. Except perhaps in an embedded environment, but they're probably in a position to compile their kernel without ipv6.

Yes. The system must not be reachable using IPv6.

Quoted 6 lines

Another possible resolution is to modify the initscripts in the
distros to perform sysctl -p (read sysctls from /etc/sysctl.conf) after ipv6 is loaded, so that the disable_ipv6 sysctl can be set. That seems like more work, and is limited to ipv6, so I don't see it as being better than a "kernel shut off AF_xxx" type of solution.

This not enough. You need to disable parts of IPv6 at module initiation time and the only way to do that is with a parameter. Otherwise, you will have a small window of time when the system has ipv6 configured and is potentially vulnerable.

We can have our own sysfs parameter calls that can turn the functionality back on to get back to a fully functional ipv6 implementation.

-vlad

Quoted 10 lines

Quoted 10 lines

Quoted 4 lines

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded yet, although I haven't really checked).

Correct, that's the problem.

We could create a blocker bitmap. Two sysctls, "block_af" and "unblock_af". You write the AF_foo value for the protocol there and it sets or clears the assosciated bit in the internal blocker bitmap.

Quoted 3 lines

Quoted 1 line

Things like sys_socket() et al. key off of this.

I'm open to suggestions at this point in time, I just don't see how this will solve the bonding problem since it still wouldn't load, right?

It would permit users to load ipv6 (thus allowing bonding to
load), but prevent ipv6 from actually doing anything. (because sys_socket, e.g., won't open an ipv6 socket if block_af includes ipv6).

Quoted 13 lines · Jay Vosburgh

Quoted 13 lines

Quoted 11 lines

Quoted 4 lines

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded yet, although I haven't really checked).

Correct, that's the problem.

We could create a blocker bitmap. Two sysctls, "block_af" and "unblock_af". You write the AF_foo value for the protocol there and it sets or clears the assosciated bit in the internal blocker bitmap.

Things like sys_socket() et al. key off of this.

I'm open to suggestions at this point in time, I just don't see how this will solve the bonding problem since it still wouldn't load, right?

Quoted 3 lines

It would permit users to load ipv6 (thus allowing bonding to
load), but prevent ipv6 from actually doing anything. (because sys_socket, e.g., won't open an ipv6 socket if block_af includes ipv6).

Right, but it doesn't help someone that changed /etc/modprobe.conf to have "install ipv6 /bin/true" - they'll have to stop doing that.

I think changing ipv6 to support a disable_ipv6 module parameter like Vlad suggested would work, as long as we're not worried about someone opening an AF_INET6 socket - even if they do they won't get anywhere.

That, along with the patch below to actually not add the addresses, would work (sorry in advance for using an attachment). I'll get started on that...

-Brian

--

The disable_ipv6 knob was meant to be used for the kernel to disable IPv6 on an interface when DAD failed for the link-local address based on the MAC, but we should also be able to administratively disable it on an interface, or the entire system. This patch fixes the per-interface problem.

Signed-off-by Brian Haley <[email protected]>

net/ipv6/addrconf.c +5 -0

index f8f76d6..90f2a81 100644
@@ -603,6 +603,11 @@ ipv6_add_addr(struct inet6_dev *idev, const  
 struct in6_addr *addr, int pfxlen,
 		goto out2;
 	}
 
+	if (idev->cnf.disable_ipv6) {
+		err = -EPERM;
+		goto out2;
+	}
+
 	write_lock(&addrconf_hash_lock);
 

	/* Ignore adding duplicate addresses on an interface */

Quoted 24 lines · Brian Haley

Quoted 12 lines · Jay Vosburgh

Quoted 12 lines

Quoted 12 lines

Quoted 6 lines

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up
from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded
yet, although I haven't really checked).

Correct, that's the problem.

We could create a blocker bitmap. Two sysctls, "block_af" and "unblock_af". You write the AF_foo value for the protocol there and it sets or clears the assosciated bit in the internal blocker bitmap.

Quoted 7 lines

Quoted 4 lines

Quoted 1 line

Things like sys_socket() et al. key off of this.

I'm open to suggestions at this point in time, I just don't see how this
will solve the bonding problem since it still wouldn't load, right?

It would permit users to load ipv6 (thus allowing bonding to
load), but prevent ipv6 from actually doing anything. (because sys_socket, e.g., won't open an ipv6 socket if block_af includes ipv6).

Right, but it doesn't help someone that changed /etc/modprobe.conf to have "install ipv6 /bin/true" - they'll have to stop doing that.

Quoted 3 lines

I think changing ipv6 to support a disable_ipv6 module parameter like Vlad suggested would work, as long as we're not worried about someone opening an AF_INET6 socket - even if they do they won't get anywhere.

In this case, if IPV6ONLY is set on an AF_INET6 listener, it should still get AF_INET traffic, correct?

Quoted 34 lines

That, along with the patch below to actually not add the addresses, would work (sorry in advance for using an attachment). I'll get started on that...

-Brian

Signature · 7 lines

The disable_ipv6 knob was meant to be used for the kernel to disable
IPv6 on an interface when DAD failed for the link-local address based
on the MAC, but we should also be able to administratively disable it
on an interface, or the entire system.  This patch fixes the
per-interface problem.

Signed-off-by: Brian Haley <[email protected]>

net/ipv6/addrconf.c +5 -0

index f8f76d6..90f2a81 100644
@@ -603,6 +603,11 @@ ipv6_add_addr(struct inet6_dev *idev, const
 struct in6_addr *addr, int pfxlen,
        goto out2;
    }
 
+    if (idev->cnf.disable_ipv6) {
+        err = -EPERM;
+        goto out2;
+    }
+
    write_lock(&addrconf_hash_lock);
 

    /* Ignore adding duplicate addresses on an interface */

Quoted 1 line

I think changing ipv6 to support a disable_ipv6 module parameter like Vlad suggested would work, as long as we're not worried about someone opening an AF_INET6 socket - even if they do they won't get anywhere.

In this case, if IPV6ONLY is set on an AF_INET6 listener, it should still get AF_INET traffic, correct?

Quoted 2 lines · Chuck Lever

Quoted 1 line

I think changing ipv6 to support a disable_ipv6 module parameter like Vlad suggested would work, as long as we're not worried about someone opening an AF_INET6 socket - even if they do they won't get anywhere.

In this case, if IPV6ONLY is set on an AF_INET6 listener, it should still get AF_INET traffic, correct?

No, it should get nothing, and a send should get ENETUNREACH.

Quoted 5 lines · Brian Haley

Quoted 4 lines · Chuck Lever

Quoted 4 lines

I think changing ipv6 to support a disable_ipv6 module parameter like Vlad suggested would work, as long as we're not worried about someone opening an AF_INET6 socket - even if they do they won't get anywhere.

Quoted 4 lines

Quoted 2 lines

In this case, if IPV6ONLY is set on an AF_INET6 listener, it should still get AF_INET traffic, correct?

No, it should get nothing, and a send should get ENETUNREACH.

Sorry, I got my logic backwards. If IPV6ONLY is intentionally cleared on an AF_INET6 socket, it should still be able to handle AF_INET traffic.

Signature · 3 lines

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Quoted 11 lines · Jay Vosburgh

Quoted 11 lines

Quoted 11 lines

Quoted 5 lines

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded
yet, although I haven't really checked).

Correct, that's the problem.

We could create a blocker bitmap. Two sysctls, "block_af" and "unblock_af". You write the AF_foo value for the protocol there and it sets or clears the assosciated bit in the internal blocker bitmap.

Quoted 7 lines

Quoted 3 lines

Quoted 1 line

Things like sys_socket() et al. key off of this.

I'm open to suggestions at this point in time, I just don't see how this will solve the bonding problem since it still wouldn't load, right?

It would permit users to load ipv6 (thus allowing bonding to
load), but prevent ipv6 from actually doing anything. (because sys_socket, e.g., won't open an ipv6 socket if block_af includes ipv6).

Right, but it doesn't help someone that changed /etc/modprobe.conf to have "install ipv6 /bin/true" - they'll have to stop doing that.

I think changing ipv6 to support a disable_ipv6 module parameter like Vlad suggested would work, as long as we're not worried about someone opening an AF_INET6 socket - even if they do they won't get anywhere. That, along with the patch below to actually not add the addresses, would work (sorry in advance for using an attachment). I'll get started on that...

-Brian

Signature · 7 lines

The disable_ipv6 knob was meant to be used for the kernel to disable
IPv6 on an interface when DAD failed for the link-local address based on
the MAC, but we should also be able to administratively disable it on an
interface, or the entire system.  This patch fixes the per-interface
problem.

Signed-off-by: Brian Haley <[email protected]>

net/ipv6/addrconf.c +0 -0

index f8f76d6..90f2a81 100644

net/ipv6/addrconf.c +0 -0

 	/* Ignore adding duplicate addresses on an interface */

Quoted 14 lines · Jay Vosburgh

Quoted 13 lines

Quoted 11 lines

Quoted 4 lines

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded yet, although I haven't really checked).

Correct, that's the problem.

We could create a blocker bitmap. Two sysctls, "block_af" and "unblock_af". You write the AF_foo value for the protocol there and it sets or clears the assosciated bit in the internal blocker bitmap.

Things like sys_socket() et al. key off of this.

I'm open to suggestions at this point in time, I just don't see how this will solve the bonding problem since it still wouldn't load, right?

Quoted 3 lines

It would permit users to load ipv6 (thus allowing bonding to
load), but prevent ipv6 from actually doing anything. (because sys_socket, e.g., won't open an ipv6 socket if block_af includes ipv6).

Right, but it doesn't help someone that changed /etc/modprobe.conf to have "install ipv6 /bin/true" - they'll have to stop doing that.

I think changing ipv6 to support a disable_ipv6 module parameter like Vlad suggested would work, as long as we're not worried about someone opening an AF_INET6 socket - even if they do they won't get anywhere. That, along with the patch below to actually not add the addresses, would work (sorry in advance for using an attachment). I'll get started on that...

-Brian

--

The disable_ipv6 knob was meant to be used for the kernel to disable IPv6 on an interface when DAD failed for the link-local address based on the MAC, but we should also be able to administratively disable it on an interface, or the entire system. This patch fixes the per-interface problem.

Signed-off-by Brian Haley <[email protected]>

net/ipv6/addrconf.c +0 -0

index f8f76d6..90f2a81 100644
@@ -603,6 +603,11 @@ ipv6_add_addr(struct inet6_dev *idev, const struct in6_addr *addr, int pfxlen,
 		goto out2;

Quoted 11 lines · Brian Haley

Quoted 10 lines · Jay Vosburgh

Quoted 10 lines

Quoted 10 lines

Quoted 4 lines

I've been fooling with the disable_ipv6 sysctl, and one issue is
that, at least on the distro I'm testing on (SLES), it's not picked up from /etc/sysctl.conf at boot time (presumably because ipv6 isn't loaded yet, although I haven't really checked).

Correct, that's the problem.

We could create a blocker bitmap. Two sysctls, "block_af" and "unblock_af". You write the AF_foo value for the protocol there and it sets or clears the assosciated bit in the internal blocker bitmap.

Quoted 8 lines

Quoted 6 lines

Quoted 3 lines

Quoted 1 line

Things like sys_socket() et al. key off of this.

I'm open to suggestions at this point in time, I just don't see how this will solve the bonding problem since it still wouldn't load, right?

It would permit users to load ipv6 (thus allowing bonding to
load), but prevent ipv6 from actually doing anything. (because sys_socket, e.g., won't open an ipv6 socket if block_af includes ipv6).

Right, but it doesn't help someone that changed /etc/modprobe.conf to have "install ipv6 /bin/true" - they'll have to stop doing that.

Yes. There's no reasonable solution that won't require some
change for users that have aliased out ipv6.

Quoted 5 lines

I think changing ipv6 to support a disable_ipv6 module parameter like Vlad suggested would work, as long as we're not worried about someone opening an AF_INET6 socket - even if they do they won't get anywhere. That, along with the patch below to actually not add the addresses, would work (sorry in advance for using an attachment). I'll get started on that...

I agree that it would work, and could even be set up such that
opening sockets doesn't work, either (if ipv6 never registered via sock_register, for example). I'm sticking some on the opening sockets failure behavior because it's the current behavior if ipv6 is aliased out. It just seems like a logical place for the permission denial to occur, rather than later, and is consistent with what happens if ipv6 isn't loaded at all or is not configured in the kernel.

I still tend to like the bitmask to disable address family
gizmo. It's not specific to one particular protocol (although it would likely need a check in the protocols for things like addrconf). As somebody pointed out, there are likely to be (if not now, then relatively soon) users somewhere that want to turn off ipv4 and run ipv6 only.

Quoted 19 lines

-Brian

--

The disable_ipv6 knob was meant to be used for the kernel to disable IPv6 on an interface when DAD failed for the link-local address based on the MAC, but we should also be able to administratively disable it on an interface, or the entire system. This patch fixes the per-interface problem.

Signed-off-by Brian Haley <[email protected]>

net/ipv6/addrconf.c +0 -0

index f8f76d6..90f2a81 100644
@@ -603,6 +603,11 @@ ipv6_add_addr(struct inet6_dev *idev, const struct in6_addr *addr, int pfxlen,
 		goto out2;

net/ipv6/addrconf.c ipv6_add_addr(struct inet6_dev *idev, const struct in6_addr *addr, int pfxlen, · line 613

Quoted 10 lines

net/ipv6/addrconf.c +5 -0

 	}
 
+	if (idev->cnf.disable_ipv6) {
+		err = -EPERM;
+		goto out2;
+	}
+
 	write_lock(&addrconf_hash_lock);
 
 	/* Ignore adding duplicate addresses on an interface */

	-J

---
-Jay Vosburgh, IBM Linux Technology Center, [email protected]

Jay Vosburgh wrote:

Quoted 1 line

Brian Haley <[email protected]> wrote:

Quoted 14 lines

Quoted 5 lines

I think changing ipv6 to support a disable_ipv6 module parameter like Vlad suggested would work, as long as we're not worried about someone opening an AF_INET6 socket - even if they do they won't get anywhere. That, along with the patch below to actually not add the addresses, would work (sorry in advance for using an attachment). I'll get started on that...

I agree that it would work, and could even be set up such that
opening sockets doesn't work, either (if ipv6 never registered via sock_register, for example). I'm sticking some on the opening sockets failure behavior because it's the current behavior if ipv6 is aliased out. It just seems like a logical place for the permission denial to occur, rather than later, and is consistent with what happens if ipv6 isn't loaded at all or is not configured in the kernel.

Quoted 6 lines

I still tend to like the bitmask to disable address family
gizmo. It's not specific to one particular protocol (although it would likely need a check in the protocols for things like addrconf). As somebody pointed out, there are likely to be (if not now, then relatively soon) users somewhere that want to turn off ipv4 and run ipv6 only.

Yes. The bitmask to disable certain family can be useful, but it's orthogonal the issue of IPv6 support. As you said, it can be used to disable any address family that user wishes. The slight issue with this might be, should the settings affect already create sockets?

I guess it comes down how many levels of control to do we want to provide. Things that have been suggested so far:
1) Global on/off switch (i.e module parameter)
2) Per interface on/off switch (currently exists, but has bugs).
3) Socket on/off switch (i.e blocker bitmask)

I think numbers 1 and 2 turn off the IPv6 protocol on the wire, while number 3 turns off the interface to the user. The two can be done independent.

Yes.  The bitmask to disable certain family can be useful, but it's orthogonal the issue of IPv6 support.  As you said, it can be used to disable any address family that user wishes.  The slight issue with this might be, should the settings affect already create sockets?

I guess it comes down how many levels of control to do we want to provide. Things that have been suggested so far:
       1) Global on/off switch (i.e module parameter)
       2) Per interface on/off switch (currently exists, but has bugs).
       3) Socket on/off switch (i.e blocker bitmask)

I think numbers 1 and 2 turn off the IPv6 protocol on the wire, while number 3 turns off the interface to the user.  The two can be done independent.

I feel extremely nervous about people discussing disabling IPv6 going forward. Current estimates are that the first RIRs will begin to exhaust their address spaces (after IANA's address space is exhausted) early in 2011. If you consider that any change probably won't be in a released kernel until June or so, there would be all of 18 months left until IPv6 is *required* to contact some hosts on the internet.