igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 15 lines · Nicolas de Pesloüan

Quoted 6 lines · Le 23 déc. 2011 10:42, "Chris Boot" <[email protected] <mailto:[email protected]>>…

Hi folks,

As per Eric Dumazet and Dave Miller, I'm opening up a separate thread on this issue.

I have two identical servers in a cluster for running KVM virtual machines. They each have a

single connection to the Internet (irrelevant for this) and two gigabit connections between each other for cluster replication, etc... These two connections are in a balance-rr bonded connection, which is itself member of a bridge that the VMs attach to. I'm running v3.2-rc6-140-gb9e26df on Debian Wheezy.

Quoted 2 lines

When the bridge is brought up, IPv4 works fine but IPv6 does not. I can use neither the

Quoted 9 lines

automatic link-local on the brid ge nor the static global address I assign. Neither machine can perform neighbour discovery over the link until I put the bond members (eth0 and eth1) into promiscuous mode. I can do this either with tcpdump or 'ip link set dev ethX promisc on' and this is enough to make the link spring to life.

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too. And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc mode anyway... but you shoudn't have to do it by hand.

Sorry, I should have added that I tried this. Setting bond0 or br0 to promisc has no effect. I discovered this by running tcpdump on br0 first, then bond0, then eventually each bond member in turn. Only at the last stage did things jump to life.

Quoted 7 lines

Quoted 2 lines

This cluster is not currently live so I can easily test patches and various configurations.

Can you try to remove the bonding part, connecting eth0 and eth1 directly to br0 and see if it works better? (This is a test ony. I perfectly understand that you would loose balance-rr in this setup.)

Good call. Let's see.

I took br0 and bond0 apart, took eth0 and eth1 out of enforced promisc mode, then manually built a br0 with eth0 in only so I didn't cause a network loop. Adding eth0 to br0 did not make it go into promisc mode, but IPv6 does work over this setup. I also made sure ip -6 neigh was empty on both machines before I started.

I then decided to try the test with just the bond0 in balance-rr mode. Once again I took everything down and ensured no promisc mode and no ip -6 neigh. I noticed bond0 wasn't getting a link-local and I found out for some reason /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both servers so I set it to 0. That brought things to life.

So then I put it all back together again and it didn't work. I once again noticed disable_ipv6 was set on the bond0 interfaces, now part of the bridge. Toggling this on the _bond_ interface made things work again.

What's setting disable_ipv6? Should this be having an impact if the port is part of a bridge?

Chris

--
Chris Boot
[email protected]

[ Forwarded to netdev, because two previous e-mail erroneously sent in HTML ]

Quoted 60 lines · Chris Boot

Quoted 23 lines · Nicolas de Pesloüan

Quoted 4 lines · Le 23 déc. 2011 10:42, "Chris Boot" <[email protected] <mailto:[email protected]>>…

Hi folks,

As per Eric Dumazet and Dave Miller, I'm opening up a separate

thread on this issue.

Quoted 2 lines

I have two identical servers in a cluster for running KVM virtual

machines. They each have a
single connection to the Internet (irrelevant for this) and two gigabit connections between each other for cluster replication, etc... These two connections are in a balance-rr bonded connection, which is itself member of a bridge that the VMs attach to. I'm running v3.2-rc6-140-gb9e26df on Debian Wheezy.

Quoted 2 lines

When the bridge is brought up, IPv4 works fine but IPv6 does not.

I can use neither the
automatic link-local on the brid ge nor the static global address I assign. Neither machine can perform neighbour discovery over the link until I put the bond members (eth0 and eth1) into promiscuous mode. I can do this either with tcpdump or 'ip link set dev ethX promisc on' and this is enough to make the link spring to life.

Quoted 4 lines

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too. And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc mode anyway... but you shoudn't have to do it by hand.

Sorry, I should have added that I tried this. Setting bond0 or br0 to promisc has no effect. I discovered this by running tcpdump on br0 first, then bond0, then eventually each bond member in turn. Only at the last stage did things jump to life.

Quoted 8 lines

Quoted 2 lines

This cluster is not currently live so I can easily test patches

and various configurations.

Can you try to remove the bonding part, connecting eth0 and eth1 directly to br0 and see if it works better? (This is a test ony. I perfectly understand that you would loose balance-rr in this setup.)

Good call. Let's see.

I took br0 and bond0 apart, took eth0 and eth1 out of enforced promisc mode, then manually built a br0 with eth0 in only so I didn't cause a network loop. Adding eth0 to br0 did not make it go into promisc mode, but IPv6 does work over this setup. I also made sure ip
-6 neigh was empty on both
machines before I started.

I then decided to try the test with just the bond0 in balance-rr mode. Once again I took everything down and ensured no promisc mode and no ip -6 neigh. I noticed bond0 wasn't getting a link-local and I found out for some reason /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both servers so I set it to 0. That brought things to life.

So then I put it all back together again and it didn't work. I once again noticed disable_ipv6 was set on the bond0 interfaces, now part of the bridge. Toggling this on the _bond_ interface made things work again.

What's setting disable_ipv6? Should this be having an impact if the port is part of a bridge?

Quoted 83 lines · Nicolas de Pesloüan

[ Forwarded to netdev, because two previous e-mail erroneously sent in HTML ]

Quoted 79 lines · Chris Boot

Quoted 30 lines · Nicolas de Pesloüan

Quoted 4 lines · Le 23 déc. 2011 10:42, "Chris Boot" <[email protected] <mailto:[email protected]>>…

Hi folks,

As per Eric Dumazet and Dave Miller, I'm opening up a separate

thread on this issue.

Quoted 2 lines

I have two identical servers in a cluster for running KVM virtual

machines. They each have a
single connection to the Internet (irrelevant for this) and two gigabit connections between each
other for cluster replication, etc... These two connections are in a balance-rr bonded connection,
which is itself member of a bridge that the VMs attach to. I'm running v3.2-rc6-140-gb9e26df on
Debian Wheezy.

Quoted 2 lines

When the bridge is brought up, IPv4 works fine but IPv6 does not.

I can use neither the
automatic link-local on the brid ge nor the static global address I assign. Neither machine can
perform neighbour discovery over the link until I put the bond members (eth0 and eth1) into
promiscuous mode. I can do this either with tcpdump or 'ip link set dev ethX promisc on' and this
is enough to make the link spring to life.

Quoted 6 lines

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

Sorry, I should have added that I tried this. Setting bond0 or br0 to promisc has no effect. I
discovered this by running tcpdump on br0 first, then bond0, then eventually each bond member in
turn. Only at the last stage did things jump to life.

Quoted 10 lines

Quoted 2 lines

This cluster is not currently live so I can easily test patches

and various configurations.

Can you try to remove the bonding part, connecting eth0 and eth1 directly to br0 and see if it
works better? (This is a test ony. I perfectly understand that you would loose balance-rr in this
setup.)

Good call. Let's see.

I took br0 and bond0 apart, took eth0 and eth1 out of enforced promisc mode, then manually built a
br0 with eth0 in only so I didn't cause a network loop. Adding eth0 to br0 did not make it go into
promisc mode, but IPv6 does work over this setup. I also made sure ip -6 neigh was empty on both
machines before I started.

I then decided to try the test with just the bond0 in balance-rr mode. Once again I took everything
down and ensured no promisc mode and no ip -6 neigh. I noticed bond0 wasn't getting a link-local and
I found out for some reason /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both servers so I set it to 0. That brought things to life.

So then I put it all back together again and it didn't work. I once again noticed disable_ipv6 was
set on the bond0 interfaces, now part of the bridge. Toggling this on the _bond_ interface made
things work again.

Quoted 2 lines

Quoted 2 lines

What's setting disable_ipv6? Should this be having an impact if the port is part of a bridge?

Hmm, as a further update... I brought up my VMs on the bridge with disable_ipv6 turned off. The VMs on one host couldn't see what was on the other side of the bridge (on the other server) until I turned promisc back on manually. So it's not entirely disable_ipv6's fault.

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Chris Boot
Sent: Tuesday, December 27, 2011 1:53 PM
To: Nicolas de Pesloüan
Cc: netdev
Subject: Re: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 2 lines · Chris Boot

Quoted 1 line · Nicolas de Pesloüan

[ Forwarded to netdev, because two previous e-mail erroneously sent

in

Quoted 11 lines

Quoted 11 lines

HTML ]

Quoted 8 lines · Chris Boot

Quoted 7 lines · Nicolas de Pesloüan

Quoted 3 lines · Le 23 déc. 2011 10:42, "Chris Boot" <[email protected] <mailto:[email protected]>>…

Hi folks,

Quoted 8 lines

Quoted 8 lines

Quoted 8 lines

Quoted 8 lines

Quoted 1 line

As per Eric Dumazet and Dave Miller, I'm opening up a separate

thread on this issue.

Quoted 2 lines

I have two identical servers in a cluster for running KVM virtual

machines. They each have a
single connection to the Internet (irrelevant for this) and two gigabit connections between each
other for cluster replication, etc... These two connections are in

a

Quoted 15 lines

Quoted 15 lines

Quoted 15 lines

Quoted 15 lines

balance-rr bonded connection,
which is itself member of a bridge that the VMs attach to. I'm running v3.2-rc6-140-gb9e26df on
Debian Wheezy.

Quoted 2 lines

When the bridge is brought up, IPv4 works fine but IPv6 does not.

I can use neither the
automatic link-local on the brid ge nor the static global address I assign. Neither machine can
perform neighbour discovery over the link until I put the bond members (eth0 and eth1) into
promiscuous mode. I can do this either with tcpdump or 'ip link set dev ethX promisc on' and this
is enough to make the link spring to life.

Quoted 8 lines

Quoted 8 lines

Quoted 8 lines

Quoted 6 lines

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

Sorry, I should have added that I tried this. Setting bond0 or br0

to

Quoted 16 lines

Quoted 16 lines

Quoted 16 lines

promisc has no effect. I
discovered this by running tcpdump on br0 first, then bond0, then eventually each bond member in
turn. Only at the last stage did things jump to life.

Quoted 10 lines

Quoted 2 lines

This cluster is not currently live so I can easily test patches

and various configurations.

Can you try to remove the bonding part, connecting eth0 and eth1 directly to br0 and see if it
works better? (This is a test ony. I perfectly understand that you would loose balance-rr in this
setup.)

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Good call. Let's see.

I took br0 and bond0 apart, took eth0 and eth1 out of enforced promisc mode, then manually built a
br0 with eth0 in only so I didn't cause a network loop. Adding eth0 to br0 did not make it go into
promisc mode, but IPv6 does work over this setup. I also made sure

ip

Quoted 9 lines

Quoted 9 lines

Quoted 9 lines

-6 neigh was empty on both
machines before I started.

I then decided to try the test with just the bond0 in balance-rr mode. Once again I took everything
down and ensured no promisc mode and no ip -6 neigh. I noticed bond0 wasn't getting a link-local and
I found out for some reason /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both servers

so I

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

set it to 0. That brought things to life.

So then I put it all back together again and it didn't work. I once again noticed disable_ipv6 was
set on the bond0 interfaces, now part of the bridge. Toggling this

on

Quoted 10 lines

Quoted 5 lines

Quoted 5 lines

the _bond_ interface made
things work again.

What's setting disable_ipv6? Should this be having an impact if the port is part of a bridge?

Hmm, as a further update... I brought up my VMs on the bridge with disable_ipv6 turned off. The VMs on one host couldn't see what was on the other side of the bridge (on the other server) until I turned promisc back on manually. So it's not entirely disable_ipv6's fault.

Hi,

I don't want this to get lost around the Christmas break, so I'm just resending it. I'm still seeing the same behaviour as before.

From above:

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

This definitely doesn't happen, at least according to 'ip link show | grep PROMISC'.

Chris

--
Chris Boot
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Wyborny, Carolyn
Sent: Tuesday, January 03, 2012 3:24 PM
To: Chris Boot; Nicolas de Pesloüan
Cc: netdev; [email protected]
Subject: RE: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 2 lines

-----Original Message-----
From: [email protected] [mailto:netdev-

[email protected]]

Quoted 7 lines

On Behalf Of Chris Boot
Sent: Tuesday, December 27, 2011 1:53 PM
To: Nicolas de Pesloüan
Cc: netdev
Subject: Re: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 19 lines

Quoted 2 lines · Chris Boot

Quoted 1 line · Nicolas de Pesloüan

[ Forwarded to netdev, because two previous e-mail erroneously sent

in

Quoted 15 lines

Quoted 15 lines

HTML ]

Quoted 12 lines · Chris Boot

Quoted 11 lines · Nicolas de Pesloüan

Quoted 4 lines · Le 23 déc. 2011 10:42, "Chris Boot" <[email protected] <mailto:[email protected]>>…

Hi folks,

As per Eric Dumazet and Dave Miller, I'm opening up a separate

thread on this issue.

Quoted 2 lines

I have two identical servers in a cluster for running KVM

virtual

Quoted 11 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

machines. They each have a
single connection to the Internet (irrelevant for this) and two gigabit connections between each
other for cluster replication, etc... These two connections are in

a

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

balance-rr bonded connection,
which is itself member of a bridge that the VMs attach to. I'm running v3.2-rc6-140-gb9e26df on
Debian Wheezy.

Quoted 2 lines

When the bridge is brought up, IPv4 works fine but IPv6 does

not.

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

I can use neither the
automatic link-local on the brid ge nor the static global address

I

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

assign. Neither machine can
perform neighbour discovery over the link until I put the bond members (eth0 and eth1) into
promiscuous mode. I can do this either with tcpdump or 'ip link

set

Quoted 28 lines

Quoted 11 lines

Quoted 11 lines

Quoted 11 lines

Quoted 9 lines

dev ethX promisc on' and this
is enough to make the link spring to life.

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

Sorry, I should have added that I tried this. Setting bond0 or br0

to

Quoted 16 lines

Quoted 16 lines

Quoted 16 lines

promisc has no effect. I
discovered this by running tcpdump on br0 first, then bond0, then eventually each bond member in
turn. Only at the last stage did things jump to life.

Quoted 10 lines

Quoted 2 lines

This cluster is not currently live so I can easily test patches

and various configurations.

Can you try to remove the bonding part, connecting eth0 and eth1 directly to br0 and see if it
works better? (This is a test ony. I perfectly understand that you would loose balance-rr in this
setup.)

Quoted 14 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Good call. Let's see.

I took br0 and bond0 apart, took eth0 and eth1 out of enforced promisc mode, then manually built a
br0 with eth0 in only so I didn't cause a network loop. Adding eth0 to br0 did not make it go into
promisc mode, but IPv6 does work over this setup. I also made sure

ip

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

-6 neigh was empty on both
machines before I started.

I then decided to try the test with just the bond0 in balance-rr mode. Once again I took everything
down and ensured no promisc mode and no ip -6 neigh. I noticed

bond0

Quoted 39 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

wasn't getting a link-local and
I found out for some reason /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both servers

so I

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

set it to 0. That brought things to life.

So then I put it all back together again and it didn't work. I once again noticed disable_ipv6 was
set on the bond0 interfaces, now part of the bridge. Toggling this

on

Quoted 10 lines

Quoted 5 lines

Quoted 5 lines

the _bond_ interface made
things work again.

What's setting disable_ipv6? Should this be having an impact if the port is part of a bridge?

Hmm, as a further update... I brought up my VMs on the bridge with disable_ipv6 turned off. The VMs on one host couldn't see what was on the other side of the bridge (on the other server) until I turned promisc back on manually. So it's not entirely disable_ipv6's fault.

Hi,

I don't want this to get lost around the Christmas break, so I'm just resending it. I'm still seeing the same behaviour as before.

From above:

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

This definitely doesn't happen, at least according to 'ip link show | grep PROMISC'.

Chris

Quoted 7 lines

--
Chris Boot
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Sorry for the delay in responding. I'm not sure what is going on here and I'm not our bonding expert who is still out on holidays. However, we'll try to reproduce this. When I get some more advice, I may be asking for some more data.

Thanks,

Carolyn
Carolyn Wyborny
Linux Development
LAN Access Division
Intel Corporation N�����r��y���b�X��ǧv�^�)޺{.n�+���z�^�)���w* jg��������ݢj/���z�ޖ��2�ޙ���&�)ߡ�a�����G���h��j:+v���w�٥

Quoted 160 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Wyborny, Carolyn
Sent: Tuesday, January 03, 2012 3:24 PM
To: Chris Boot; Nicolas de Pesloüan
Cc: netdev; [email protected]
Subject: RE: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 2 lines

-----Original Message-----
From: [email protected] [mailto:netdev-

[email protected]]

Quoted 7 lines

On Behalf Of Chris Boot
Sent: Tuesday, December 27, 2011 1:53 PM
To: Nicolas de Pesloüan
Cc: netdev
Subject: Re: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 19 lines

Quoted 2 lines · Chris Boot

Quoted 1 line · Nicolas de Pesloüan

[ Forwarded to netdev, because two previous e-mail erroneously sent

in

Quoted 15 lines

Quoted 15 lines

HTML ]

Quoted 12 lines · Chris Boot

Quoted 11 lines · Nicolas de Pesloüan

Quoted 4 lines · Le 23 déc. 2011 10:42, "Chris Boot"<[email protected] <mailto:[email protected]>>…

Hi folks,

As per Eric Dumazet and Dave Miller, I'm opening up a separate

thread on this issue.

Quoted 2 lines

I have two identical servers in a cluster for running KVM

virtual

Quoted 11 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

machines. They each have a
single connection to the Internet (irrelevant for this) and two gigabit connections between each
other for cluster replication, etc... These two connections are in

a

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

balance-rr bonded connection,
which is itself member of a bridge that the VMs attach to. I'm running v3.2-rc6-140-gb9e26df on
Debian Wheezy.

Quoted 2 lines

When the bridge is brought up, IPv4 works fine but IPv6 does

not.

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

I can use neither the
automatic link-local on the brid ge nor the static global address

I

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

assign. Neither machine can
perform neighbour discovery over the link until I put the bond members (eth0 and eth1) into
promiscuous mode. I can do this either with tcpdump or 'ip link

set

Quoted 28 lines

Quoted 11 lines

Quoted 11 lines

Quoted 11 lines

Quoted 9 lines

dev ethX promisc on' and this
is enough to make the link spring to life.

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

Sorry, I should have added that I tried this. Setting bond0 or br0

to

Quoted 16 lines

Quoted 16 lines

Quoted 16 lines

promisc has no effect. I
discovered this by running tcpdump on br0 first, then bond0, then eventually each bond member in
turn. Only at the last stage did things jump to life.

Quoted 10 lines

Quoted 2 lines

This cluster is not currently live so I can easily test patches

and various configurations.

Can you try to remove the bonding part, connecting eth0 and eth1 directly to br0 and see if it
works better? (This is a test ony. I perfectly understand that you would loose balance-rr in this
setup.)

Quoted 14 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Good call. Let's see.

I took br0 and bond0 apart, took eth0 and eth1 out of enforced promisc mode, then manually built a
br0 with eth0 in only so I didn't cause a network loop. Adding eth0 to br0 did not make it go into
promisc mode, but IPv6 does work over this setup. I also made sure

ip

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

-6 neigh was empty on both
machines before I started.

I then decided to try the test with just the bond0 in balance-rr mode. Once again I took everything
down and ensured no promisc mode and no ip -6 neigh. I noticed

bond0

Quoted 39 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

wasn't getting a link-local and
I found out for some reason /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both servers

so I

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

set it to 0. That brought things to life.

So then I put it all back together again and it didn't work. I once again noticed disable_ipv6 was
set on the bond0 interfaces, now part of the bridge. Toggling this

on

Quoted 10 lines

Quoted 5 lines

Quoted 5 lines

the _bond_ interface made
things work again.

What's setting disable_ipv6? Should this be having an impact if the port is part of a bridge?

Hmm, as a further update... I brought up my VMs on the bridge with disable_ipv6 turned off. The VMs on one host couldn't see what was on the other side of the bridge (on the other server) until I turned promisc back on manually. So it's not entirely disable_ipv6's fault.

Hi,

I don't want this to get lost around the Christmas break, so I'm just resending it. I'm still seeing the same behaviour as before.

From above:

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

This definitely doesn't happen, at least according to 'ip link show | grep PROMISC'.

Chris

Quoted 7 lines

--
Chris Boot
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Sorry for the delay in responding. I'm not sure what is going on here and I'm not our bonding expert who is still out on holidays. However, we'll try to reproduce this. When I get some more advice, I may be asking for some more data.

Thanks,

Quoted 7 lines

Carolyn
Carolyn Wyborny
Linux Development
LAN Access Division
Intel Corporation N�����r��y���b�X��ǧv�^�)޺{.n�+���z�^�)���w* jg��������ݢj/���z�ޖ��2�ޙ���&�)ߡ�a�����G���h��j:+v���w�٥

Hello,

Check your ip_forward configuration on your bridge to make sure its configured to forward ipv6 packets and also please send the contents of /etc/modprobe.d/bonding.conf and the contents of your routing table and we'll continue to work on this.

iface bond0 inet manual
slaves eth0 eth1
bond-mode balance-rr
bond-miimon 100
bond-downdelay 200
bond-updelay 200

iface br0 inet static

        address [snip]
        netmask 255.255.255.224
        bridge_ports bond0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 5

iface br0 inet6 static
address [snip]
netmask 64

Quoted 168 lines · Carolyn

Quoted 160 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Wyborny, Carolyn
Sent: Tuesday, January 03, 2012 3:24 PM
To: Chris Boot; Nicolas de Pesloüan
Cc: netdev; [email protected]
Subject: RE: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 2 lines

-----Original Message-----
From: [email protected] [mailto:netdev-

[email protected]]

Quoted 7 lines

On Behalf Of Chris Boot
Sent: Tuesday, December 27, 2011 1:53 PM
To: Nicolas de Pesloüan
Cc: netdev
Subject: Re: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 19 lines

Quoted 2 lines · Chris Boot

Quoted 1 line · Nicolas de Pesloüan

[ Forwarded to netdev, because two previous e-mail erroneously sent

in

Quoted 15 lines

Quoted 15 lines

HTML ]

Quoted 12 lines · Chris Boot

Quoted 11 lines · Nicolas de Pesloüan

Quoted 4 lines · Le 23 déc. 2011 10:42, "Chris Boot"<[email protected] <mailto:[email protected]>>…

Hi folks,

As per Eric Dumazet and Dave Miller, I'm opening up a separate

thread on this issue.

Quoted 2 lines

I have two identical servers in a cluster for running KVM

virtual

Quoted 11 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

machines. They each have a
single connection to the Internet (irrelevant for this) and two gigabit connections between each
other for cluster replication, etc... These two connections are in

a

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

balance-rr bonded connection,
which is itself member of a bridge that the VMs attach to. I'm running v3.2-rc6-140-gb9e26df on
Debian Wheezy.

Quoted 2 lines

When the bridge is brought up, IPv4 works fine but IPv6 does

not.

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

I can use neither the
automatic link-local on the brid ge nor the static global address

I

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

assign. Neither machine can
perform neighbour discovery over the link until I put the bond members (eth0 and eth1) into
promiscuous mode. I can do this either with tcpdump or 'ip link

set

Quoted 28 lines

Quoted 11 lines

Quoted 11 lines

Quoted 11 lines

Quoted 9 lines

dev ethX promisc on' and this
is enough to make the link spring to life.

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

Sorry, I should have added that I tried this. Setting bond0 or br0

to

Quoted 16 lines

Quoted 16 lines

Quoted 16 lines

promisc has no effect. I
discovered this by running tcpdump on br0 first, then bond0, then eventually each bond member in
turn. Only at the last stage did things jump to life.

Quoted 10 lines

Quoted 2 lines

This cluster is not currently live so I can easily test patches

and various configurations.

Can you try to remove the bonding part, connecting eth0 and eth1 directly to br0 and see if it
works better? (This is a test ony. I perfectly understand that you would loose balance-rr in this
setup.)

Quoted 14 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Good call. Let's see.

I took br0 and bond0 apart, took eth0 and eth1 out of enforced promisc mode, then manually built a
br0 with eth0 in only so I didn't cause a network loop. Adding eth0 to br0 did not make it go into
promisc mode, but IPv6 does work over this setup. I also made sure

ip

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

-6 neigh was empty on both
machines before I started.

I then decided to try the test with just the bond0 in balance-rr mode. Once again I took everything
down and ensured no promisc mode and no ip -6 neigh. I noticed

bond0

Quoted 39 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

wasn't getting a link-local and
I found out for some reason /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both servers

so I

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

set it to 0. That brought things to life.

So then I put it all back together again and it didn't work. I once again noticed disable_ipv6 was
set on the bond0 interfaces, now part of the bridge. Toggling this

on

Quoted 10 lines

Quoted 5 lines

Quoted 5 lines

the _bond_ interface made
things work again.

What's setting disable_ipv6? Should this be having an impact if the port is part of a bridge?

Hmm, as a further update... I brought up my VMs on the bridge with disable_ipv6 turned off. The VMs on one host couldn't see what was on the other side of the bridge (on the other server) until I turned promisc back on manually. So it's not entirely disable_ipv6's fault.

Hi,

I don't want this to get lost around the Christmas break, so I'm just resending it. I'm still seeing the same behaviour as before.

From above:

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

This definitely doesn't happen, at least according to 'ip link show | grep PROMISC'.

Chris

Quoted 7 lines

--
Chris Boot
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Sorry for the delay in responding. I'm not sure what is going on here and I'm not our bonding expert who is still out on holidays. However, we'll try to reproduce this. When I get some more advice, I may be asking for some more data.

Thanks,

Quoted 7 lines

Carolyn
Carolyn Wyborny
Linux Development
LAN Access Division
Intel Corporation N�����r��y���b�X��ǧv�^�)޺{.n�+���z�^�)���w* jg��������ݢj/���z�ޖ��2�ޙ���&�)ߡ�a�����G���h��j:+v���w�٥

Quoted 3 lines

Hello,

Check your ip_forward configuration on your bridge to make sure its configured to forward ipv6 packets and also please send the contents of /etc/modprobe.d/bonding.conf and the contents of your routing table and we'll continue to work on this.

Hi Carolyn,

Surely ip_forward only needs to be set if I'm wanting to _route_ IPv6 rather than simply have them go through a bridge untouched? I don't want the host to route IPv6 at all. Setting this also has the unintended effect of disabling SLAAC which I wish to keep enabled.

I don't have a /etc/modprobe.d/bonding.conf; I'm using Debian and configuring my bonding and bridging using the configuration I pasted in my original email. Here it is again:

Quoted 14 lines

iface bond0 inet manual
slaves eth0 eth1
bond-mode balance-rr
bond-miimon 100
bond-downdelay 200
bond-updelay 200

iface br0 inet static

       address [snip]
       netmask 255.255.255.224
       bridge_ports bond0
       bridge_stp off
       bridge_fd 0
       bridge_maxwait 5

Quoted 3 lines

iface br0 inet6 static
address [snip]
netmask 64

Despite the static IPv6 address I use SLAAC to grab a default gateway.

My IPv6 routing table:

2001:8b0:49:200::/64 dev br0 proto kernel metric 256 expires 2592317sec fe80::/64 dev br0 proto kernel metric 256

fe80::/64 dev bond1  proto kernel  metric 256
fe80::/64 dev vnet0  proto kernel  metric 256
fe80::/64 dev vnet1  proto kernel  metric 256
fe80::/64 dev vnet2  proto kernel  metric 256
fe80::/64 dev vnet3  proto kernel  metric 256
fe80::/64 dev vnet4  proto kernel  metric 256

default via fe80::5652:ff:fe16:15a0 dev br0 proto kernel metric 1024 expires 1793sec

HTH,
Chris

Signature · 6 lines

Chris Boot
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

-----Original Message-----
From: Chris Boot [mailto:[email protected]]
Sent: Wednesday, January 04, 2012 8:58 AM
To: Wyborny, Carolyn
Cc: Nicolas de Pesloüan; netdev; [email protected]
Subject: Re: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 4 lines · Carolyn

Quoted 4 lines

-----Original Message-----
From: [email protected] [mailto:netdev-

[email protected]]

Quoted 19 lines

Quoted 19 lines

On Behalf Of Wyborny, Carolyn
Sent: Tuesday, January 03, 2012 3:24 PM
To: Chris Boot; Nicolas de Pesloüan
Cc: netdev; [email protected]
Subject: RE: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 2 lines

-----Original Message-----
From: [email protected] [mailto:netdev-

[email protected]]

Quoted 7 lines

On Behalf Of Chris Boot
Sent: Tuesday, December 27, 2011 1:53 PM
To: Nicolas de Pesloüan
Cc: netdev
Subject: Re: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines · Chris Boot

Quoted 2 lines · Nicolas de Pesloüan

Quoted 1 line

[ Forwarded to netdev, because two previous e-mail erroneously

sent

Quoted 13 lines

Quoted 13 lines

Quoted 13 lines

in

Quoted 12 lines

Quoted 12 lines

HTML ]

Quoted 9 lines · Chris Boot

Quoted 8 lines · Nicolas de Pesloüan

Quoted 4 lines · Le 23 déc. 2011 10:42, "Chris Boot"<[email protected] <mailto:[email protected]>>…

Hi folks,

As per Eric Dumazet and Dave Miller, I'm opening up a separate

Quoted 8 lines

Quoted 8 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

thread on this issue.

Quoted 2 lines

I have two identical servers in a cluster for running KVM

virtual

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

machines. They each have a
single connection to the Internet (irrelevant for this) and two gigabit connections between each
other for cluster replication, etc... These two connections are

in

Quoted 10 lines

Quoted 10 lines

Quoted 7 lines

a

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

balance-rr bonded connection,
which is itself member of a bridge that the VMs attach to. I'm running v3.2-rc6-140-gb9e26df on
Debian Wheezy.

Quoted 2 lines

When the bridge is brought up, IPv4 works fine but IPv6 does

not.

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

I can use neither the
automatic link-local on the brid ge nor the static global

address

Quoted 10 lines

Quoted 10 lines

I

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

assign. Neither machine can
perform neighbour discovery over the link until I put the bond members (eth0 and eth1) into
promiscuous mode. I can do this either with tcpdump or 'ip link

set

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

dev ethX promisc on' and this
is enough to make the link spring to life.

For as far as I remember, setting bond0 to promisc should set

the

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 5 lines

bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

Sorry, I should have added that I tried this. Setting bond0 or

br0

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

to

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

promisc has no effect. I
discovered this by running tcpdump on br0 first, then bond0, then eventually each bond member in
turn. Only at the last stage did things jump to life.

Quoted 1 line

Quoted 1 line

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 1 line

This cluster is not currently live so I can easily test patches

and various configurations.

Can you try to remove the bonding part, connecting eth0 and eth1 directly to br0 and see if it
works better? (This is a test ony. I perfectly understand that

you

Quoted 9 lines

Quoted 9 lines

Quoted 9 lines

Quoted 9 lines

Quoted 9 lines

Quoted 9 lines

Quoted 3 lines

would loose balance-rr in this
setup.)

Good call. Let's see.

I took br0 and bond0 apart, took eth0 and eth1 out of enforced promisc mode, then manually built a
br0 with eth0 in only so I didn't cause a network loop. Adding

eth0

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

to br0 did not make it go into
promisc mode, but IPv6 does work over this setup. I also made

sure

Quoted 11 lines

Quoted 11 lines

Quoted 7 lines

ip

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

-6 neigh was empty on both
machines before I started.

I then decided to try the test with just the bond0 in balance-rr mode. Once again I took everything
down and ensured no promisc mode and no ip -6 neigh. I noticed

bond0

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

wasn't getting a link-local and
I found out for some reason /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both

servers

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

so I

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

set it to 0. That brought things to life.

So then I put it all back together again and it didn't work. I

once

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

again noticed disable_ipv6 was
set on the bond0 interfaces, now part of the bridge. Toggling

this

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

on

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

the _bond_ interface made
things work again.

What's setting disable_ipv6? Should this be having an impact if

the

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 1 line

Quoted 1 line

port is part of a bridge?

Hmm, as a further update... I brought up my VMs on the bridge with disable_ipv6 turned off. The VMs on one host couldn't see what was

on

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

the other side of the bridge (on the other server) until I turned promisc back on manually. So it's not entirely disable_ipv6's

fault.

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Hi,

I don't want this to get lost around the Christmas break, so I'm

just

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

resending it. I'm still seeing the same behaviour as before.

From above:

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

For as far as I remember, setting bond0 to promisc should set

the

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

This definitely doesn't happen, at least according to 'ip link show

|

Quoted 12 lines

Quoted 12 lines

Quoted 11 lines

grep PROMISC'.

Chris

--
Chris Boot
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 1 line

Quoted 1 line

Sorry for the delay in responding. I'm not sure what is going on

here

Quoted 1 line

Quoted 1 line

and I'm not our bonding expert who is still out on holidays.

However,

Quoted 13 lines

Quoted 5 lines

we'll try to reproduce this. When I get some more advice, I may be asking for some more data.

Thanks,

Quoted 7 lines

Carolyn
Carolyn Wyborny
Linux Development
LAN Access Division
Intel Corporation N�����r��y���b�X��ǧv�^�)޺{.n�+���z�^�)���w* jg��������ݢj/���z�ޖ��2�ޙ���&�)ߡ�a�����G���h��j:+v���w�٥

Quoted 3 lines

Hello,

Check your ip_forward configuration on your bridge to make sure its

configured to forward ipv6 packets and also please send the contents of /etc/modprobe.d/bonding.conf and the contents of your routing table and we'll continue to work on this.

Hi Carolyn,

Surely ip_forward only needs to be set if I'm wanting to _route_ IPv6 rather than simply have them go through a bridge untouched? I don't want the host to route IPv6 at all. Setting this also has the unintended effect of disabling SLAAC which I wish to keep enabled.

I don't have a /etc/modprobe.d/bonding.conf; I'm using Debian and configuring my bonding and bridging using the configuration I pasted in my original email. Here it is again:

Quoted 14 lines

iface bond0 inet manual
slaves eth0 eth1
bond-mode balance-rr
bond-miimon 100
bond-downdelay 200
bond-updelay 200

iface br0 inet static

        address [snip]
        netmask 255.255.255.224
        bridge_ports bond0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 5

Quoted 3 lines

iface br0 inet6 static
address [snip]
netmask 64

Despite the static IPv6 address I use SLAAC to grab a default gateway.

My IPv6 routing table:

2001:8b0:49:200::/64 dev br0 proto kernel metric 256 expires 2592317sec
fe80::/64 dev br0 proto kernel metric 256

fe80::/64 dev bond1  proto kernel  metric 256
fe80::/64 dev vnet0  proto kernel  metric 256
fe80::/64 dev vnet1  proto kernel  metric 256
fe80::/64 dev vnet2  proto kernel  metric 256
fe80::/64 dev vnet3  proto kernel  metric 256
fe80::/64 dev vnet4  proto kernel  metric 256

default via fe80::5652:ff:fe16:15a0 dev br0 proto kernel metric 1024 expires 1793sec

HTH,
Chris

--
Chris Boot
[email protected]

Quoted 251 lines

-----Original Message-----
From: Chris Boot [mailto:[email protected]]
Sent: Wednesday, January 04, 2012 8:58 AM
To: Wyborny, Carolyn
Cc: Nicolas de Pesloüan; netdev; [email protected]
Subject: Re: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 4 lines · Carolyn

Quoted 4 lines

-----Original Message-----
From: [email protected] [mailto:netdev-

[email protected]]

Quoted 19 lines

Quoted 19 lines

On Behalf Of Wyborny, Carolyn
Sent: Tuesday, January 03, 2012 3:24 PM
To: Chris Boot; Nicolas de Pesloüan
Cc: netdev; [email protected]
Subject: RE: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 2 lines

-----Original Message-----
From: [email protected] [mailto:netdev-

[email protected]]

Quoted 7 lines

On Behalf Of Chris Boot
Sent: Tuesday, December 27, 2011 1:53 PM
To: Nicolas de Pesloüan
Cc: netdev
Subject: Re: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines · Chris Boot

Quoted 2 lines · Nicolas de Pesloüan

Quoted 1 line

[ Forwarded to netdev, because two previous e-mail erroneously

sent

Quoted 13 lines

Quoted 13 lines

Quoted 13 lines

in

Quoted 12 lines

Quoted 12 lines

HTML ]

Quoted 9 lines · Chris Boot

Quoted 8 lines · Nicolas de Pesloüan

Quoted 4 lines · Le 23 déc. 2011 10:42, "Chris Boot"<[email protected] <mailto:[email protected]>>…

Hi folks,

As per Eric Dumazet and Dave Miller, I'm opening up a separate

Quoted 8 lines

Quoted 8 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

thread on this issue.

Quoted 2 lines

I have two identical servers in a cluster for running KVM

virtual

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

machines. They each have a
single connection to the Internet (irrelevant for this) and two gigabit connections between each
other for cluster replication, etc... These two connections are

in

Quoted 10 lines

Quoted 10 lines

Quoted 7 lines

a

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

balance-rr bonded connection,
which is itself member of a bridge that the VMs attach to. I'm running v3.2-rc6-140-gb9e26df on
Debian Wheezy.

Quoted 2 lines

When the bridge is brought up, IPv4 works fine but IPv6 does

not.

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

I can use neither the
automatic link-local on the brid ge nor the static global

address

Quoted 10 lines

Quoted 10 lines

I

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

assign. Neither machine can
perform neighbour discovery over the link until I put the bond members (eth0 and eth1) into
promiscuous mode. I can do this either with tcpdump or 'ip link

set

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

dev ethX promisc on' and this
is enough to make the link spring to life.

For as far as I remember, setting bond0 to promisc should set

the

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 5 lines

bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

Sorry, I should have added that I tried this. Setting bond0 or

br0

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

to

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

promisc has no effect. I
discovered this by running tcpdump on br0 first, then bond0, then eventually each bond member in
turn. Only at the last stage did things jump to life.

Quoted 1 line

Quoted 1 line

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 1 line

This cluster is not currently live so I can easily test patches

and various configurations.

Can you try to remove the bonding part, connecting eth0 and eth1 directly to br0 and see if it
works better? (This is a test ony. I perfectly understand that

you

Quoted 9 lines

Quoted 9 lines

Quoted 9 lines

Quoted 9 lines

Quoted 9 lines

Quoted 9 lines

Quoted 3 lines

would loose balance-rr in this
setup.)

Good call. Let's see.

I took br0 and bond0 apart, took eth0 and eth1 out of enforced promisc mode, then manually built a
br0 with eth0 in only so I didn't cause a network loop. Adding

eth0

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

to br0 did not make it go into
promisc mode, but IPv6 does work over this setup. I also made

sure

Quoted 11 lines

Quoted 11 lines

Quoted 7 lines

ip

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

-6 neigh was empty on both
machines before I started.

I then decided to try the test with just the bond0 in balance-rr mode. Once again I took everything
down and ensured no promisc mode and no ip -6 neigh. I noticed

bond0

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

wasn't getting a link-local and
I found out for some reason /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both

servers

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

so I

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

set it to 0. That brought things to life.

So then I put it all back together again and it didn't work. I

once

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

again noticed disable_ipv6 was
set on the bond0 interfaces, now part of the bridge. Toggling

this

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

on

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

the _bond_ interface made
things work again.

What's setting disable_ipv6? Should this be having an impact if

the

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 1 line

Quoted 1 line

port is part of a bridge?

Hmm, as a further update... I brought up my VMs on the bridge with disable_ipv6 turned off. The VMs on one host couldn't see what was

on

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

the other side of the bridge (on the other server) until I turned promisc back on manually. So it's not entirely disable_ipv6's

fault.

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Hi,

I don't want this to get lost around the Christmas break, so I'm

just

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

resending it. I'm still seeing the same behaviour as before.

From above:

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

For as far as I remember, setting bond0 to promisc should set

the

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

bonding member to promisc too.
And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
mode anyway... but you shoudn't have to do it by hand.

This definitely doesn't happen, at least according to 'ip link show

|

Quoted 12 lines

Quoted 12 lines

Quoted 11 lines

grep PROMISC'.

Chris

--
Chris Boot
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 1 line

Quoted 1 line

Sorry for the delay in responding. I'm not sure what is going on

here

Quoted 1 line

Quoted 1 line

and I'm not our bonding expert who is still out on holidays.

However,

Quoted 13 lines

Quoted 5 lines

we'll try to reproduce this. When I get some more advice, I may be asking for some more data.

Thanks,

Quoted 7 lines

Carolyn
Carolyn Wyborny
Linux Development
LAN Access Division
Intel Corporation N�����r��y���b�X��ǧv�^�)޺{.n�+���z�^�)���w* jg��������ݢj/���z�ޖ��2�ޙ���&�)ߡ�a�����G���h��j:+v���w�٥

Quoted 3 lines

Hello,

Check your ip_forward configuration on your bridge to make sure its

configured to forward ipv6 packets and also please send the contents of /etc/modprobe.d/bonding.conf and the contents of your routing table and we'll continue to work on this.

Hi Carolyn,

Surely ip_forward only needs to be set if I'm wanting to _route_ IPv6 rather than simply have them go through a bridge untouched? I don't want the host to route IPv6 at all. Setting this also has the unintended effect of disabling SLAAC which I wish to keep enabled.

I don't have a /etc/modprobe.d/bonding.conf; I'm using Debian and configuring my bonding and bridging using the configuration I pasted in my original email. Here it is again:

Quoted 14 lines

iface bond0 inet manual
slaves eth0 eth1
bond-mode balance-rr
bond-miimon 100
bond-downdelay 200
bond-updelay 200

iface br0 inet static

         address [snip]
         netmask 255.255.255.224
         bridge_ports bond0
         bridge_stp off
         bridge_fd 0
         bridge_maxwait 5

Quoted 3 lines

iface br0 inet6 static
address [snip]
netmask 64

Despite the static IPv6 address I use SLAAC to grab a default gateway.

My IPv6 routing table:

2001:8b0:49:200::/64 dev br0 proto kernel metric 256 expires 2592317sec
fe80::/64 dev br0 proto kernel metric 256

fe80::/64 dev bond1  proto kernel  metric 256
fe80::/64 dev vnet0  proto kernel  metric 256
fe80::/64 dev vnet1  proto kernel  metric 256
fe80::/64 dev vnet2  proto kernel  metric 256
fe80::/64 dev vnet3  proto kernel  metric 256
fe80::/64 dev vnet4  proto kernel  metric 256

default via fe80::5652:ff:fe16:15a0 dev br0 proto kernel metric 1024 expires 1793sec

Quoted 6 lines

HTH,
Chris

--
Chris Boot
[email protected]

This does seem more like a bonding problem than a driver problem and we haven't seen a lot of ipv6 using with bonding, so we may be in new territory here. Do you have any other adapters, our or anyone else's to try the same setup and see if the problem persists?

There is a situation between the bonding driver and all our drivers where the promiscuous setting is not passed down to the driver. We are not sure if this is expected or not and it has not been addressed yet, but this explains why promiscuous has to be set directly on the device.