net packet storms with multiple NICs
Hello.
I vaguely remember something like this has been reported and/or discussed already, but I can't find anything related. I'm also not sure if it's kvm-specific or exists in qemu too.
I want some clarification wrt vlan= parameter in -net definition.
What started this all is a problem report I had with kvm package. It turns out that the OP had 2 network adaptors defined for one of his guests, and right when his guest started getting an IP address over DHCP for one of them, the network saw huge packet storm consisting of DHCP and ARP packets.
I immediately reproduced the problem locally. It turns out that kvm reflects packets coming from one guest NIC on another guest NIC, and since both are connected to the same bridge we're getting endless packet storm. To a level when kvm process becomes 100% busy and does not respond to anything but `kill -9'.
The problem is easily solvable by specifying explicit different vlan indexes for different pairs of host/guest -net components.
I've two questions:
o what's the intended usage of all-vlan-equal case, when kvm (or qemu) reflects packets from one interface to another? It's what bridge in linux is for, I think.
o why different -net guest -net host pairs are not getting different vlan= indexes by default, to stop the above-mentioned packet storms right away? I think it's a wise default to assign different pairs to different vlans, by counting -net host and -net guest sequences.
Thanks!
/mjt
Quoted 6 lines · Michael Tokarev
o why different -net guest -net host pairs are not getting different vlan= indexes by default, to stop the above-mentioned packet storms right away? I think it's a wise default to assign different pairs to different vlans, by counting -net host and -net guest sequences.
I think the major issue is that the definition of a "pair" is when the vlan-options match. There's no requirement to define the two after each other.
Signature · 1 line
Andreas
Quoted 10 lines · Andreas Plesner Jacobsen
Quoted 5 lines · Michael Tokarev
o why different -net guest -net host pairs are not getting different vlan= indexes by default, to stop the above-mentioned packet storms right away? I think it's a wise default to assign different pairs to different vlans, by counting -net host and -net guest sequences.
I think the major issue is that the definition of a "pair" is when the vlan-options match. There's no requirement to define the two after each other.
I mean something like this (rough approximation:
int host_vlan_no = 0, guest_vlan_no = 0;
while(parse_option()) {
if (option == host-side-net) {
if (!explicit_vlan) vlan = host_vlan_no;
++host_vlan_no;
}
if (option == guest-side-net) {
if (!explicit_vlan) vlan = guest_vlan_no;
++guest_vlan_no;
}}
this way, consecutive host-side -net will receive consecutive vlan=NN, and consecutive guest-side -net also consecutive vlan=NN numbers.
But having in mind Marc's reply I don't think it's necessary anymore.
Thanks!
/mjt
Quoted 5 lines · Michael Tokarev
I've two questions:
o what's the intended usage of all-vlan-equal case, when kvm (or qemu) reflects packets from one interface to another? It's what bridge in linux is for, I think.
I don't think it's necessarily an intended use-case for the vlan feature
Quoted 5 lines
o why different -net guest -net host pairs are not getting different vlan= indexes by default, to stop the above-mentioned packet storms right away? I think it's a wise default to assign different pairs to different vlans, by counting -net host and -net guest sequences.
With 0.12, we're going to be de-emphasising the vlan feature and instead have NICs directly connected to host backends. The vlan feature will be just another host backend, but optional
You'll start guests with e.g.:
-netdev tap,id=tap.0 -device virtio-net-pci,netdev=tap.0
Cheers,
Mark.
Quoted 10 lines · Mark McLoughlin
Quoted 6 lines · Michael Tokarev
I've two questions:
o what's the intended usage of all-vlan-equal case, when kvm (or qemu) reflects packets from one interface to another? It's what bridge in linux is for, I think.
I don't think it's necessarily an intended use-case for the vlan feature
Well, it is. vlan=x really means "the ethernet segment named x". If you connect all your guest nics to one vlan, you are connecting them all to one ethernet segment, so any packet transmitted on one will be reflected on others.
Whether this is a useful feature is another matter, but the code is functioning as expected.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 4 lines
-----Original Message-----
From:
[email protected] [mailto:[email protected]
rg] On Behalf Of Avi Kivity
Quoted 20 lines
Sent: Sunday, October 25, 2009 9:23 AM
To: Mark McLoughlin
Cc: Michael Tokarev; [email protected]; KVM list
Subject: Re: [Qemu-devel] net packet storms with multiple NICsQuoted 5 lines · Mark McLoughlin
Quoted 3 lines · Michael Tokarev
I've two questions:
o what's the intended usage of all-vlan-equal case, when
kvm (or qemu)
Quoted 1 line
Quoted 1 line
reflects packets from one interface to another? It's
what bridge
Quoted 3 lines
Quoted 2 lines
in linux is for, I think.
I don't think it's necessarily an intended use-case for the
vlan feature
Quoted 1 line
Quoted 8 lines
Well, it is. vlan=x really means "the ethernet segment named x". If you connect all your guest nics to one vlan, you are connecting them all
to one ethernet segment, so any packet transmitted on one will be reflected on others.Whether this is a useful feature is another matter, but the code is functioning as expected.
Hello,
We had one environment where the NIC understood by u-boot and the NIC understood by the kernel where different. We just attached both to the same VLAN. During u-boot one was used for downloading the kernel, then once the kernel booted the other was used. Not ideal, and maybe not important enough to keep the "feature" around, but it does get used now and again.
Thanks
Chris
Quoted 7 lines
Signature · 2 lines
-- error compiling committee.c: too many arguments to function
Quoted 10 lines · Chris
Quoted 10 lines
Well, it is. vlan=x really means "the ethernet segment named x". If you connect all your guest nics to one vlan, you are connecting them all
to one ethernet segment, so any packet transmitted on one will be reflected on others.Whether this is a useful feature is another matter, but the code is functioning as expected.
Quoted 9 lines
Hello,
We had one environment where the NIC understood by u-boot and the NIC understood by the kernel where different. We just attached both to the same VLAN. During u-boot one was used for downloading the kernel, then once the kernel booted the other was used. Not ideal, and maybe not important enough to keep the "feature" around, but it does get used now and again.
You could get the same behaviour by using two different vlans connected to the same bridge.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 19 lines · Mark McLoughlin
Quoted 6 lines · Michael Tokarev
I've two questions:
o what's the intended usage of all-vlan-equal case, when kvm (or qemu) reflects packets from one interface to another? It's what bridge in linux is for, I think.
I don't think it's necessarily an intended use-case for the vlan feature
Quoted 6 lines
o why different -net guest -net host pairs are not getting different vlan= indexes by default, to stop the above-mentioned packet storms right away? I think it's a wise default to assign different pairs to different vlans, by counting -net host and -net guest sequences.
Quoted 8 lines
With 0.12, we're going to be de-emphasising the vlan feature and instead have NICs directly connected to host backends. The vlan feature will be just another host backend, but optional
You'll start guests with e.g.:
-netdev tap,id=tap.0 -device virtio-net-pci,netdev=tap.0
Which is not necessarily more friendly to a user, but it lays the foundation to be able to do a config file which in turns could be easily generated from a nic GUI/command line tool.
Regards,
Anthony Liguori
Quoted 5 lines
I immediately reproduced the problem locally. It turns out that kvm reflects packets coming from one guest NIC on another guest NIC, and since both are connected to the same bridge we're getting endless packet storm. To a level when kvm process becomes 100% busy and does not respond to anything but `kill -9'.
You created a network loop. It is working exactly as expected. Create the same topology with a physical network hub and a pair of NICs and you'll get the same end result.
Paul
Quoted 8 lines · Paul Brook
Quoted 5 lines
I immediately reproduced the problem locally. It turns out that kvm reflects packets coming from one guest NIC on another guest NIC, and since both are connected to the same bridge we're getting endless packet storm. To a level when kvm process becomes 100% busy and does not respond to anything but `kill -9'.
You created a network loop. It is working exactly as expected.
Create the same topology with a physical network hub and a pair of NICs and you'll get the same end result.
Paul, I know what the result is. But mind you, I talked about something different, namely: why, using "obvious" way, you get a packet storm/loop instead of a working network. And if that loop is useful in the first place.
I've got answers to both already :)
Thanks
/mjt