Does macvtap support host to guest communication?

Hi, folks

I am trying to use qemu/qemu-kvm with macvtap using following commands:

# ip link add link eth0 name v0 type macvtap mode {vepa,bridge,private}
# ip link set v0 address da:4e:17:88:42:b1 up
# idx=`ip link show v0 | grep mtu| awk -F":" '{print $1}'`
# kvm -net nic,macaddr=da:4e:17:88:42:b1 -net tap,fd=3 -hda
/home/asias/qemu-stuff/sid.img  3<>/dev/tap${idx}

I found that guest can access other hosts on the LAN except the host where guest lives, and host where guest lives can not access guest.

My question is: Does macvtap support host(hypervisor host) to guest communication?

Quoted 11 lines · Asias He

Hi, folks

I am trying to use qemu/qemu-kvm with macvtap using following commands:

# ip link add link eth0 name v0 type macvtap mode {vepa,bridge,private}
# ip link set v0 address da:4e:17:88:42:b1 up
# idx=`ip link show v0 | grep mtu| awk -F":" '{print $1}'`
# kvm -net nic,macaddr=da:4e:17:88:42:b1 -net tap,fd=3 -hda
/home/asias/qemu-stuff/sid.img  3<>/dev/tap${idx}

Quoted 6 lines

I found that guest can access other hosts on the LAN except the host where guest lives, and host where guest lives can not access guest.

My question is: Does macvtap support host(hypervisor host) to guest communication?

You can communicate between macvtap and macvlan devices when they are in bridge mode, but these devices cannot communicate with clients that run on the underlying device.

Just add a macvlan device to your hardware interface and use that in the host instead of running on the low-level device directly.

The other option is to use a vepa enabled bridge, but these are relatively rare.

Arnd

(1) Is it possible to add an interface to macvtap like /dev/net/tun, eg, /dev/net/macvtap. Currently, it is hard to use macvtap programmatically.

(2) Adding another macvlan device(e.g., macvlan0) to the hardware interface(e.g., eth0) and using it as the old eth0 make the process of using macvtap complicate. One has to reconfigure the network. This is not optimal from the user perspective. Is it possible to leave the low-level device as is when using the macvtap device?

Quoted 2 lines · Asias He

(1) Is it possible to add an interface to macvtap like /dev/net/tun, eg, /dev/net/macvtap. Currently, it is hard to use macvtap programmatically.

I decided against having a multiplexor device because it makes permission handling rather hard. One chardev per network interface makes it possible to handle permissions in multiuser setups.

Quoted 5 lines

(2) Adding another macvlan device(e.g., macvlan0) to the hardware interface(e.g., eth0) and using it as the old eth0 make the process of using macvtap complicate. One has to reconfigure the network. This is not optimal from the user perspective. Is it possible to leave the low-level device as is when using the macvtap device?

Only in VEPA mode. Note that a similar restriction applies when using the bridge device, for the same technical reasons.

Quoted 2 lines

Only in VEPA mode. Note that a similar restriction applies when using the bridge device, for the same technical reasons.

Just to sum things up, our goal is to allow the tools/kvm/ unprivileged tool to provide TCP connectivity to Linux guests transparently, with the following parameters:

- the kvm tool runs unprivileged - as ordinary user

- without having to configure much (preferably zero configuration: without having to configure anything) on the guest Linux side

- multiple guests should just work without interfering with each other

- the kvm tool wants to be stateless - i.e. it does not want to allocate or manage host side devices - it just wants to provide the kind of TCP/IP connectivity host unprivileged user-space has, to the guest. The tool wants to be a generic tool with no global state, not a daemon.

So it wants to be a stateless, unprivileged and zero-configuration solution.

Is this possible with macvtap, and if yes, what kind of macvtap mode and usage would you recommend for that goal?

Quoted 19 lines · Ingo Molnar

Quoted 2 lines

Only in VEPA mode. Note that a similar restriction applies when using the bridge device, for the same technical reasons.

Just to sum things up, our goal is to allow the tools/kvm/ unprivileged tool to provide TCP connectivity to Linux guests transparently, with the following parameters:

- the kvm tool runs unprivileged - as ordinary user

- without having to configure much (preferably zero configuration: without having to configure anything) on the guest Linux side

- multiple guests should just work without interfering with each other

- the kvm tool wants to be stateless - i.e. it does not want to allocate or manage host side devices - it just wants to provide the kind of TCP/IP connectivity host unprivileged user-space has, to the guest. The tool wants to be a generic tool with no global state, not a daemon.

Quoted 4 lines

So it wants to be a stateless, unprivileged and zero-configuration solution.

Is this possible with macvtap, and if yes, what kind of macvtap mode and usage would you recommend for that goal?

With the above requirements, I would suggest using something like the the qemu user networking. This is slower and does not allow servers to be present in the guest, but those are not your goal as it seems.

The primary goals of macvtap are to allow efficient networking (zero-copy, multi-queue, although we're not completely there yet) and proper security abstractions.

If you want a guest to appear on the same network as the host, you can not do that without privileges to manage the host network setup, and I guess that will have to stay that way.

Arnd

We do need "guest appearing on the same network as the host" support as well. The reason I am considering using macvatp instead of tap plus brctl is that it simplifies the bridge configuration and it is more efficient.

However, IMHO, the interface of macvtap is not user friendly, at least for me. I have no idea about the technical reasons that make the low-level device inaccessible. But if it is accessible, a lot of configuration can be eliminated. I know virtualbox's bridge mode has this kind of restriction, while VMware's bridge mode does not.

Quoted 4 lines · Asias He

We do need "guest appearing on the same network as the host" support as well. The reason I am considering using macvatp instead of tap plus brctl is that it simplifies the bridge configuration and it is more efficient.

Right, you certainly don't need to consider tap/brctl any more.

Quoted 5 lines

However, IMHO, the interface of macvtap is not user friendly, at least for me. I have no idea about the technical reasons that make the low-level device inaccessible. But if it is accessible, a lot of configuration can be eliminated. I know virtualbox's bridge mode has this kind of restriction, while VMware's bridge mode does not.

The main reason is that having a MAC address scan in the regular networking core would make the common TX case where there is no macvlan device more complex. Macvtap is derived from the plain macvlan driver, which used to support only sending out to the wire until I added the optional bridge mode.

If you want a regular device to be able to send to a macvlan port, that would require at least these changes:

* Add an option to put a plain device into macvlan-bridge mode
* Add support for that option into iproute2
* Add a hook into dev_queue_xmit() to check for macvlan ports

Quoted 7 lines

If you want a regular device to be able to send to a macvlan port, that would require at least these changes:

* Add an option to put a plain device into macvlan-bridge mode
* Add support for that option into iproute2
* Add a hook into dev_queue_xmit() to check for macvlan ports

Cool! Arnd, mind to add this feature to macvtap?

@Arnd, I would like to initiate the dig into into macvtap driver. Could you please provide me some pointers/docs to start with.

Quoted 9 lines · Asias He

Quoted 7 lines

If you want a regular device to be able to send to a macvlan port, that would require at least these changes:

* Add an option to put a plain device into macvlan-bridge mode
* Add support for that option into iproute2
* Add a hook into dev_queue_xmit() to check for macvlan ports

Cool! Arnd, mind to add this feature to macvtap?

No, not after I just explained why I haven't done it before and why it's so controversial.

Also, I have moved on to other projects and am no longer doing active development of the macvtap driver. I'd be happy to pass on the ownership to someone else and help him or her extend it.

Arnd

- the kvm tool wants to be stateless - i.e. it does not want to allocate or manage host side devices - it just wants to provide the kind of TCP/IP connectivity host unprivileged user-space has, to the guest. The tool wants to be a generic tool with no global state, not a daemon.

So it wants to be a stateless, unprivileged and zero-configuration solution.

Is this possible with macvtap, and if yes, what kind of macvtap mode and usage would you recommend for that goal?