Upstreaming virtio-wayland (or an alternative)

FD <-> VFD mappings would have to be created
by the subsystem in charge of the object backing the FD (virtio-gpu for exported GEM buffers, virtio-vdec for video buffers, vsock for unix sockets if we decide to bridge unix and vsock sockets to make it transparent, ...). The FD <-> VFD mapping would also have to be created on the host side, probably by the virtio device implementation (virglrenderer for GEM bufs for instance), which means host and guest need a way to inform the other end that a new FD <-> VFD mapping has been created so the other end can create a similar mapping (I guess this requires extra device-specific commands to work).

Quoted 10 lines

FD <-> VFD mappings would have to be created
by the subsystem in charge of the object backing the FD (virtio-gpu for exported GEM buffers, virtio-vdec for video buffers, vsock for unix sockets if we decide to bridge unix and vsock sockets to make it transparent, ...). The FD <-> VFD mapping would also have to be created on the host side, probably by the virtio device implementation (virglrenderer for GEM bufs for instance), which means host and guest need a way to inform the other end that a new FD <-> VFD mapping has been created so the other end can create a similar mapping (I guess this requires extra device-specific commands to work).

My recent proposal for cross device resource sharing seems like it could be relevant here: https://markmail.org/thread/jsaoqy7phrqdcpqu.

Quoted 11 lines

Quoted 10 lines

FD <-> VFD mappings would have to be created
by the subsystem in charge of the object backing the FD (virtio-gpu for exported GEM buffers, virtio-vdec for video buffers, vsock for unix sockets if we decide to bridge unix and vsock sockets to make it transparent, ...). The FD <-> VFD mapping would also have to be created on the host side, probably by the virtio device implementation (virglrenderer for GEM bufs for instance), which means host and guest need a way to inform the other end that a new FD <-> VFD mapping has been created so the other end can create a similar mapping (I guess this requires extra device-specific commands to work).

Quoted 2 lines

My recent proposal for cross device resource sharing seems like it could be relevant here: https://markmail.org/thread/jsaoqy7phrqdcpqu.

Thanks for sharing this link. I had a quick look at this proposal, and, maybe I'm wrong, but I'm not sure it actually addresses Tomasz' concern
[1] if we keep letting a userspace proxy do the FD <-> UUID conversion and sending the UUID through the VSOCK. To me, a UUID only guarantees that 2 buffers will get different UUIDs (assuming they use the same algorithm to generate this UUID), but nothing prevents a malicious app from opening a connection to the host proxy and sending valid wayland messages with forged UUIDs, in the hope that one of them will match an already exported resource.

#1 might require extra care if we want to make it safe, as pointed out by Stefan here [4] (but I wonder if the problem is not the same for a virtio-wayland based solution). Of course you also need a bit of infrastructure to register FD <-> VFD mappings (VFD being a virtual file descriptor that's only used as unique IDs identifying the resource backed by the local FD). FD <-> VFD mappings would have to be created by the subsystem in charge of the object backing the FD (virtio-gpu for exported GEM buffers, virtio-vdec for video buffers, vsock for unix sockets if we decide to bridge unix and vsock sockets to make it transparent, ...). The FD <-> VFD mapping would also have to be created on the host side, probably by the virtio device implementation (virglrenderer for GEM bufs for instance), which means host and guest

need a way to inform the other end that a new FD <-> VFD mapping has been created so the other end can create a similar mapping (I guess this requires extra device-specific commands to work). Note that this solution doesn't look so different from the virtio-dmabuf [5] approach proposed by Gerd a few months back, it's just extended to be a global VFD <-> FD registry instead of a dmabuf <-> unique-handle one. One great thing about this approach is that we can re-use it for any kind of FD sharing, not just dmabufs.

#2 is a bit challenging, since it requires the proxy to know about all possible kind of FDs and do a FD <-> unique handle conversion with some help from the subsystem backing the FD. For dmabufs, that means we need to know who created the dmabuf, or assume that only one device is used for all allocations (virtio-gpu?).

#3 is pretty similar to #1 in its design except that, instead of using the VSOCK infrastructure it's using a new type of virtio device. I guess it has the same pros and cons #1 has, and the name should probably be changed to reflect the fact that it can transmit any kind of data not just wayland.

Quoted 5 lines

#3 is pretty similar to #1 in its design except that, instead of using the VSOCK infrastructure it's using a new type of virtio device. I guess it has the same pros and cons #1 has, and the name should probably be changed to reflect the fact that it can transmit any kind of data not just wayland.

Even though vsock looks simple at first it isn't when you look at the details. You'll want support more streams than virtqueues. So you'll go multiplex. You want good performance, but you also don't want allow streams to DoS the device by filling up the queue.

Thats why I don't like the new virtio device idea much and would prefer vhost being reused, either directly (#1) or via proxy (#2).

Note that vhost aims to be hypervisor-agnostic and we have (unless I missed something) three transports for it: virtio, vmware and hyperv. So extending that with virtio-only features might not be the best idea.

Also it is a fact that approach #1 didn't went anywhere so far but we have a working implementation of approach #3. So I guess I wouldn't veto approach #3 if you pick it after evaluating the options on the table.

Final note: We have a new kid on the block: virtio-fs. I think virtio-fs with dax enabled should allow for shared file mappings between host and guest. That is something a proxy (#2) might be able to make use of.

Hi,

Quoted 12 lines

#1 might require extra care if we want to make it safe, as pointed out by Stefan here [4] (but I wonder if the problem is not the same for a virtio-wayland based solution). Of course you also need a bit of infrastructure to register FD <-> VFD mappings (VFD being a virtual file descriptor that's only used as unique IDs identifying the resource backed by the local FD). FD <-> VFD mappings would have to be created by the subsystem in charge of the object backing the FD (virtio-gpu for exported GEM buffers, virtio-vdec for video buffers, vsock for unix sockets if we decide to bridge unix and vsock sockets to make it transparent, ...). The FD <-> VFD mapping would also have to be created on the host side, probably by the virtio device implementation (virglrenderer for GEM bufs for instance), which means host and guest

Quoted 8 lines

need a way to inform the other end that a new FD <-> VFD mapping has been created so the other end can create a similar mapping (I guess this requires extra device-specific commands to work). Note that this solution doesn't look so different from the virtio-dmabuf [5] approach proposed by Gerd a few months back, it's just extended to be a global VFD <-> FD registry instead of a dmabuf <-> unique-handle one. One great thing about this approach is that we can re-use it for any kind of FD sharing, not just dmabufs.

A dedicated is most likely not going to happen. Probably virtio-gpu and possibly other devices too will support attaching a unique handle (a uuid probably) to a dma-buf, then share buffers like this:

(1) guest userspace export virtio-gpu resource as dma-buf
(2) guest kernel generates uuid, sends uuid to the host (or maybe we let the host generate the uuid, not clear yet), attaches it to the (guest) dma-buf.
(3) guest userspace imports the dma-buf into virtio-vdev
(4) virtio-vdec driver finds the uuid attached to the buffer and passes it on to the host,
(5) virtio-vdev device can use the uuid to lookup the buffer on the host, then have the host decoder send the data directly to the host gpu ...

It certainly makes sense to use this for wayland too, so you just pass around the uuid if you want pass around a dma-buf (no matter which of the three solutions).

Quoted 6 lines

#2 is a bit challenging, since it requires the proxy to know about all possible kind of FDs and do a FD <-> unique handle conversion with some help from the subsystem backing the FD. For dmabufs, that means we need to know who created the dmabuf, or assume that only one device is used for all allocations (virtio-gpu?).

See above, the uuid idea should simplify this.

Quoted 6 lines

#3 is pretty similar to #1 in its design except that, instead of using the VSOCK infrastructure it's using a new type of virtio device. I guess it has the same pros and cons #1 has, and the name should probably be changed to reflect the fact that it can transmit any kind of data not just wayland.

Even though vsock looks simple at first it isn't when you look at the details. You'll want support more streams than virtqueues. So you'll go multiplex. You want good performance, but you also don't want allow streams to DoS the device by filling up the queue.

Thats why I don't like the new virtio device idea much and would prefer vhost being reused, either directly (#1) or via proxy (#2).

Note that vhost aims to be hypervisor-agnostic and we have (unless I missed something) three transports for it: virtio, vmware and hyperv. So extending that with virtio-only features might not be the best idea.

Also it is a fact that approach #1 didn't went anywhere so far but we have a working implementation of approach #3. So I guess I wouldn't veto approach #3 if you pick it after evaluating the options on the table.

Final note: We have a new kid on the block: virtio-fs. I think virtio-fs with dax enabled should allow for shared file mappings between host and guest. That is something a proxy (#2) might be able to make use of.

As pointed in my reply to David's email, I'm a bit worried by the security implications of this approach. As long as the dmabuf -> UUID conversion stays in kernel space we should be safe, but if we start allowing a guest proxy (running in userland) to send raw UUIDs on a VSOCK connection, we lose the ability to check if the process is actually allowed to use this resource.

to attach UUIDs to resources. This could be addressed with the virtio-dmabuf infrastructure you proposed, but it looks like David's proposal does not generalize this interface (it's a virtio-gpu-only thing right now).

Quoted 4 lines

Also it is a fact that approach #1 didn't went anywhere so far but we have a working implementation of approach #3. So I guess I wouldn't veto approach #3 if you pick it after evaluating the options on the table.

I'd really like to investigate option #1, unless you have strong reasons to think this is a dead end.

Hi,

Quoted 6 lines

As pointed in my reply to David's email, I'm a bit worried by the security implications of this approach. As long as the dmabuf -> UUID conversion stays in kernel space we should be safe, but if we start allowing a guest proxy (running in userland) to send raw UUIDs on a VSOCK connection, we lose the ability to check if the process is actually allowed to use this resource.

Correct.

The UUID namespace is huge (128bit), so playing guessing games to access resources you don't own isn't exactly trivial.

I would not be concerned
too much.

If you want avoid uuids nevertheless the only way is option #1 (extent vsock) I think.

Quoted 4 lines

to attach UUIDs to resources. This could be addressed with the virtio-dmabuf infrastructure you proposed, but it looks like David's proposal does not generalize this interface (it's a virtio-gpu-only thing right now).

The idea is that virtio-gpu guest driver adds a uuid to any dma-buf it exports. Then other drivers importing the dma-buf can figure what the uuid is. We could also add a ioctl (on the dma-buf fd) to query the uuid.

Quoted 6 lines

Quoted 5 lines

Also it is a fact that approach #1 didn't went anywhere so far but we have a working implementation of approach #3. So I guess I wouldn't veto approach #3 if you pick it after evaluating the options on the table.

Quoted 2 lines

I'd really like to investigate option #1, unless you have strong reasons to think this is a dead end.

Last time I discussed this with Stefan Hajnoczi he didn't like the idea too much. IIRC the main concern was that it would work on specific kinds of file handles only and that vsock would need special support code for every file handle type.

It quite a while ago though, maybe time for a re-evaluation. The limitation to specific file handles wouldn't go away, and likewise the need for special support code. We have a more clear plan for dma-buf support though. Also we got virtio-fs meanwhile and being able to pass virtio-fs file handles over vsock looks like a pretty cool feature to me.

Stefan?

Quoted 85 lines · Boris Brezillon

Hello everyone,

I recently took over Tomeu's task of upstreaming virtio-wayland. After spending quite a bit of time collecting information from his different attempts [1][2] I wanted to sync with all the people that were involved in the previous discussions (if I missed some of them, feel free to add them back).

The goal here is to get a rough idea of the general direction this should take so I can start implementing a PoC and see if it fits everyone's needs.

virtio-wayland [3] started as a solution to pass wayland messages between host and guests so the guest can execute wayland apps whose surface buffers are passed to the wayland compositor running on the host. While this was its primary use case, I've heard it's been used to transport other protocols. And that's not surprising, when looking at the code I noticed it was providing a protocol-agnostic message passing interface between host and guests, similar to what VSOCK provides but with FD passing as an extra feature.

Based on all previous discussions, I could identify 3 different approaches:

1/ Use VSOCK and extend it to support passing (some) FDs 2/ Use a user space VSOCK-based proxy that's in charge of
a/ passing regular messages
b/ passing specific handles to describe objects shared
between host and guest (most focus has been on dmabufs as this is what we really care about for the gfx use case, but other kind of FDs can be emulated through a VSOCK <-> UNIX_SOCK bridging)
3/ Have a dedicated kernel space solution that provides features
exposed by #1 but through a virtio device interface (basically what virtio-wayland does today)

Each of them has its pros and cons, which I'll try to sum-up (please correct me if I'm wrong, and add new things if you think they are missing).

#1 might require extra care if we want to make it safe, as pointed out by Stefan here [4] (but I wonder if the problem is not the same for a virtio-wayland based solution). Of course you also need a bit of infrastructure to register FD <-> VFD mappings (VFD being a virtual file descriptor that's only used as unique IDs identifying the resource backed by the local FD). FD <-> VFD mappings would have to be created by the subsystem in charge of the object backing the FD (virtio-gpu for exported GEM buffers, virtio-vdec for video buffers, vsock for unix sockets if we decide to bridge unix and vsock sockets to make it transparent, ...). The FD <-> VFD mapping would also have to be created on the host side, probably by the virtio device implementation (virglrenderer for GEM bufs for instance), which means host and guest need a way to inform the other end that a new FD <-> VFD mapping has been created so the other end can create a similar mapping (I guess this requires extra device-specific commands to work). Note that this solution doesn't look so different from the virtio-dmabuf [5] approach proposed by Gerd a few months back, it's just extended to be a global VFD <-> FD registry instead of a dmabuf <-> unique-handle one. One great thing about this approach is that we can re-use it for any kind of FD sharing, not just dmabufs.

#2 is a bit challenging, since it requires the proxy to know about all possible kind of FDs and do a FD <-> unique handle conversion with some help from the subsystem backing the FD. For dmabufs, that means we need to know who created the dmabuf, or assume that only one device is used for all allocations (virtio-gpu?). AFAIU, there's also a security issue as one could pass random (but potentially valid) handles to the host proxy (pointed out by Tomasz [6]).

#3 is pretty similar to #1 in its design except that, instead of using the VSOCK infrastructure it's using a new type of virtio device. I guess it has the same pros and cons #1 has, and the name should probably be changed to reflect the fact that it can transmit any kind of data not just wayland.

This is just a high level view of the problem and the solutions proposed by various people over the years. I'm sure I'm missing tons of details and don't realize yet all the complexity behind solution #1, but looking at this summary, I wonder if I should investigate this solution in priority. An alternative could be to rebrand virtio-wayland, but as I said, it's close enough to VSOCK to try to merge the missing features in VSOCK instead. This being said, I'm not yet set on any of those solutions, and the point of this email is to see with all of you which option I should investigate first.

Quoted 7 lines

Note that option #3 is already implemented (would have to be polished for upstream), IIRC option #2 has been partially implemented by Tomeu but I'm not sure it was finished, and option #1 has just been discussed so far [2].

Any feedback/comment is welcome.

One of the key things I feel we need is for the host-side to be aware of what's going on with the life cycle of the buffers in one single place.

Today we are making suboptimal buffer allocations because we have to pessimistically assume that all buffers with the display flag are going to be displayed (even though they might not be used that way). Furthermore, we can't reallocate buffers on the fly to fit changing usage models, for example to go to different overlays. If virglrenderer knew about the usage in real time, it could make more optimal allocation decisions. Today this is one of the main aspects limiting the performance we can get out of our VMs.

Based on all previous discussions, I could identify 3 different approaches:

1/ Use VSOCK and extend it to support passing (some) FDs 2/ Use a user space VSOCK-based proxy that's in charge of
a/ passing regular messages
b/ passing specific handles to describe objects shared
between host and guest (most focus has been on dmabufs as this is what we really care about for the gfx use case, but other kind of FDs can be emulated through a VSOCK <-> UNIX_SOCK bridging)
3/ Have a dedicated kernel space solution that provides features
exposed by #1 but through a virtio device interface (basically what virtio-wayland does today)

Each of them has its pros and cons, which I'll try to sum-up (please correct me if I'm wrong, and add new things if you think they are missing).

#1 might require extra care if we want to make it safe, as pointed out by Stefan here [4] (but I wonder if the problem is not the same for a virtio-wayland based solution). Of course you also need a bit of infrastructure to register FD <-> VFD mappings (VFD being a virtual file descriptor that's only used as unique IDs identifying the resource backed by the local FD). FD <-> VFD mappings would have to be created by the subsystem in charge of the object backing the FD (virtio-gpu for exported GEM buffers, virtio-vdec for video buffers, vsock for unix sockets if we decide to bridge unix and vsock sockets to make it transparent, ...). The FD <-> VFD mapping would also have to be created on the host side, probably by the virtio device implementation (virglrenderer for GEM bufs for instance), which means host and guest need a way to inform the other end that a new FD <-> VFD mapping has been created so the other end can create a similar mapping (I guess this requires extra device-specific commands to work).

Quoted 2 lines

Thats why I don't like the new virtio device idea much and would prefer vhost being reused, either directly (#1) or via proxy (#2).

For crosvm's purposes, we are looking at ways to reduce vhost usage in order to reduce host kernel exposure to untrusted guest input, including from the guest kernel. That is why a non-vhost based solution would be prefered.

Quoted 3 lines

Any potential security hole should be considered IMHO, even if it's not easily exploitable.

I second Boris's statement. In the age of Spectre-like exploits, keeping secrets from other processes should be considered riskier than using FDs to prove that a process can access a resource.

Quoted 5 lines

We might be able to do that lazily if we add ->virtio_fd_to_vfd()/->virtio_vfd_to_fd() hooks to the file_operations struct, instead of expecting the subsystem to pro-actively create those mappings (when those hooks are not implemented, that means the resource backed by this FD can't be passed on a VSOCK).

virtio-wayland actually will handle passing arbitrary FDs back and forth, but it does have special handling for virtio-gpu dmabufs. For everything else, virtio-wayland will determine if the FD is readable/writable/mappable and will recreate a pipe/memfd on the other side to emulate the FD. For mappable FDs, this is limited host->guest passing because in practice this is used only by the compositor to send the keyboard mapping to clients. The pipes are needed for both host->guest and guest->host because they are used to request clipboard content. The only FDs that require special handling are ones where there already is a host-side FD corresponding to the guest side FD, such as the case of wayland socket connections or dmabufs.

As for the question of which solution to go for, I would suggest either 3 (dedicated virtio device) or combining features from virtio-wayland with virtio-gpu. I don't think extending vsock is a good idea because there's enough going on with virtio-wayland with respect to FD passing and emulation that a generic interface will be too opinionated.

I like the combination with virtio-gpu because it
already has two related but separate graphics concepts combined into one device (display and rendering), so wayland would feel right at home.

Because wayland benefits from allocation and sharing of virtio-gpu buffers, a virtio-gpu combo device simplifies access to those buffers, whereas the separate virtio devices as implemented in crosvm requires bridging of resource handles (in guest kernel) and FDs (in crosvm).

Quoted 8 lines

Quoted 4 lines

<snip> I don't think extending vsock is a
good idea because there's enough going on with virtio-wayland with respect to FD passing and emulation that a generic interface will be too opinionated.

Can you be more specific? What would be the limitations of such a generic interface that would make it unusable for the virtio-wayland use case?

What I meant by that statement was that virtio-wayland implements juuuust enough FD semantics to make wayland work. It's true that some other protocols also work over virtio-wayland, but that was a combination of filling in the holes for that protocol and coincidence. I'm afraid that a generic interface would have to contend with a long tail of FD semantics that don't line up with most other protocols. To give you a specific example, when virtio-wayland encounters an FD in the guest->host direction that it doesn't recognize, but that can be written or read to, it will create a unix pipe on the host side and send it along the wayland connection. This choice works fine for wayland, but that's because that was the entire point of the design. A more generic system would have to realize that interaction more authentically (e.g. for sending a bi-directional socket),

and I'm not
sure it's going to be possible.

Quoted 11 lines

Quoted 5 lines

I like the combination with virtio-gpu because it
already has two related but separate graphics concepts combined into one device (display and rendering), so wayland would feel right at home.

Except that, looking at the virtio-wayland code nothing in there looks fundamentally wayland specific, or even display-server specific. Sure, FD passing support is focused on what wayland needs (pipes, dmabuf FDs, ...), but I see nothing that would prevent making it generic enough to pass other kind of resources if we ever need to.

That was intentional, and you have a good point. The reason I was suggesting integration with virtio-gpu was because I saw that as the most viable path towards upstreaming. It seems that creating a dedicated virtio device is highly contentious.

Quoted 10 lines

I must admit I'm not a huge fan of integrating a protocol-agnostic message passing interface (with FD-passing support) to virtio-gpu. Sounds like something that could be re-used without virtio-gpu being involved to me. As for the resource bridging thing, that's more or less what I proposed with the generic interface to translate resource handles (what I called VFDs) to local FDs. It would probably be easier with a vhost-based solution, but your resource bridge interface in crosvm seems to provide the same kind of abstraction. Any reason you want to avoid this split?

If the upstreamed version of this actually becomes agnostic, than keeping it separate from virtio-gpu would make the most sense, but that seems to be unsettled.

Quoted 7 lines

Quoted 2 lines

Thats why I don't like the new virtio device idea much and would prefer vhost being reused, either directly (#1) or via proxy (#2).

For crosvm's purposes, we are looking at ways to reduce vhost usage in order to reduce host kernel exposure to untrusted guest input, including from the guest kernel. That is why a non-vhost based solution would be prefered.

Okay, I didn't know you were avoiding vhost-based solutions to reduce the attack surface.

Quoted 8 lines · On Mon, 17 Feb 2020 19:21:50 +0100 Boris Brezillon…

Quoted 7 lines

Quoted 2 lines

Thats why I don't like the new virtio device idea much and would prefer vhost being reused, either directly (#1) or via proxy (#2).

For crosvm's purposes, we are looking at ways to reduce vhost usage in order to reduce host kernel exposure to untrusted guest input, including from the guest kernel. That is why a non-vhost based solution would be prefered.

Quoted 2 lines

Okay, I didn't know you were avoiding vhost-based solutions to reduce the attack surface.

Looks like they implemented vhost-less vsock in Firecracker[1]. Not sure how much work that would be to port this implementation to crosvm, but maybe that's an option.

[1]https://github.com/firecracker-microvm/firecracker/pull/1176

Quoted 11 lines · Boris Brezillon

Quoted 8 lines · On Mon, 17 Feb 2020 19:21:50 +0100 Boris Brezillon…

Quoted 7 lines

Quoted 2 lines

Thats why I don't like the new virtio device idea much and would prefer vhost being reused, either directly (#1) or via proxy (#2).

For crosvm's purposes, we are looking at ways to reduce vhost usage in order to reduce host kernel exposure to untrusted guest input, including from the guest kernel. That is why a non-vhost based solution would be prefered.

Quoted 8 lines

Quoted 2 lines

Okay, I didn't know you were avoiding vhost-based solutions to reduce the attack surface.

Looks like they implemented vhost-less vsock in Firecracker[1]. Not sure how much work that would be to port this implementation to crosvm, but maybe that's an option.

[1]https://github.com/firecracker-microvm/firecracker/pull/1176

Well, the nice thing about vsock is that (a) you have the same interface on guest and host, and (b) it is hypervisor-agnostic. Applications can simply use the usual system calls (socket, bind, listen, connect) with AF_VSOCK socket type and be done with it.

When not using vhost this goes away (on the host side). firecracker seems to have a vsock <-> unix socket bridge in vmm userspace instead (pull req msg sounds like that, didn't check the code).

I think there can be only one vsock device per guest. So you can't have multiple instances, one with a vsock backend on the host and one with a userspace backend on the host, then pick one of the two depending on the use case.

So, one advantage of a separate device would be that we'll gain some flexibility in terms of the host side implementation. vsock vhost can be used together with a wayland userspace backend.

Quoted 21 lines · On Mon, 24 Feb 2020 13:12:55 +0100 Gerd Hoffmann <[email protected]> wrote:

Quoted 11 lines · Boris Brezillon

Quoted 8 lines · On Mon, 17 Feb 2020 19:21:50 +0100 Boris Brezillon…

Quoted 7 lines

Quoted 2 lines

Thats why I don't like the new virtio device idea much and would prefer vhost being reused, either directly (#1) or via proxy (#2).

For crosvm's purposes, we are looking at ways to reduce vhost usage in order to reduce host kernel exposure to untrusted guest input, including from the guest kernel. That is why a non-vhost based solution would be prefered.

Quoted 8 lines

Quoted 2 lines

Okay, I didn't know you were avoiding vhost-based solutions to reduce the attack surface.

Looks like they implemented vhost-less vsock in Firecracker[1]. Not sure how much work that would be to port this implementation to crosvm, but maybe that's an option.

[1]https://github.com/firecracker-microvm/firecracker/pull/1176

Quoted 8 lines

Well, the nice thing about vsock is that (a) you have the same interface on guest and host, and (b) it is hypervisor-agnostic. Applications can simply use the usual system calls (socket, bind, listen, connect) with AF_VSOCK socket type and be done with it.

When not using vhost this goes away (on the host side). firecracker seems to have a vsock <-> unix socket bridge in vmm userspace instead (pull req msg sounds like that, didn't check the code).

That's also my understanding. Note that you still keep the genericity on the guest side, which is a good thing I guess.

Quoted 9 lines

I think there can be only one vsock device per guest. So you can't have multiple instances, one with a vsock backend on the host and one with a userspace backend on the host, then pick one of the two depending on the use case.

So, one advantage of a separate device would be that we'll gain some flexibility in terms of the host side implementation. vsock vhost can be used together with a wayland userspace backend.

Yes, though if you really want to avoid vhost to reduce the attack surface, you'll probably want to never use vhost instead of having this restriction on a per-use-case basis.

But let's say we go for a dedicated virtio-device to preserve this granularity. Should we aim at providing a generic virtio-msg device or should we keep this so-called wayland-specific virtio device (I'd like to remind you that it's actually protocol-agnostic)?

Quoted 4 lines

But let's say we go for a dedicated virtio-device to preserve this granularity. Should we aim at providing a generic virtio-msg device or should we keep this so-called wayland-specific virtio device (I'd like to remind you that it's actually protocol-agnostic)?

Maybe there's another option (not sure I mentioned it previously):
de-correlate the message passing and fd-passing interfaces. If we add a virtio-fd device that takes care of FD passing between guest and host, we can still use a regular vsock to pass messages (can be vhost or !vhost depending on your level of paranoïa). The wayland proxies would then queue/dequeue messages to/from the vsock and FDs to/from virtio-fd.

Hi,

Quoted 11 lines

Quoted 4 lines

But let's say we go for a dedicated virtio-device to preserve this granularity. Should we aim at providing a generic virtio-msg device or should we keep this so-called wayland-specific virtio device (I'd like to remind you that it's actually protocol-agnostic)?

Maybe there's another option (not sure I mentioned it previously):
de-correlate the message passing and fd-passing interfaces. If we add a virtio-fd device that takes care of FD passing between guest and host, we can still use a regular vsock to pass messages (can be vhost or !vhost depending on your level of paranoïa). The wayland proxies would then queue/dequeue messages to/from the vsock and FDs to/from virtio-fd.

Hmm, I suspect supporting unix socket passing (by creating a tunnel for them) is going to be tricky in with that approach.

Quoted 14 lines · On Mon, 24 Feb 2020 14:42:31 +0100 Gerd Hoffmann <[email protected]> wrote:

Hi,

Quoted 11 lines

Quoted 4 lines

But let's say we go for a dedicated virtio-device to preserve this granularity. Should we aim at providing a generic virtio-msg device or should we keep this so-called wayland-specific virtio device (I'd like to remind you that it's actually protocol-agnostic)?

Maybe there's another option (not sure I mentioned it previously):
de-correlate the message passing and fd-passing interfaces. If we add a virtio-fd device that takes care of FD passing between guest and host, we can still use a regular vsock to pass messages (can be vhost or !vhost depending on your level of paranoïa). The wayland proxies would then queue/dequeue messages to/from the vsock and FDs to/from virtio-fd.

Quoted 2 lines

Hmm, I suspect supporting unix socket passing (by creating a tunnel for them) is going to be tricky in with that approach.

Yes, If you want a generic vsock <-> unix proxy, you'll have to encapsulate the 'message+number of FDs passed' information so the other end knows how many FDs should be dequeued on the virtio-fd dev. The other option being to implement proxies that can parse the messages and extract the 'number of FDs' from there.

But let's say we go for a dedicated virtio-device to preserve this granularity. Should we aim at providing a generic virtio-msg device or should we keep this so-called wayland-specific virtio device (I'd like to remind you that it's actually protocol-agnostic)?

Quoted 5 lines

Because wayland benefits from allocation and sharing of virtio-gpu buffers, a virtio-gpu combo device simplifies access to those buffers, whereas the separate virtio devices as implemented in crosvm requires bridging of resource handles (in guest kernel) and FDs (in crosvm).

I must admit I'm not a huge fan of integrating a protocol-agnostic message passing interface (with FD-passing support) to virtio-gpu. Sounds like something that could be re-used without virtio-gpu being involved to me.

As for the resource bridging thing, that's more or less
what I proposed with the generic interface to translate resource handles (what I called VFDs) to local FDs.

Hello everyone,

I recently took over Tomeu's task of upstreaming virtio-wayland. After spending quite a bit of time collecting information from his different attempts [1][2] I wanted to sync with all the people that were involved in the previous discussions (if I missed some of them, feel free to add them back).

The goal here is to get a rough idea of the general direction this should take so I can start implementing a PoC and see if it fits everyone's needs.

virtio-wayland [3] started as a solution to pass wayland messages between host and guests so the guest can execute wayland apps whose surface buffers are passed to the wayland compositor running on the host. While this was its primary use case, I've heard it's been used to transport other protocols. And that's not surprising, when looking at the code I noticed it was providing a protocol-agnostic message passing interface between host and guests, similar to what VSOCK provides but with FD passing as an extra feature.

Based on all previous discussions, I could identify 3 different approaches:

1/ Use VSOCK and extend it to support passing (some) FDs 2/ Use a user space VSOCK-based proxy that's in charge of
a/ passing regular messages
b/ passing specific handles to describe objects shared
between host and guest (most focus has been on dmabufs as this is what we really care about for the gfx use case, but other kind of FDs can be emulated through a VSOCK <-> UNIX_SOCK bridging)
3/ Have a dedicated kernel space solution that provides features
exposed by #1 but through a virtio device interface (basically what virtio-wayland does today)

Each of them has its pros and cons, which I'll try to sum-up (please correct me if I'm wrong, and add new things if you think they are missing).

#1 might require extra care if we want to make it safe, as pointed out by Stefan here [4] (but I wonder if the problem is not the same for a virtio-wayland based solution). Of course you also need a bit of infrastructure to register FD <-> VFD mappings (VFD being a virtual file descriptor that's only used as unique IDs identifying the resource backed by the local FD). FD <-> VFD mappings would have to be created by the subsystem in charge of the object backing the FD (virtio-gpu for exported GEM buffers, virtio-vdec for video buffers, vsock for unix sockets if we decide to bridge unix and vsock sockets to make it transparent, ...). The FD <-> VFD mapping would also have to be created on the host side, probably by the virtio device implementation (virglrenderer for GEM bufs for instance), which means host and guest need a way to inform the other end that a new FD <-> VFD mapping has been created so the other end can create a similar mapping (I guess this requires extra device-specific commands to work). Note that this solution doesn't look so different from the virtio-dmabuf [5] approach proposed by Gerd a few months back, it's just extended to be a global VFD <-> FD registry instead of a dmabuf <-> unique-handle one. One great thing about this approach is that we can re-use it for any kind of FD sharing, not just dmabufs.

#2 is a bit challenging, since it requires the proxy to know about all possible kind of FDs and do a FD <-> unique handle conversion with some help from the subsystem backing the FD. For dmabufs, that means we need to know who created the dmabuf, or assume that only one device is used for all allocations (virtio-gpu?). AFAIU, there's also a security issue as one could pass random (but potentially valid) handles to the host proxy (pointed out by Tomasz [6]).

#3 is pretty similar to #1 in its design except that, instead of using the VSOCK infrastructure it's using a new type of virtio device. I guess it has the same pros and cons #1 has, and the name should probably be changed to reflect the fact that it can transmit any kind of data not just wayland.

This is just a high level view of the problem and the solutions proposed by various people over the years. I'm sure I'm missing tons of details and don't realize yet all the complexity behind solution #1, but looking at this summary, I wonder if I should investigate this solution in priority. An alternative could be to rebrand virtio-wayland, but as I said, it's close enough to VSOCK to try to merge the missing features in VSOCK instead. This being said, I'm not yet set on any of those solutions, and the point of this email is to see with all of you which option I should investigate first.

Note that option #3 is already implemented (would have to be polished for upstream), IIRC option #2 has been partially implemented by Tomeu but I'm not sure it was finished, and option #1 has just been discussed so far [2].

Hi all,

Quoted 36 lines · On Fri, 7 Feb 2020 18:28:42 +0100 Boris Brezillon…

Hello everyone,

I recently took over Tomeu's task of upstreaming virtio-wayland. After spending quite a bit of time collecting information from his different attempts [1][2] I wanted to sync with all the people that were involved in the previous discussions (if I missed some of them, feel free to add them back).

The goal here is to get a rough idea of the general direction this should take so I can start implementing a PoC and see if it fits everyone's needs.

virtio-wayland [3] started as a solution to pass wayland messages between host and guests so the guest can execute wayland apps whose surface buffers are passed to the wayland compositor running on the host. While this was its primary use case, I've heard it's been used to transport other protocols. And that's not surprising, when looking at the code I noticed it was providing a protocol-agnostic message passing interface between host and guests, similar to what VSOCK provides but with FD passing as an extra feature.

Based on all previous discussions, I could identify 3 different approaches:

1/ Use VSOCK and extend it to support passing (some) FDs 2/ Use a user space VSOCK-based proxy that's in charge of
a/ passing regular messages
b/ passing specific handles to describe objects shared
between host and guest (most focus has been on dmabufs as this is what we really care about for the gfx use case, but other kind of FDs can be emulated through a VSOCK <-> UNIX_SOCK bridging)
3/ Have a dedicated kernel space solution that provides features
exposed by #1 but through a virtio device interface (basically what virtio-wayland does today)

Option #1 sounds like a good idea on
the paper, but the fact that Google wants a solution that does not involve vhost and given Gerd's concern about being able to use vsock in some cases and do !vsock message-passing in others, it might not be possible to use vsock (there can only be one vsock implementation active).

That leaves us with option #3: have a dedicated virtio device providing a message+FD passing interface.

So, I'm about to start working on virtio-pipe (I realize the name is not that great since pipes are normally unidirectional, but I'm sure we'll have plenty of time to bikeshed on that particular aspect once the other bits are sorted out :)). This device would be a singleton (there can only be one per VM), and would provide the following features:

* allow you to connect to named/public pipes (similar to named unix sockets). That's an easy way to expose host services to guests, and AFAICT, that's not so far from the virtio-wayland way of doing things (VIRTWL_IOCTL_NEW_CTX_NAMED).
* manage a central UUID <-> 'struct file' map that allows virtio-pipe to convert FDs to UUIDs, pass UUIDs through a pipe and convert those UUIDs back to FDs on the other end
- we need to expose an API to let each subsystem register/unregister

    their UUID <-> FD mapping (subsystems are responsible for the UUID
    creation/negotiation)

* allow you to create private/unnamed pipes whose FDs can be passed to the other end (that implies creating a UUID <-> FD mapping for each unnamed pipe). This feature is here to allow unamed unix-socket <-> virtio-pipe bridging. AFAICT, that's what VIRTWL_IOCTL_NEW_CTX is providing. Looks like virtio-wayland also provides support for actual pipes (uni-directional msg-passing) with its VIRTWL_IOCTL_NEW_PIPE_* ioctls, which we can always support at some point if we see the need.

Once we have those basic building blocks in place, we can start patching virtio-gpu to create a UUID <-> FD mapping when a buffer is exported as a dmabuf (using the virtio-pipe public API). And this can easily be applied to any kind of FDs we want to share.

Note that this solution could be extended to support FD passing on top of VSOCK if we ever need/want to. We'd only have to move the UUID <-> FD map/API out of virtio-pipe (virtio-fd?) and let virtio-vsock use it.

Gerd, Stefan, Zach, what do you think? Should I start looking into that, maybe work on a PoC and post it as an RFC, or do you see any problem making this proposal not possible/irrelevant.

Any feedback is welcome!

Thanks,

Boris

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

Quoted 41 lines · Boris Brezillon

Hi all,

Quoted 36 lines · On Fri, 7 Feb 2020 18:28:42 +0100 Boris Brezillon…

Hello everyone,

I recently took over Tomeu's task of upstreaming virtio-wayland. After spending quite a bit of time collecting information from his different attempts [1][2] I wanted to sync with all the people that were involved in the previous discussions (if I missed some of them, feel free to add them back).

The goal here is to get a rough idea of the general direction this should take so I can start implementing a PoC and see if it fits everyone's needs.

virtio-wayland [3] started as a solution to pass wayland messages between host and guests so the guest can execute wayland apps whose surface buffers are passed to the wayland compositor running on the host. While this was its primary use case, I've heard it's been used to transport other protocols. And that's not surprising, when looking at the code I noticed it was providing a protocol-agnostic message passing interface between host and guests, similar to what VSOCK provides but with FD passing as an extra feature.

Based on all previous discussions, I could identify 3 different approaches:

1/ Use VSOCK and extend it to support passing (some) FDs 2/ Use a user space VSOCK-based proxy that's in charge of
a/ passing regular messages
b/ passing specific handles to describe objects shared
between host and guest (most focus has been on dmabufs as this is what we really care about for the gfx use case, but other kind of FDs can be emulated through a VSOCK <-> UNIX_SOCK bridging)
3/ Have a dedicated kernel space solution that provides features
exposed by #1 but through a virtio device interface (basically what virtio-wayland does today)

<snip>

Quoted 6 lines

Option #1 sounds like a good idea on
the paper, but the fact that Google wants a solution that does not involve vhost and given Gerd's concern about being able to use vsock in some cases and do !vsock message-passing in others, it might not be possible to use vsock (there can only be one vsock implementation active).

Forgive me for jumping in but what is the concern about vhost here? Is it purely the increased attack surface to the kernel or some more fundamental issue? Is this something that can be addressed with a vhost-user backend where all the complications of the host end are dealt with in user-space?

So, I'm about to start working on virtio-pipe (I realize the name is not that great since pipes are normally unidirectional, but I'm sure we'll have plenty of time to bikeshed on that particular aspect once the other bits are sorted out :)).

This device would be a singleton
(there can only be one per VM),

* manage a central UUID <-> 'struct file' map that allows virtio-pipe to convert FDs to UUIDs, pass UUIDs through a pipe and convert those UUIDs back to FDs on the other end
- we need to expose an API to let each subsystem register/unregister

    their UUID <-> FD mapping (subsystems are responsible for the UUID
    creation/negotiation)

Hi,

Quoted 4 lines

So, I'm about to start working on virtio-pipe (I realize the name is not that great since pipes are normally unidirectional, but I'm sure we'll have plenty of time to bikeshed on that particular aspect once the other bits are sorted out :)).

virtio-ipc?

Quoted 2 lines

This device would be a singleton
(there can only be one per VM),

Why?

Alex already mentioned vhost-user. This is basically a way to emulate virtio devices outside qemu (not sure whenever other vmms support that too). With vhost emulation is done in the kernel, with vhost-user emulation is done in another process instead. Communication with qemu runs over a socked with messages roughly equivalent to vhost ioctls.

There is a vhost-user implementation of virtio-gpu (see contrib/vhost-user-gpu/ in qemu git repo). Main motivation is that a separate process can have a much stricter sandbox (no need to have access to network or guest disk images), which is especially useful for a complex thing like 3D rendering.

So one possible approach for the host side implementation would be to write the virtio protocol parser as support library, then implement the host applications (i.e. the host wayland proxy) as vhost-user process using that library.

That would not work well with the singleton device approach though.

A vhost-user approach would allow for multiple vmms sharing the implementation. It would allow to easily pick a language other than C (mostly relevant for qemu, crosvm isn't C anyway ...).

Quoted 6 lines

* manage a central UUID <-> 'struct file' map that allows virtio-pipe to convert FDs to UUIDs, pass UUIDs through a pipe and convert those UUIDs back to FDs on the other end
- we need to expose an API to let each subsystem register/unregister

    their UUID <-> FD mapping (subsystems are responsible for the UUID
    creation/negotiation)  

Do you have a rough plan how to do that?
On the guest side?
On the host side?

Quoted 6 lines

So one possible approach for the host side implementation would be to write the virtio protocol parser as support library, then implement the host applications (i.e. the host wayland proxy) as vhost-user process using that library.

That would not work well with the singleton device approach though.

Hm, so you'd have several virtio-ipc devices exposed to the guest and the guest user would have to select one,

or would the guest only see one virtio-ipc device? When I said singleton it was from the guest perspective (though I thought this would also apply to the host).

guest side prime export + FD transfer on an IPC connection:
-----------------------------------------------------------

* guest_virtio_gpu->prime_handle_to_fd()
+ virtio_gpu_resource_to_uuid() request sent on link3 to get a UUID
- virglrenderer_get_resource_uuid() (the UUID is generated if it doesn't exist yet)
- host_virtio_gpu exports the resource to a pseudo-'struct file' representation ('struct file' only exists kernel side, but maybe we can add the concept of resource on the host user space side and define an interface for this resource object?)

We'll also need resource objs to be exported as plain FDs so we can pass them through a unix socket.

I'm still unsure how FDs coming from a host application can be converted to resource objects (and then UUIDs so they can be passed to the ipc_connection) if they've not been previously created/passed by the guest though.

Quoted 4 lines

I'm still unsure how FDs coming from a host application can be converted to resource objects (and then UUIDs so they can be passed to the ipc_connection) if they've not been previously created/passed by the guest though.

Depends on the kind of fd I think. For unix sockets or virtio-fs files it should be easy. dma-bufs or sysv shmem would be more tricky because we have to map them into the guest address space (if we want support that). Not impossible though, we could use the new shared memory support for that.

* manage a central UUID <-> 'struct file' map that allows virtio-pipe to convert FDs to UUIDs, pass UUIDs through a pipe and convert those UUIDs back to FDs on the other end
- we need to expose an API to let each subsystem register/unregister

    their UUID <-> FD mapping (subsystems are responsible for the UUID
    creation/negotiation)

Quoted 6 lines

* manage a central UUID <-> 'struct file' map that allows virtio-pipe to convert FDs to UUIDs, pass UUIDs through a pipe and convert those UUIDs back to FDs on the other end
- we need to expose an API to let each subsystem register/unregister

    their UUID <-> FD mapping (subsystems are responsible for the UUID
    creation/negotiation)  

Can you provide more detail about the envisioned scope of this framework?

What sort of FDs and subsystems will use the framework?

How
do operations on the guest FD affect the host FD, and vice versa?

Quoted 7 lines · Boris Brezillon

What's not entirely clear is whether we need to support cases where the host create its own resources (without being asked by the guest) and decides to pass them to a virtio-ipc-connection (through the unix <-> virtio-ipc bridge). If that's needed, we need interfaces to let host apps import their resources in the VM (again, those interfaces are likely to be resource-type dependent).

When the wayland compositor sends the keyboard map or requests clipboard content, the compositor (or another client on the host) will create a resource (shm or pipe) and send it over the unix domain socket. Would this be a case of the above situation you mentioned above? It seems like we need to support the host creating resources passed over the virtio-ipc-connection.

Quoted 7 lines · On Thu, 27 Feb 2020 13:20:51 +0900 David Stevens <[email protected]> wrote:

Quoted 6 lines

* manage a central UUID <-> 'struct file' map that allows virtio-pipe to convert FDs to UUIDs, pass UUIDs through a pipe and convert those UUIDs back to FDs on the other end
- we need to expose an API to let each subsystem register/unregister

    their UUID <-> FD mapping (subsystems are responsible for the UUID
    creation/negotiation)

Quoted 2 lines

Can you provide more detail about the envisioned scope of this framework?

The scope is "generic message+FD passing" interface, which is pretty much what virtio-wl provides.

Quoted 2 lines

How
do operations on the guest FD affect the host FD, and vice versa?

Depends what you mean by operations. If we're talking about regular read/write/ioctl/mmap operations on the guest side, it's up to the subsystem/driver to implement the expected behavior.

Quoted 14 lines · Boris Brezillon

Quoted 9 lines · On Thu, 27 Feb 2020 13:20:51 +0900 David Stevens <[email protected]> wrote:

Quoted 6 lines

* manage a central UUID <-> 'struct file' map that allows virtio-pipe to convert FDs to UUIDs, pass UUIDs through a pipe and convert those UUIDs back to FDs on the other end
- we need to expose an API to let each subsystem register/unregister

    their UUID <-> FD mapping (subsystems are responsible for the UUID
    creation/negotiation)  

Can you provide more detail about the envisioned scope of this framework?

Quoted 2 lines

The scope is "generic message+FD passing" interface, which is pretty much what virtio-wl provides.

I think that scope is too broad. A socket is a 'generic message+FD' interface. Unless there's the expectation that the interface should eventually be as flexible as a regular domain socket, I think it would be a good idea to frame the scope of the interface more precisely.

Part of this ambiguity comes from the informal usage of the term 'FD'. An FD is a concept in Linux and other operating systems (and not even all operating systems - e.g. Fuchsia). At present, FDs are not a concept in virtio. Talking about sending FDs over virtio handwaves a lot of details about what that's actually going on.

Quoted 7 lines

Quoted 3 lines

How
do operations on the guest FD affect the host FD, and vice versa?

Depends what you mean by operations. If we're talking about regular read/write/ioctl/mmap operations on the guest side, it's up to the subsystem/driver to implement the expected behavior.

I think part of my confusion comes from the fact that virtio-wayland seems to provide both the IPC mechanism described for virtio-ipc as well as some additional guest/host file sharing support.

If that is
actually the case, then I guess a striped down version of virtio-wayland would still be necessary.

Quoted 5 lines

Quoted 2 lines

Can you provide more detail about the envisioned scope of this framework?

The scope is "generic message+FD passing" interface, which is pretty much what virtio-wl provides.

I think that scope is too broad. A socket is a 'generic message+FD' interface. Unless there's the expectation that the interface should eventually be as flexible as a regular domain socket, I think it would be a good idea to frame the scope of the interface more precisely.

Hi,

Quoted 10 lines

Quoted 5 lines

Quoted 2 lines

Can you provide more detail about the envisioned scope of this framework?

The scope is "generic message+FD passing" interface, which is pretty much what virtio-wl provides.

I think that scope is too broad. A socket is a 'generic message+FD' interface. Unless there's the expectation that the interface should eventually be as flexible as a regular domain socket, I think it would be a good idea to frame the scope of the interface more precisely.

Yes, sure, we need to exactly specify the different kinds of file handles / resources. I think it makes sense to have a virtio feature flag for each of them, so guest+host can easily negotiate what they are able to handle and what not.

Quoted 4 lines

Yes, sure, we need to exactly specify the different kinds of file handles / resources. I think it makes sense to have a virtio feature flag for each of them, so guest+host can easily negotiate what they are able to handle and what not.

I was expecting that to be a feature of the resource producers (virtio-gpu, virtio-fs, ...) rather than a feature of virtio-ipc itself.

Quoted 8 lines

Quoted 4 lines

Yes, sure, we need to exactly specify the different kinds of file handles / resources. I think it makes sense to have a virtio feature flag for each of them, so guest+host can easily negotiate what they are able to handle and what not.

I was expecting that to be a feature of the resource producers (virtio-gpu, virtio-fs, ...) rather than a feature of virtio-ipc itself.

"resources from other virtio devices" would be one virtio-ipc feature flag. And, yes, that would for the most part have the other device handle the problem.

But there also is "unix socket", or maybe a somewhat broader "stream", which would be another feature flag I guess because virtio-ipc would just tunnel the stream without the help from other devices.

Quoted 3 lines

But there also is "unix socket", or maybe a somewhat broader "stream", which would be another feature flag I guess because virtio-ipc would just tunnel the stream without the help from other devices.

Can you elaborate on what you mean by this? I can envision how virtio-ipc would be a generic mechanism for passing data+virtio resources, including any new types of resources it itself defines. However, if "unix sockets" or a generic "stream" expands beyond virtio, that seems too broad, with too many edge cases to be feasible to implement.

Quoted 10 lines · David Stevens

Quoted 3 lines

But there also is "unix socket", or maybe a somewhat broader "stream", which would be another feature flag I guess because virtio-ipc would just tunnel the stream without the help from other devices.

Can you elaborate on what you mean by this? I can envision how virtio-ipc would be a generic mechanism for passing data+virtio resources, including any new types of resources it itself defines. However, if "unix sockets" or a generic "stream" expands beyond virtio, that seems too broad, with too many edge cases to be feasible to implement.

As far I know this is exactly what virtio-wayland does today if you try to pass a unix socket file descriptor to the other side, so I assume this functionality is needed ...

Quoted 13 lines · Gerd Hoffmann

Quoted 10 lines · David Stevens

Quoted 3 lines

But there also is "unix socket", or maybe a somewhat broader "stream", which would be another feature flag I guess because virtio-ipc would just tunnel the stream without the help from other devices.

Can you elaborate on what you mean by this? I can envision how virtio-ipc would be a generic mechanism for passing data+virtio resources, including any new types of resources it itself defines. However, if "unix sockets" or a generic "stream" expands beyond virtio, that seems too broad, with too many edge cases to be feasible to implement.

Quoted 3 lines

As far I know this is exactly what virtio-wayland does today if you try to pass a unix socket file descriptor to the other side, so I assume this functionality is needed ...

As Zach said earlier, virtio-wayland implements just enough FD sharing support to get wayland working. However, there are many other situations where virtio-wayland's unix socket support isn't sufficient. I think adding support for sharing unix sockets to a generic virtio-ipc would imply that the support is more comprehensive than would be feasible to implement. So while virtio-ipc should support wayland, either directly or with the support of userspace/another virtio device, it also shouldn't overpromise what it is capable of doing.

Quoted 9 lines

Quoted 8 lines

Quoted 4 lines

Yes, sure, we need to exactly specify the different kinds of file handles / resources. I think it makes sense to have a virtio feature flag for each of them, so guest+host can easily negotiate what they are able to handle and what not.

I was expecting that to be a feature of the resource producers (virtio-gpu, virtio-fs, ...) rather than a feature of virtio-ipc itself.

Quoted 7 lines

"resources from other virtio devices" would be one virtio-ipc feature flag. And, yes, that would for the most part have the other device handle the problem.

But there also is "unix socket", or maybe a somewhat broader "stream", which would be another feature flag I guess because virtio-ipc would just tunnel the stream without the help from other devices.

Can you elaborate on what you mean by this? I can envision how virtio-ipc would be a generic mechanism for passing data+virtio resources, including any new types of resources it itself defines. However, if "unix sockets" or a generic "stream" expands beyond virtio, that seems too broad, with too many edge cases to be feasible to implement.