Discuss about an new idea &#34;Vsock over Virtio-net&#34;

Quoted 51 lines · jiangyiwen

Quoted 50 lines · Jason Wang
Quoted 9 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
Quoted 6 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(
I think we should have a new feature flag for this. E.g VIRTIO_NET_F_VSOCK. And host should fail the negotiation if guest doesn't support this to avoid confusion. When this feature is negotiated, we will use it only for VOSCK transport. This can simplify things somehow.
Quoted 29 lines
-------------------------Simple idea------------------------------
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+
2. The Guest->Host basic code flow as follow:
                              +------------+
                              |   Client   |
                              +------------+
                                    |
                                    |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|ops->sendmsg; (vsock_stream_sendmsg)                              |
|  -> alloc_skb; /* it will packet a skb buffer, and include vsock |
|                 * hdr and payload */                             |
|  -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */|
|vsock hdr and payload, and then call                              |
+------------------------------------------------------------------+

Quoted 7 lines

Quoted 1 line
Note, if we've negotiated the feature, virtio-net driver must not use register_netdev to register it to network core. This can avoid lots of confusion.
Hi Jason,
You mean we should not register netdev if use vsock, and in order to avoid confusion, then I think whether we should keep vsock and export some virtio-net's functions that can be shared. In this way, first, vsock may keep existing architecture and will not affect virtio-net.

At least it needs new feature bits which will be a functional duplication of virtio-net (e.g mrg_rxbuf).

Quoted 3 lines

In addition,
vsock doesn't need to use virtio_net header too, then it don't need to pack skb structure.

For mergeable rx buffer it need I think?

Quoted 51 lines

Thanks,
Yiwen.
Quoted 34 lines
Quoted 10 lines
                                    |
                                    |
+------------------------------------------------------------------+
|Virtio-net Module                                                 |
|start_xmit                                                        |
|  -> add virtio_net_hdr and pack sg in ring desc, notify Host     |
+------------------------------------------------------------------+
                                    |
                                    |
Quoted 8 lines
+------------------------------------------------------------------+
|Vhost-net Module                                                  |
|handle_tx                                                         |
|  -> get tx buffer, skip virtio_net_hdr and call Vsock function.  |
| /* This point has some differences, vhost-net use ->sendmsg to   |
|  * forward information, however vsock only need to notify server |
|  * that data ready. */                                           |
+------------------------------------------------------------------+
When VIRTIO_NET_F_VOSCK is negotiated, we know that it's a vsock transport, we can then forward it to vsock core.
Quoted 12 lines
                                    |
                                    |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|alloc_pkt, copy skb data to pkt.                                  |
|add pkt to rx_queue and notify server to get data.                |
+------------------------------------------------------------------+
3. To Host->Guest
I have a problem and difficult, mainly I know about virtio-net a little), because I have been doing work related with storage and file system.
Quoted 7 lines
Quoted 6 lines
The problem as follows:
we should monitor all of socket of vsock in handle_rx, when there are data coming, and copy data to vq desc. Vhost-net use ->recvmsg to get data, it is different with socket. To vsock, I think host will not call ->recvmsg when it need to send message to guest. To net, vhost-net only as forwarding layer.
Know not much here, but is it possible to have a vsock(tap) to be passed to vhost_net and let vhost call its recvmgs()? Bascially it was a socket on host as well I believe?
For vsock, Host->Guest, it's code flow as follows:
Server call send()
-> sendmsg(); (vsock_stream_sendmsg)
     -> virtio_trasnport_send_pkt_info
       -> alloc pkt, add pkt to send_pkt_list, wake up vhost_worker

Quoted 8 lines

Vhost_worker
-> vhost_transport_send_pkt_work
     -> get pkt from send_pkt_list
     -> get vq input desc and then fill data to desc addr
     -> update used ring and then signal guest
In the whole process, host don't call recvmsg() because it is a net device, and it also receives any messages.

If I understand this correctly, there's no a socket object on host that represent vosck in guest? If yes, maybe we can invent one.

Thanks

Quoted 18 lines

For vhost-net, I understand it is a tap device, so it can receive messages from other net device.
This is my understanding, it may have some errors.
Thanks.
Quoted 9 lines
If this doesn't work, we can have vsock specific receiving routine in vhost_net if VIRTIO_NET_F_VOSCK is negotiated.
Generally, I think we should try out best to keep the exist sendmsg()/recvmsg() interfaces and only consider the alternatives if we meet some real blocker.
Thanks
.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

jiangyiwen

author

2018-11-15 09:02 UTC #5 lore raw

Quoted 56 lines · Jason Wang

Quoted 54 lines · jiangyiwen
Quoted 50 lines · Jason Wang
Quoted 9 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
Quoted 6 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(
I think we should have a new feature flag for this. E.g VIRTIO_NET_F_VSOCK. And host should fail the negotiation if guest doesn't support this to avoid confusion. When this feature is negotiated, we will use it only for VOSCK transport. This can simplify things somehow.
Quoted 16 lines
-------------------------Simple idea------------------------------
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+
2. The Guest->Host basic code flow as follow:
Quoted 13 lines
                              +------------+
                              |   Client   |
                              +------------+
                                    |
                                    |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|ops->sendmsg; (vsock_stream_sendmsg)                              |
|  -> alloc_skb; /* it will packet a skb buffer, and include vsock |
|                 * hdr and payload */                             |
|  -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */|
|vsock hdr and payload, and then call                              |
+------------------------------------------------------------------+
Quoted 1 line
Note, if we've negotiated the feature, virtio-net driver must not use register_netdev to register it to network core. This can avoid lots of confusion.
Hi Jason,

Quoted 7 lines

Quoted 4 lines
You mean we should not register netdev if use vsock, and in order to avoid confusion, then I think whether we should keep vsock and export some virtio-net's functions that can be shared. In this way, first, vsock may keep existing architecture and will not affect virtio-net.
At least it needs new feature bits which will be a functional duplication of virtio-net (e.g mrg_rxbuf).

Hi Jason,

Actually I mean only use some shared function to make vsock support these features, in that way, guest see the device is still vsock device instead of virtio-net device, in addition, it can have less codes and easier to be compatible with old vsock version.

Thanks,
Yiwen.

Quoted 8 lines

Quoted 5 lines
In addition,
vsock doesn't need to use virtio_net header too, then it don't need to pack skb structure.
For mergeable rx buffer it need I think?

As said above, I will define the related structure in the virtio-vsock module.

Quoted 59 lines

Quoted 59 lines
Thanks,
Yiwen.
Quoted 34 lines
Quoted 10 lines
                                    |
                                    |
+------------------------------------------------------------------+
|Virtio-net Module                                                 |
|start_xmit                                                        |
|  -> add virtio_net_hdr and pack sg in ring desc, notify Host     |
+------------------------------------------------------------------+
                                    |
                                    |
Quoted 8 lines
+------------------------------------------------------------------+
|Vhost-net Module                                                  |
|handle_tx                                                         |
|  -> get tx buffer, skip virtio_net_hdr and call Vsock function.  |
| /* This point has some differences, vhost-net use ->sendmsg to   |
|  * forward information, however vsock only need to notify server |
|  * that data ready. */                                           |
+------------------------------------------------------------------+
When VIRTIO_NET_F_VOSCK is negotiated, we know that it's a vsock transport, we can then forward it to vsock core.
Quoted 12 lines
                                    |
                                    |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|alloc_pkt, copy skb data to pkt.                                  |
|add pkt to rx_queue and notify server to get data.                |
+------------------------------------------------------------------+
3. To Host->Guest
I have a problem and difficult, mainly I know about virtio-net a little), because I have been doing work related with storage and file system.
Quoted 7 lines
Quoted 6 lines
The problem as follows:
we should monitor all of socket of vsock in handle_rx, when there are data coming, and copy data to vq desc. Vhost-net use ->recvmsg to get data, it is different with socket. To vsock, I think host will not call ->recvmsg when it need to send message to guest. To net, vhost-net only as forwarding layer.
Know not much here, but is it possible to have a vsock(tap) to be passed to vhost_net and let vhost call its recvmgs()? Bascially it was a socket on host as well I believe?
For vsock, Host->Guest, it's code flow as follows:
Server call send()
-> sendmsg(); (vsock_stream_sendmsg)
     -> virtio_trasnport_send_pkt_info
       -> alloc pkt, add pkt to send_pkt_list, wake up vhost_worker
Vhost_worker
-> vhost_transport_send_pkt_work
     -> get pkt from send_pkt_list
     -> get vq input desc and then fill data to desc addr
     -> update used ring and then signal guest

Quoted 8 lines

Quoted 2 lines
In the whole process, host don't call recvmsg() because it is a net device, and it also receives any messages.
If I understand this correctly, there's no a socket object on host that represent vosck in guest? If yes, maybe we can invent one.
Thanks

Sorry, I am not understanding you very much, vsock only a socket channel, it does not have a network device entity, so it only transmit the data between server and client, the data is only saved in server and client. Instead of vhost-net, I feel it has a network device that can saved the data, so when host send message to guest, it can use recvmsg() from the network device(tap). For Vsock, recvmsg() interface will read message from tx vq.

Quoted 22 lines

Quoted 19 lines
For vhost-net, I understand it is a tap device, so it can receive messages from other net device.
This is my understanding, it may have some errors.
Thanks.
Quoted 9 lines
If this doesn't work, we can have vsock specific receiving routine in vhost_net if VIRTIO_NET_F_VOSCK is negotiated.
Generally, I think we should try out best to keep the exist sendmsg()/recvmsg() interfaces and only consider the alternatives if we meet some real blocker.
Thanks
.
.

Jason Wang

2018-11-15 09:21 UTC #6 lore raw

Quoted 59 lines · jiangyiwen

Quoted 58 lines · Jason Wang
Quoted 56 lines · jiangyiwen
Quoted 48 lines · Jason Wang
Quoted 9 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
Quoted 6 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(
I think we should have a new feature flag for this. E.g VIRTIO_NET_F_VSOCK. And host should fail the negotiation if guest doesn't support this to avoid confusion. When this feature is negotiated, we will use it only for VOSCK transport. This can simplify things somehow.
Quoted 16 lines
-------------------------Simple idea------------------------------
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+
2. The Guest->Host basic code flow as follow:
Quoted 13 lines
                               +------------+
                               |   Client   |
                               +------------+
                                     |
                                     |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|ops->sendmsg; (vsock_stream_sendmsg)                              |
|  -> alloc_skb; /* it will packet a skb buffer, and include vsock |
|                 * hdr and payload */                             |
|  -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */|
|vsock hdr and payload, and then call                              |
+------------------------------------------------------------------+
Quoted 1 line
Note, if we've negotiated the feature, virtio-net driver must not use register_netdev to register it to network core. This can avoid lots of confusion.
Hi Jason,
You mean we should not register netdev if use vsock, and in order to avoid confusion, then I think whether we should keep vsock and export some virtio-net's functions that can be shared. In this way, first, vsock may keep existing architecture and will not affect virtio-net.

Quoted 7 lines

Quoted 1 line
At least it needs new feature bits which will be a functional duplication of virtio-net (e.g mrg_rxbuf).
Hi Jason,
Actually I mean only use some shared function to make vsock support these features, in that way, guest see the device is still vsock device instead of virtio-net device, in addition, it can have less codes and easier to be compatible with old vsock version.

Yes, I think we're talking about same thing. Both of us want to share codes. What you want is to export and share some common helpers between virtio-net and vsock. What I meant is to e.g probe vsock device and merge vsock specific codes into virtio-net driver. I agree it's not a small project. We can start from e.g patches that try to share the codes. This could also give us inspiration for how to unify them.

Quoted 74 lines

Thanks,
Yiwen.
Quoted 6 lines
Quoted 4 lines
In addition,
vsock doesn't need to use virtio_net header too, then it don't need to pack skb structure.
For mergeable rx buffer it need I think?
As said above, I will define the related structure in the virtio-vsock module.
Quoted 55 lines
Quoted 55 lines
Thanks,
Yiwen.
Quoted 32 lines
Quoted 9 lines
                                     |
                                     |
+------------------------------------------------------------------+
|Virtio-net Module                                                 |
|start_xmit                                                        |
|  -> add virtio_net_hdr and pack sg in ring desc, notify Host     |
+------------------------------------------------------------------+
                                     |
                                     |
Quoted 8 lines
+------------------------------------------------------------------+
|Vhost-net Module                                                  |
|handle_tx                                                         |
|  -> get tx buffer, skip virtio_net_hdr and call Vsock function.  |
| /* This point has some differences, vhost-net use ->sendmsg to   |
|  * forward information, however vsock only need to notify server |
|  * that data ready. */                                           |
+------------------------------------------------------------------+
When VIRTIO_NET_F_VOSCK is negotiated, we know that it's a vsock transport, we can then forward it to vsock core.
Quoted 12 lines
                                     |
                                     |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|alloc_pkt, copy skb data to pkt.                                  |
|add pkt to rx_queue and notify server to get data.                |
+------------------------------------------------------------------+
3. To Host->Guest
I have a problem and difficult, mainly I know about virtio-net a little), because I have been doing work related with storage and file system.
Quoted 7 lines
Quoted 6 lines
The problem as follows:
we should monitor all of socket of vsock in handle_rx, when there are data coming, and copy data to vq desc. Vhost-net use ->recvmsg to get data, it is different with socket. To vsock, I think host will not call ->recvmsg when it need to send message to guest. To net, vhost-net only as forwarding layer.
Know not much here, but is it possible to have a vsock(tap) to be passed to vhost_net and let vhost call its recvmgs()? Bascially it was a socket on host as well I believe?
For vsock, Host->Guest, it's code flow as follows:
Server call send()
    -> sendmsg(); (vsock_stream_sendmsg)
      -> virtio_trasnport_send_pkt_info
        -> alloc pkt, add pkt to send_pkt_list, wake up vhost_worker
Vhost_worker
    -> vhost_transport_send_pkt_work
      -> get pkt from send_pkt_list
      -> get vq input desc and then fill data to desc addr
      -> update used ring and then signal guest
Quoted 7 lines
Quoted 2 lines
In the whole process, host don't call recvmsg() because it is a net device, and it also receives any messages.
If I understand this correctly, there's no a socket object on host that represent vosck in guest? If yes, maybe we can invent one.
Thanks

Quoted 8 lines

Sorry, I am not understanding you very much, vsock only a socket channel, it does not have a network device entity, so it only transmit the data between server and client, the data is only saved in server and client. Instead of vhost-net, I feel it has a network device that can saved the data, so when host send message to guest, it can use recvmsg() from the network device(tap). For Vsock, recvmsg() interface will read message from tx vq.

So I understand the model is not a real socket pair on host which AF_UNIX did. Maybe it's better to have one. What I meant is, have a socket that represent for each guest vsock device on host (guest socket). Then when you transfer packets from host to guest, you can queue the packets into the receive queue of the guest socket and wake up vhost-net and it will call recvmsg() for the guest socket. And when you want to transfer packets form guest to host, vhost_net will call sendmsg() to the guest socket on host then it can search the correct destination and queue packet on the receive of host socket. This can make vsock much more easier to be integrated with vhost_net.

Thanks

Quoted 20 lines

Quoted 19 lines
Quoted 17 lines
For vhost-net, I understand it is a tap device, so it can receive messages from other net device.
This is my understanding, it may have some errors.
Thanks.
Quoted 9 lines
If this doesn't work, we can have vsock specific receiving routine in vhost_net if VIRTIO_NET_F_VOSCK is negotiated.
Generally, I think we should try out best to keep the exist sendmsg()/recvmsg() interfaces and only consider the alternatives if we meet some real blocker.
Thanks
.
.

jiangyiwen

author

2018-11-16 02:32 UTC #7 lore raw

Quoted 64 lines · Jason Wang

Quoted 62 lines · jiangyiwen
Quoted 51 lines · Jason Wang
Quoted 50 lines · jiangyiwen
Quoted 49 lines · Jason Wang
Quoted 9 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
Quoted 6 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(
I think we should have a new feature flag for this. E.g VIRTIO_NET_F_VSOCK. And host should fail the negotiation if guest doesn't support this to avoid confusion. When this feature is negotiated, we will use it only for VOSCK transport. This can simplify things somehow.
Quoted 16 lines
-------------------------Simple idea------------------------------
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+
2. The Guest->Host basic code flow as follow:
Quoted 13 lines
                               +------------+
                               |   Client   |
                               +------------+
                                     |
                                     |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|ops->sendmsg; (vsock_stream_sendmsg)                              |
|  -> alloc_skb; /* it will packet a skb buffer, and include vsock |
|                 * hdr and payload */                             |
|  -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */|
|vsock hdr and payload, and then call                              |
+------------------------------------------------------------------+
Note, if we've negotiated the feature, virtio-net driver must not use register_netdev to register it to network core. This can avoid lots of confusion.
Quoted 8 lines
Quoted 6 lines
Hi Jason,
You mean we should not register netdev if use vsock, and in order to avoid confusion, then I think whether we should keep vsock and export some virtio-net's functions that can be shared. In this way, first, vsock may keep existing architecture and will not affect virtio-net.
At least it needs new feature bits which will be a functional duplication of virtio-net (e.g mrg_rxbuf).
Hi Jason,

Quoted 9 lines

Quoted 4 lines
Actually I mean only use some shared function to make vsock support these features, in that way, guest see the device is still vsock device instead of virtio-net device, in addition, it can have less codes and easier to be compatible with old vsock version.
Yes, I think we're talking about same thing. Both of us want to share codes. What you want is to export and share some common helpers between virtio-net and vsock. What I meant is to e.g probe vsock device and merge vsock specific codes into virtio-net driver. I agree it's not a small project. We can start from e.g patches that try to share the codes. This could also give us inspiration for how to unify them.

Then we have two ways to implement it.
1. For Guest, it is a virtio-net device(VIRTIO_ID_NET), use a feature bit(e.g VIRTIO_NET_F_VSOCK) to distinguish vsock special device and other virtio-net device. For old vsock device, it still use old driver to drive it.

2. For Guest, it is a virtio-vsock device(VIRTIO_ID_VSOCK), use virtio device id to distinguish them, it will integrate old driver, but it may be more complicated, because it needs to consider the compatibility with old vsock device.

Quoted 103 lines

Quoted 74 lines
Thanks,
Yiwen.
Quoted 6 lines
Quoted 4 lines
In addition,
vsock doesn't need to use virtio_net header too, then it don't need to pack skb structure.
For mergeable rx buffer it need I think?
As said above, I will define the related structure in the virtio-vsock module.
Quoted 55 lines
Quoted 55 lines
Thanks,
Yiwen.
Quoted 32 lines
Quoted 9 lines
                                     |
                                     |
+------------------------------------------------------------------+
|Virtio-net Module                                                 |
|start_xmit                                                        |
|  -> add virtio_net_hdr and pack sg in ring desc, notify Host     |
+------------------------------------------------------------------+
                                     |
                                     |
Quoted 8 lines
+------------------------------------------------------------------+
|Vhost-net Module                                                  |
|handle_tx                                                         |
|  -> get tx buffer, skip virtio_net_hdr and call Vsock function.  |
| /* This point has some differences, vhost-net use ->sendmsg to   |
|  * forward information, however vsock only need to notify server |
|  * that data ready. */                                           |
+------------------------------------------------------------------+
When VIRTIO_NET_F_VOSCK is negotiated, we know that it's a vsock transport, we can then forward it to vsock core.
Quoted 12 lines
                                     |
                                     |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|alloc_pkt, copy skb data to pkt.                                  |
|add pkt to rx_queue and notify server to get data.                |
+------------------------------------------------------------------+
3. To Host->Guest
I have a problem and difficult, mainly I know about virtio-net a little), because I have been doing work related with storage and file system.
Quoted 7 lines
Quoted 6 lines
The problem as follows:
we should monitor all of socket of vsock in handle_rx, when there are data coming, and copy data to vq desc. Vhost-net use ->recvmsg to get data, it is different with socket. To vsock, I think host will not call ->recvmsg when it need to send message to guest. To net, vhost-net only as forwarding layer.
Know not much here, but is it possible to have a vsock(tap) to be passed to vhost_net and let vhost call its recvmgs()? Bascially it was a socket on host as well I believe?
For vsock, Host->Guest, it's code flow as follows:
Server call send()
    -> sendmsg(); (vsock_stream_sendmsg)
      -> virtio_trasnport_send_pkt_info
        -> alloc pkt, add pkt to send_pkt_list, wake up vhost_worker
Vhost_worker
    -> vhost_transport_send_pkt_work
      -> get pkt from send_pkt_list
      -> get vq input desc and then fill data to desc addr
      -> update used ring and then signal guest
Quoted 7 lines
Quoted 2 lines
In the whole process, host don't call recvmsg() because it is a net device, and it also receives any messages.
If I understand this correctly, there's no a socket object on host that represent vosck in guest? If yes, maybe we can invent one.
Thanks
Quoted 8 lines
Sorry, I am not understanding you very much, vsock only a socket channel, it does not have a network device entity, so it only transmit the data between server and client, the data is only saved in server and client. Instead of vhost-net, I feel it has a network device that can saved the data, so when host send message to guest, it can use recvmsg() from the network device(tap). For Vsock, recvmsg() interface will read message from tx vq.
So I understand the model is not a real socket pair on host which AF_UNIX did. Maybe it's better to have one. What I meant is, have a socket that represent for each guest vsock device on host (guest socket). Then when you transfer packets from host to guest, you can queue the packets into the receive queue of the guest socket and wake up vhost-net and it will call recvmsg() for the guest socket. And when you want to transfer packets form guest to host, vhost_net will call sendmsg() to the guest socket on host then it can search the correct destination and queue packet on the receive of host socket. This can make vsock much more easier to be integrated with vhost_net.
Thanks
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
For vhost-net, I understand it is a tap device, so it can receive messages from other net device.
This is my understanding, it may have some errors.
Thanks.
Quoted 7 lines
If this doesn't work, we can have vsock specific receiving routine in vhost_net if VIRTIO_NET_F_VOSCK is negotiated.
Generally, I think we should try out best to keep the exist sendmsg()/recvmsg() interfaces and only consider the alternatives if we meet some real blocker.
Thanks

Quoted 8 lines

Quoted 5 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
.
.
.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Jason Wang

2018-11-16 06:35 UTC #8 lore raw

Quoted 23 lines · jiangyiwen

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Note, if we've negotiated the feature, virtio-net driver must not use register_netdev to register it to network core. This can avoid lots of confusion.
Hi Jason,
You mean we should not register netdev if use vsock, and in order to avoid confusion, then I think whether we should keep vsock and export some virtio-net's functions that can be shared. In this way, first, vsock may keep existing architecture and will not affect virtio-net.
Quoted 10 lines
Quoted 7 lines
Quoted 1 line
At least it needs new feature bits which will be a functional duplication of virtio-net (e.g mrg_rxbuf).
Hi Jason,
Actually I mean only use some shared function to make vsock support these features, in that way, guest see the device is still vsock device instead of virtio-net device, in addition, it can have less codes and easier to be compatible with old vsock version.
Yes, I think we're talking about same thing. Both of us want to share codes. What you want is to export and share some common helpers between virtio-net and vsock. What I meant is to e.g probe vsock device and merge vsock specific codes into virtio-net driver. I agree it's not a small project. We can start from e.g patches that try to share the codes. This could also give us inspiration for how to unify them.
Then we have two ways to implement it.
1. For Guest, it is a virtio-net device(VIRTIO_ID_NET), use a feature bit(e.g VIRTIO_NET_F_VSOCK) to distinguish vsock special device and other virtio-net device. For old vsock device, it still use old driver to drive it.

Quoted 4 lines

2. For Guest, it is a virtio-vsock device(VIRTIO_ID_VSOCK), use virtio device id to distinguish them, it will integrate old driver, but it may be more complicated, because it needs to consider the compatibility with old vsock device.

This looks even better, btw what compatibility do you mean here? Looking at vsock driver, it does not have vosck specific feature.

Thanks

Jason Wang

2018-11-16 06:35 UTC #9 lore raw

Quoted 23 lines · jiangyiwen

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Note, if we've negotiated the feature, virtio-net driver must not use register_netdev to register it to network core. This can avoid lots of confusion.
Hi Jason,
You mean we should not register netdev if use vsock, and in order to avoid confusion, then I think whether we should keep vsock and export some virtio-net's functions that can be shared. In this way, first, vsock may keep existing architecture and will not affect virtio-net.
Quoted 10 lines
Quoted 7 lines
Quoted 1 line
At least it needs new feature bits which will be a functional duplication of virtio-net (e.g mrg_rxbuf).
Hi Jason,
Actually I mean only use some shared function to make vsock support these features, in that way, guest see the device is still vsock device instead of virtio-net device, in addition, it can have less codes and easier to be compatible with old vsock version.
Yes, I think we're talking about same thing. Both of us want to share codes. What you want is to export and share some common helpers between virtio-net and vsock. What I meant is to e.g probe vsock device and merge vsock specific codes into virtio-net driver. I agree it's not a small project. We can start from e.g patches that try to share the codes. This could also give us inspiration for how to unify them.
Then we have two ways to implement it.
1. For Guest, it is a virtio-net device(VIRTIO_ID_NET), use a feature bit(e.g VIRTIO_NET_F_VSOCK) to distinguish vsock special device and other virtio-net device. For old vsock device, it still use old driver to drive it.

Quoted 4 lines

2. For Guest, it is a virtio-vsock device(VIRTIO_ID_VSOCK), use virtio device id to distinguish them, it will integrate old driver, but it may be more complicated, because it needs to consider the compatibility with old vsock device.

This looks even better, btw what compatibility do you mean here? Looking at vsock driver, it does not have vosck specific feature.

Thanks

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Jason Wang

2018-11-15 09:21 UTC #10 lore raw

Quoted 59 lines · jiangyiwen

Quoted 58 lines · Jason Wang
Quoted 56 lines · jiangyiwen
Quoted 48 lines · Jason Wang
Quoted 9 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
Quoted 6 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(
I think we should have a new feature flag for this. E.g VIRTIO_NET_F_VSOCK. And host should fail the negotiation if guest doesn't support this to avoid confusion. When this feature is negotiated, we will use it only for VOSCK transport. This can simplify things somehow.
Quoted 16 lines
-------------------------Simple idea------------------------------
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+
2. The Guest->Host basic code flow as follow:
Quoted 13 lines
                               +------------+
                               |   Client   |
                               +------------+
                                     |
                                     |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|ops->sendmsg; (vsock_stream_sendmsg)                              |
|  -> alloc_skb; /* it will packet a skb buffer, and include vsock |
|                 * hdr and payload */                             |
|  -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */|
|vsock hdr and payload, and then call                              |
+------------------------------------------------------------------+
Quoted 1 line
Note, if we've negotiated the feature, virtio-net driver must not use register_netdev to register it to network core. This can avoid lots of confusion.
Hi Jason,
You mean we should not register netdev if use vsock, and in order to avoid confusion, then I think whether we should keep vsock and export some virtio-net's functions that can be shared. In this way, first, vsock may keep existing architecture and will not affect virtio-net.

Quoted 7 lines

Quoted 1 line
At least it needs new feature bits which will be a functional duplication of virtio-net (e.g mrg_rxbuf).
Hi Jason,
Actually I mean only use some shared function to make vsock support these features, in that way, guest see the device is still vsock device instead of virtio-net device, in addition, it can have less codes and easier to be compatible with old vsock version.

Quoted 74 lines

Thanks,
Yiwen.
Quoted 6 lines
Quoted 4 lines
In addition,
vsock doesn't need to use virtio_net header too, then it don't need to pack skb structure.
For mergeable rx buffer it need I think?
As said above, I will define the related structure in the virtio-vsock module.
Quoted 55 lines
Quoted 55 lines
Thanks,
Yiwen.
Quoted 32 lines
Quoted 9 lines
                                     |
                                     |
+------------------------------------------------------------------+
|Virtio-net Module                                                 |
|start_xmit                                                        |
|  -> add virtio_net_hdr and pack sg in ring desc, notify Host     |
+------------------------------------------------------------------+
                                     |
                                     |
Quoted 8 lines
+------------------------------------------------------------------+
|Vhost-net Module                                                  |
|handle_tx                                                         |
|  -> get tx buffer, skip virtio_net_hdr and call Vsock function.  |
| /* This point has some differences, vhost-net use ->sendmsg to   |
|  * forward information, however vsock only need to notify server |
|  * that data ready. */                                           |
+------------------------------------------------------------------+
When VIRTIO_NET_F_VOSCK is negotiated, we know that it's a vsock transport, we can then forward it to vsock core.
Quoted 12 lines
                                     |
                                     |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|alloc_pkt, copy skb data to pkt.                                  |
|add pkt to rx_queue and notify server to get data.                |
+------------------------------------------------------------------+
3. To Host->Guest
I have a problem and difficult, mainly I know about virtio-net a little), because I have been doing work related with storage and file system.
Quoted 7 lines
Quoted 6 lines
The problem as follows:
we should monitor all of socket of vsock in handle_rx, when there are data coming, and copy data to vq desc. Vhost-net use ->recvmsg to get data, it is different with socket. To vsock, I think host will not call ->recvmsg when it need to send message to guest. To net, vhost-net only as forwarding layer.
Know not much here, but is it possible to have a vsock(tap) to be passed to vhost_net and let vhost call its recvmgs()? Bascially it was a socket on host as well I believe?
For vsock, Host->Guest, it's code flow as follows:
Server call send()
    -> sendmsg(); (vsock_stream_sendmsg)
      -> virtio_trasnport_send_pkt_info
        -> alloc pkt, add pkt to send_pkt_list, wake up vhost_worker
Vhost_worker
    -> vhost_transport_send_pkt_work
      -> get pkt from send_pkt_list
      -> get vq input desc and then fill data to desc addr
      -> update used ring and then signal guest
Quoted 7 lines
Quoted 2 lines
In the whole process, host don't call recvmsg() because it is a net device, and it also receives any messages.
If I understand this correctly, there's no a socket object on host that represent vosck in guest? If yes, maybe we can invent one.
Thanks

Quoted 8 lines

Sorry, I am not understanding you very much, vsock only a socket channel, it does not have a network device entity, so it only transmit the data between server and client, the data is only saved in server and client. Instead of vhost-net, I feel it has a network device that can saved the data, so when host send message to guest, it can use recvmsg() from the network device(tap). For Vsock, recvmsg() interface will read message from tx vq.

Thanks

Quoted 12 lines

Quoted 12 lines
Quoted 12 lines
For vhost-net, I understand it is a tap device, so it can receive messages from other net device.
This is my understanding, it may have some errors.
Thanks.
Quoted 4 lines
If this doesn't work, we can have vsock specific receiving routine in vhost_net if VIRTIO_NET_F_VOSCK is negotiated.
Generally, I think we should try out best to keep the exist sendmsg()/recvmsg() interfaces and only consider the alternatives if we meet some real blocker.

Quoted 8 lines

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Thanks
.
.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

jiangyiwen

author

2018-11-15 09:02 UTC #11 lore raw

Quoted 56 lines · Jason Wang

Quoted 54 lines · jiangyiwen
Quoted 50 lines · Jason Wang
Quoted 9 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
Quoted 6 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(
I think we should have a new feature flag for this. E.g VIRTIO_NET_F_VSOCK. And host should fail the negotiation if guest doesn't support this to avoid confusion. When this feature is negotiated, we will use it only for VOSCK transport. This can simplify things somehow.
Quoted 16 lines
-------------------------Simple idea------------------------------
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+
2. The Guest->Host basic code flow as follow:
Quoted 13 lines
                              +------------+
                              |   Client   |
                              +------------+
                                    |
                                    |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|ops->sendmsg; (vsock_stream_sendmsg)                              |
|  -> alloc_skb; /* it will packet a skb buffer, and include vsock |
|                 * hdr and payload */                             |
|  -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */|
|vsock hdr and payload, and then call                              |
+------------------------------------------------------------------+
Quoted 1 line
Note, if we've negotiated the feature, virtio-net driver must not use register_netdev to register it to network core. This can avoid lots of confusion.
Hi Jason,

Quoted 7 lines

Quoted 4 lines
You mean we should not register netdev if use vsock, and in order to avoid confusion, then I think whether we should keep vsock and export some virtio-net's functions that can be shared. In this way, first, vsock may keep existing architecture and will not affect virtio-net.
At least it needs new feature bits which will be a functional duplication of virtio-net (e.g mrg_rxbuf).

Hi Jason,

Thanks,
Yiwen.

Quoted 8 lines

Quoted 5 lines
In addition,
vsock doesn't need to use virtio_net header too, then it don't need to pack skb structure.
For mergeable rx buffer it need I think?

As said above, I will define the related structure in the virtio-vsock module.

Quoted 59 lines

Quoted 59 lines
Thanks,
Yiwen.
Quoted 34 lines
Quoted 10 lines
                                    |
                                    |
+------------------------------------------------------------------+
|Virtio-net Module                                                 |
|start_xmit                                                        |
|  -> add virtio_net_hdr and pack sg in ring desc, notify Host     |
+------------------------------------------------------------------+
                                    |
                                    |
Quoted 8 lines
+------------------------------------------------------------------+
|Vhost-net Module                                                  |
|handle_tx                                                         |
|  -> get tx buffer, skip virtio_net_hdr and call Vsock function.  |
| /* This point has some differences, vhost-net use ->sendmsg to   |
|  * forward information, however vsock only need to notify server |
|  * that data ready. */                                           |
+------------------------------------------------------------------+
When VIRTIO_NET_F_VOSCK is negotiated, we know that it's a vsock transport, we can then forward it to vsock core.
Quoted 12 lines
                                    |
                                    |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|alloc_pkt, copy skb data to pkt.                                  |
|add pkt to rx_queue and notify server to get data.                |
+------------------------------------------------------------------+
3. To Host->Guest
I have a problem and difficult, mainly I know about virtio-net a little), because I have been doing work related with storage and file system.
Quoted 7 lines
Quoted 6 lines
The problem as follows:
we should monitor all of socket of vsock in handle_rx, when there are data coming, and copy data to vq desc. Vhost-net use ->recvmsg to get data, it is different with socket. To vsock, I think host will not call ->recvmsg when it need to send message to guest. To net, vhost-net only as forwarding layer.
Know not much here, but is it possible to have a vsock(tap) to be passed to vhost_net and let vhost call its recvmgs()? Bascially it was a socket on host as well I believe?
For vsock, Host->Guest, it's code flow as follows:
Server call send()
-> sendmsg(); (vsock_stream_sendmsg)
     -> virtio_trasnport_send_pkt_info
       -> alloc pkt, add pkt to send_pkt_list, wake up vhost_worker
Vhost_worker
-> vhost_transport_send_pkt_work
     -> get pkt from send_pkt_list
     -> get vq input desc and then fill data to desc addr
     -> update used ring and then signal guest

Quoted 8 lines

Quoted 2 lines
In the whole process, host don't call recvmsg() because it is a net device, and it also receives any messages.
If I understand this correctly, there's no a socket object on host that represent vosck in guest? If yes, maybe we can invent one.
Thanks

Quoted 13 lines

Quoted 13 lines
For vhost-net, I understand it is a tap device, so it can receive messages from other net device.
This is my understanding, it may have some errors.
Thanks.
Quoted 4 lines
If this doesn't work, we can have vsock specific receiving routine in vhost_net if VIRTIO_NET_F_VOSCK is negotiated.
Generally, I think we should try out best to keep the exist sendmsg()/recvmsg() interfaces and only consider the alternatives if we meet some real blocker.

Quoted 9 lines

Quoted 6 lines
Quoted 5 lines
Thanks
.
.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Jason Wang

2018-11-15 04:19 UTC #12 lore raw

Quoted 9 lines · jiangyiwen

Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.

Quoted 6 lines

Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(

Quoted 16 lines

-------------------------Simple idea------------------------------

1. The packet layout will become as follows:

+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+

2. The Guest->Host basic code flow as follow:

Quoted 13 lines

                             +------------+
                             |   Client   |
                             +------------+
                                   |
                                   |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|ops->sendmsg; (vsock_stream_sendmsg)                              |
|  -> alloc_skb; /* it will packet a skb buffer, and include vsock |
|                 * hdr and payload */                             |
|  -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */|
|vsock hdr and payload, and then call                              |
+------------------------------------------------------------------+

Note, if we've negotiated the feature, virtio-net driver must not use register_netdev to register it to network core. This can avoid lots of confusion.

Quoted 9 lines

                                   |
                                   |
+------------------------------------------------------------------+
|Virtio-net Module                                                 |
|start_xmit                                                        |
|  -> add virtio_net_hdr and pack sg in ring desc, notify Host     |
+------------------------------------------------------------------+
                                   |
                                   |

Quoted 8 lines

+------------------------------------------------------------------+
|Vhost-net Module                                                  |
|handle_tx                                                         |
|  -> get tx buffer, skip virtio_net_hdr and call Vsock function.  |
| /* This point has some differences, vhost-net use ->sendmsg to   |
|  * forward information, however vsock only need to notify server |
|  * that data ready. */                                           |
+------------------------------------------------------------------+

When VIRTIO_NET_F_VOSCK is negotiated, we know that it's a vsock transport, we can then forward it to vsock core.

Quoted 12 lines

                                   |
                                   |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|alloc_pkt, copy skb data to pkt.                                  |
|add pkt to rx_queue and notify server to get data.                |
+------------------------------------------------------------------+

3. To Host->Guest
I have a problem and difficult, mainly I know about virtio-net a little), because I have been doing work related with storage and file system.

Quoted 6 lines

The problem as follows:
we should monitor all of socket of vsock in handle_rx, when there are data coming, and copy data to vq desc. Vhost-net use ->recvmsg to get data, it is different with socket. To vsock, I think host will not call ->recvmsg when it need to send message to guest. To net, vhost-net only as forwarding layer.

Know not much here, but is it possible to have a vsock(tap) to be passed to vhost_net and let vhost call its recvmgs()? Bascially it was a socket on host as well I believe?

If this doesn't work, we can have vsock specific receiving routine in vhost_net if VIRTIO_NET_F_VOSCK is negotiated.

Generally, I think we should try out best to keep the exist sendmsg()/recvmsg() interfaces and only consider the alternatives if we meet some real blocker.

Thanks

Quoted 1 line

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Michael S. Tsirkin

maintainer

2018-11-15 07:04 UTC #13 lore raw

Quoted 8 lines · jiangyiwen

Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.

I'm not sure I get the motivation. Which features of virtio net are relevant to vsock? The ones that you mention all seem to be mostly of use to the networking stack.

Quoted 7 lines

Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(

Quoted 7 lines

-------------------------Simple idea------------------------------
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|

Which fields in virtio_net_hdr_mrg_rxbuf are of interest to vsock?

Quoted 7 lines

+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+

Thanks,

Signature · 1 line

MST

jiangyiwen

author

2018-11-15 07:38 UTC #14 lore raw

Quoted 24 lines · Michael S. Tsirkin

Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock? The ones that you mention all seem to be mostly of use to the networking stack.
Quoted 9 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(
-------------------------Simple idea------------------------------

Quoted 8 lines

Quoted 5 lines
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
Which fields in virtio_net_hdr_mrg_rxbuf are of interest to vsock?

Hi Michael,

Yes, currently vsock has poor performance, first, it only support transport small packet, in order to make the balance between performance and guest memory.

In order to solve this problem, there are two features vsock can used, mergeable rx buffer and multiqueue. Based on this, there are some shared codes vsock can use.

Thanks,
Yiwen.

Quoted 10 lines

Quoted 7 lines

+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+

Thanks,

Michael S. Tsirkin

maintainer

2018-11-15 08:10 UTC #15 lore raw

Quoted 39 lines · jiangyiwen

Quoted 24 lines · Michael S. Tsirkin
Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock? The ones that you mention all seem to be mostly of use to the networking stack.
Quoted 9 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(
-------------------------Simple idea------------------------------
Quoted 8 lines
Quoted 5 lines
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
Which fields in virtio_net_hdr_mrg_rxbuf are of interest to vsock?
Hi Michael,
Yes, currently vsock has poor performance, first, it only support transport small packet, in order to make the balance between performance and guest memory.

Quoted 6 lines

In order to solve this problem, there are two features vsock can used, mergeable rx buffer and multiqueue. Based on this, there are some shared codes vsock can use.
Thanks,
Yiwen.

Supporting more queues with vsock is probably significantly less work than a completely new interface.
For mergeable, as buffers are split arbitrarily, why can't you combine them within guest driver before sending them up the stack? Probably better than relying on host to do it.

Quoted 11 lines

Quoted 10 lines

Quoted 7 lines

+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+

Thanks,

Jason Wang

2018-11-15 08:27 UTC #16 lore raw

Quoted 41 lines · Michael S. Tsirkin

Quoted 40 lines · jiangyiwen
Quoted 21 lines · Michael S. Tsirkin
Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock? The ones that you mention all seem to be mostly of use to the networking stack.
Quoted 7 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity.:(
Quoted 9 lines
Quoted 7 lines
-------------------------Simple idea------------------------------
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
Which fields in virtio_net_hdr_mrg_rxbuf are of interest to vsock?
Hi Michael,
Yes, currently vsock has poor performance, first, it only support transport small packet, in order to make the balance between performance and guest memory.
In order to solve this problem, there are two features vsock can used, mergeable rx buffer and multiqueue. Based on this, there are some shared codes vsock can use.

Quoted 6 lines

Quoted 2 lines
Thanks,
Yiwen.
Supporting more queues with vsock is probably significantly less work than a completely new interface.
For mergeable, as buffers are split arbitrarily, why can't you combine them within guest driver before sending them up the stack?

I don't get this question. But I think the fact that we use mergeable buffer by default for virtio-net answer the question. Or anything special in vsock?

Thanks

Quoted 2 lines

Probably better than relying on host to do it.

Jason Wang

2018-11-15 08:27 UTC #17 lore raw

Quoted 41 lines · Michael S. Tsirkin

Quoted 40 lines · jiangyiwen
Quoted 21 lines · Michael S. Tsirkin
Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock? The ones that you mention all seem to be mostly of use to the networking stack.
Quoted 7 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity.:(
Quoted 9 lines
Quoted 7 lines
-------------------------Simple idea------------------------------
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
Which fields in virtio_net_hdr_mrg_rxbuf are of interest to vsock?
Hi Michael,
Yes, currently vsock has poor performance, first, it only support transport small packet, in order to make the balance between performance and guest memory.
In order to solve this problem, there are two features vsock can used, mergeable rx buffer and multiqueue. Based on this, there are some shared codes vsock can use.

Quoted 6 lines

Quoted 2 lines
Thanks,
Yiwen.
Supporting more queues with vsock is probably significantly less work than a completely new interface.
For mergeable, as buffers are split arbitrarily, why can't you combine them within guest driver before sending them up the stack?

I don't get this question. But I think the fact that we use mergeable buffer by default for virtio-net answer the question. Or anything special in vsock?

Thanks

Quoted 2 lines

Probably better than relying on host to do it.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

jiangyiwen

author

2018-11-15 08:38 UTC #18 lore raw

Quoted 44 lines · Michael S. Tsirkin

Quoted 43 lines · jiangyiwen
Quoted 24 lines · Michael S. Tsirkin
Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock? The ones that you mention all seem to be mostly of use to the networking stack.
Quoted 9 lines
Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(
-------------------------Simple idea------------------------------
Quoted 8 lines
Quoted 5 lines
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
Which fields in virtio_net_hdr_mrg_rxbuf are of interest to vsock?
Hi Michael,
Yes, currently vsock has poor performance, first, it only support transport small packet, in order to make the balance between performance and guest memory.
In order to solve this problem, there are two features vsock can used, mergeable rx buffer and multiqueue. Based on this, there are some shared codes vsock can use.

Quoted 9 lines

Quoted 2 lines
Thanks,
Yiwen.
Supporting more queues with vsock is probably significantly less work than a completely new interface.
For mergeable, as buffers are split arbitrarily, why can't you combine them within guest driver before sending them up the stack? Probably better than relying on host to do it.

Actually, I want to said the mergeable *rx* buffer, it cause the default packet size of vsock is set to 4k. For tx buffer, it is actually can be scattered in tx vq only need to modify very few codes.

In addition, I has already first version about vsock support mergeable rx buffer and send patch to the VSOCK community, these codes are revisited from virtio-net, and some codes are duplicated. Based on this, we want to use virtio-net as transport of vsock. It can make the vsock code easy to maintained.

Thanks.

Quoted 14 lines

Quoted 11 lines

Quoted 10 lines

Quoted 7 lines

+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+

Thanks,

Jason Wang

2018-11-15 08:24 UTC #19 lore raw

Quoted 11 lines · Michael S. Tsirkin

Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock?

Vsock is just a L2 (and above) protocol from the view of the device. So I think we should answer the question why we need two different paths for networking traffic? Or what is the fundamental reason that makes vsock does not go for virtio-net?

I agree they could be different type of devices but codes could be shared in both guest and host (or even qemu) for not duplicating features(bugs).

Thanks

Quoted 4 lines

The ones that you mention
all seem to be mostly of use to the networking stack.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Michael S. Tsirkin

maintainer

2018-11-29 14:00 UTC #20 lore raw

Quoted 11 lines · Jason Wang

Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.

Quoted 5 lines

Quoted 2 lines
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock?
Vsock is just a L2 (and above) protocol from the view of the device.

I don't believe so. I think virtio-vsock operates at a transport level. There is in theory a bit of network level but we don't really implement it as it's only host to guest. I am not aware of any data link functionality n virtio-vsock. virtio-vsock provides services such as connection-oriented communication, reliability, flow control and multiplexing.

Quoted 4 lines

So I
think we should answer the question why we need two different paths for networking traffic? Or what is the fundamental reason that makes vsock does not go for virtio-net?

So virtio-vsock ensures reliability. If you want to compare it with something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.

Quoted 10 lines

I agree they could be different type of devices but codes could be shared in both guest and host (or even qemu) for not duplicating features(bugs).
Thanks
Quoted 4 lines
The ones that you mention
all seem to be mostly of use to the networking stack.

Jason Wang

2018-11-30 12:45 UTC #21 lore raw

Quoted 14 lines · Michael S. Tsirkin

Quoted 13 lines · Jason Wang
Quoted 11 lines · Michael S. Tsirkin
Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock?

Quoted 7 lines

Quoted 1 line
Vsock is just a L2 (and above) protocol from the view of the device.
I don't believe so. I think virtio-vsock operates at a transport level. There is in theory a bit of network level but we don't really implement it as it's only host to guest. I am not aware of any data link functionality n virtio-vsock. virtio-vsock provides services such as connection-oriented communication, reliability, flow control and multiplexing.

Ok, consider it doesn't implement L2, it's pretty fit for virtio-net I believe?

Quoted 6 lines

Quoted 5 lines
So I
think we should answer the question why we need two different paths for networking traffic? Or what is the fundamental reason that makes vsock does not go for virtio-net?
So virtio-vsock ensures reliability.

It's done at the level of protocol instead of virtio transport or virtio device.

Quoted 5 lines

If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.

Virtio-net knows nothing above L2, so they are totally transparent to device itself. I still don't get why not using virtio-net instead.

Thanks

Quoted 12 lines

Quoted 12 lines
I agree they could be different type of devices but codes could be shared in both guest and host (or even qemu) for not duplicating features(bugs).
Thanks
Quoted 4 lines
The ones that you mention
all seem to be mostly of use to the networking stack.

Michael S. Tsirkin

maintainer

2018-11-30 12:52 UTC #22 lore raw

Quoted 48 lines · Jason Wang

Quoted 14 lines · Michael S. Tsirkin
Quoted 13 lines · Jason Wang
Quoted 11 lines · Michael S. Tsirkin
Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock?
Quoted 7 lines
Quoted 1 line
Vsock is just a L2 (and above) protocol from the view of the device.
I don't believe so. I think virtio-vsock operates at a transport level. There is in theory a bit of network level but we don't really implement it as it's only host to guest. I am not aware of any data link functionality n virtio-vsock. virtio-vsock provides services such as connection-oriented communication, reliability, flow control and multiplexing.
Ok, consider it doesn't implement L2, it's pretty fit for virtio-net I believe?
Quoted 6 lines
Quoted 5 lines
So I
think we should answer the question why we need two different paths for networking traffic? Or what is the fundamental reason that makes vsock does not go for virtio-net?
So virtio-vsock ensures reliability.
It's done at the level of protocol instead of virtio transport or virtio device.
Quoted 5 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.

Quoted 5 lines

Virtio-net knows nothing above L2, so they are totally transparent to device itself. I still don't get why not using virtio-net instead.
Thanks

Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?

Quoted 13 lines

Quoted 13 lines
Quoted 13 lines
I agree they could be different type of devices but codes could be shared in both guest and host (or even qemu) for not duplicating features(bugs).
Thanks
Quoted 4 lines
The ones that you mention
all seem to be mostly of use to the networking stack.

Jason Wang

2018-11-30 12:55 UTC #23 lore raw

Quoted 5 lines · Michael S. Tsirkin

Quoted 5 lines
Quoted 5 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.

Quoted 9 lines

Quoted 5 lines
Virtio-net knows nothing above L2, so they are totally transparent to device itself. I still don't get why not using virtio-net instead.
Thanks
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?

No, my question is why not do vsock through virtio-net.

Thanks

Jason Wang

2018-11-30 13:10 UTC #24 lore raw

Quoted 12 lines · Jason Wang

Quoted 10 lines · Michael S. Tsirkin
Quoted 10 lines
Quoted 5 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device itself. I still don't get why not using virtio-net instead.
Thanks

Quoted 9 lines

Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks

Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:

- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.

- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).

The change should be totally transparent to userspace applications.

Thanks

Michael S. Tsirkin

maintainer

2018-11-30 13:40 UTC #25 lore raw

Quoted 34 lines · Jason Wang

Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.

Quoted 6 lines

- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks

Which code is duplicated between virtio vsock and virtio net right now?

Signature · 1 line

MST

jiangyiwen

author

2018-12-03 03:10 UTC #26 lore raw

Quoted 35 lines · Michael S. Tsirkin

Quoted 34 lines · Jason Wang
Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.

Quoted 9 lines

Quoted 6 lines
- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks
Which code is duplicated between virtio vsock and virtio net right now?

Hi Michael,

AFAIK, there is almost no duplicate code between virtio vsock and virtio net now.

But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, it has some duplicate codes from virtio net. Based on it, we both think vsock may use virtio net as a transport channel, in this way, vsock can use some of virtio net great features.

Thanks,
Yiwen.

Michael S. Tsirkin

maintainer

2018-12-04 01:31 UTC #27 lore raw

Quoted 50 lines · jiangyiwen

Quoted 35 lines · Michael S. Tsirkin
Quoted 34 lines · Jason Wang
Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.
Quoted 9 lines
Quoted 6 lines
- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks
Which code is duplicated between virtio vsock and virtio net right now?
Hi Michael,
AFAIK, there is almost no duplicate code between virtio vsock and virtio net now.

Quoted 7 lines

But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, it has some duplicate codes from virtio net. Based on it, we both think vsock may use virtio net as a transport channel, in this way, vsock can use some of virtio net great features.
Thanks,
Yiwen.

What I would do is just copy some code and show a performance benefit. If that works out it will be clearer which code should be shared.

Signature · 1 line

MST

jiangyiwen

author

2018-12-04 02:21 UTC #28 lore raw

Quoted 56 lines · Michael S. Tsirkin

Quoted 55 lines · jiangyiwen
Quoted 35 lines · Michael S. Tsirkin
Quoted 34 lines · Jason Wang
Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.
Quoted 9 lines
Quoted 6 lines
- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks
Which code is duplicated between virtio vsock and virtio net right now?
Hi Michael,
AFAIK, there is almost no duplicate code between virtio vsock and virtio net now.
But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, it has some duplicate codes from virtio net. Based on it, we both think vsock may use virtio net as a transport channel, in this way, vsock can use some of virtio net great features.

Quoted 7 lines

Quoted 2 lines
Thanks,
Yiwen.
What I would do is just copy some code and show a performance benefit. If that works out it will be clearer which code should be shared.

Hi Michael,

I have already sent a series of patches (VSOCK: support mergeable rx buffer in vhost-vsock) a month ago, and the performance as follows:

I write a tool to test the vhost-vsock performance, mainly send big packet(64K) included guest->Host and Host->Guest. The result as follows:

Before performance:

              Single socket            Multiple sockets(Max Bandwidth)
Guest->Host   ~400MB/s                 ~480MB/s
Host->Guest   ~1450MB/s                ~1600MB/s

After performance:

              Single socket            Multiple sockets(Max Bandwidth)
Guest->Host   ~1700MB/s                ~2900MB/s
Host->Guest   ~1700MB/s                ~2900MB/s

Quoted 1 line

From the test results, the performance is improved obviously, and guest

memory will not be wasted.

In addition, multiqueue feature I have not implemented it yet.

Thanks,
Yiwen.

Michael S. Tsirkin

maintainer

2018-12-04 04:08 UTC #29 lore raw

Quoted 79 lines · jiangyiwen

Quoted 56 lines · Michael S. Tsirkin
Quoted 55 lines · jiangyiwen
Quoted 35 lines · Michael S. Tsirkin
Quoted 34 lines · Jason Wang
Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.
Quoted 9 lines
Quoted 6 lines
- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks
Which code is duplicated between virtio vsock and virtio net right now?
Hi Michael,
AFAIK, there is almost no duplicate code between virtio vsock and virtio net now.
But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, it has some duplicate codes from virtio net. Based on it, we both think vsock may use virtio net as a transport channel, in this way, vsock can use some of virtio net great features.
Quoted 7 lines
Quoted 2 lines
Thanks,
Yiwen.
What I would do is just copy some code and show a performance benefit. If that works out it will be clearer which code should be shared.
Hi Michael,
I have already sent a series of patches (VSOCK: support mergeable rx buffer in vhost-vsock) a month ago, and the performance as follows:
I write a tool to test the vhost-vsock performance, mainly send big packet(64K) included guest->Host and Host->Guest. The result as follows:
Before performance:
              Single socket            Multiple sockets(Max Bandwidth)
Guest->Host   ~400MB/s                 ~480MB/s
Host->Guest   ~1450MB/s                ~1600MB/s

Quoted 7 lines

After performance:
              Single socket            Multiple sockets(Max Bandwidth)
Guest->Host   ~1700MB/s                ~2900MB/s
Host->Guest   ~1700MB/s                ~2900MB/s
Quoted 1 line
From the test results, the performance is improved obviously, and guest
memory will not be wasted.

Oh I didn't see that one. Pls CC me in the future.

Looking at it I agree zero page allocation looks like an issue but besides that, I think we can merge something similar and look at refactoring and future extensions later.

However, any interface change (e.g. a new feature) must be CC'd to one of virtio lists (subscriber-only).

Quoted 5 lines

In addition, multiqueue feature I have not implemented it yet.
Thanks,
Yiwen.

jiangyiwen

author

2018-12-04 06:01 UTC #30 lore raw

Quoted 88 lines · Michael S. Tsirkin

Quoted 79 lines · jiangyiwen
Quoted 56 lines · Michael S. Tsirkin
Quoted 55 lines · jiangyiwen
Quoted 35 lines · Michael S. Tsirkin
Quoted 34 lines · Jason Wang
Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.
Quoted 9 lines
Quoted 6 lines
- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks
Which code is duplicated between virtio vsock and virtio net right now?
Hi Michael,
AFAIK, there is almost no duplicate code between virtio vsock and virtio net now.
But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, it has some duplicate codes from virtio net. Based on it, we both think vsock may use virtio net as a transport channel, in this way, vsock can use some of virtio net great features.
Quoted 7 lines
Quoted 2 lines
Thanks,
Yiwen.
What I would do is just copy some code and show a performance benefit. If that works out it will be clearer which code should be shared.
Hi Michael,
I have already sent a series of patches (VSOCK: support mergeable rx buffer in vhost-vsock) a month ago, and the performance as follows:
I write a tool to test the vhost-vsock performance, mainly send big packet(64K) included guest->Host and Host->Guest. The result as follows:
Before performance:
              Single socket            Multiple sockets(Max Bandwidth)
Guest->Host   ~400MB/s                 ~480MB/s
Host->Guest   ~1450MB/s                ~1600MB/s
Quoted 7 lines
After performance:
              Single socket            Multiple sockets(Max Bandwidth)
Guest->Host   ~1700MB/s                ~2900MB/s
Host->Guest   ~1700MB/s                ~2900MB/s
Quoted 1 line
From the test results, the performance is improved obviously, and guest
memory will not be wasted.

Quoted 10 lines

Oh I didn't see that one. Pls CC me in the future.
Looking at it I agree zero page allocation looks like an issue but besides that, I think we can merge something similar and look at refactoring and future extensions later.
However, any interface change (e.g. a new feature) must be CC'd to one of virtio lists (subscriber-only).

Okay, previously I send Virtio-vsock patch only CC stefan and mailing lists based on MAINTAINERS, because it only be related to Virtio-vsock.

Then, first I send v2 patch based on Jason's suggestions, and then let's see how to combine with virtio-vsock and virtio-net. What do you think?

Thanks,
Yiwen.

Quoted 12 lines

Quoted 6 lines
In addition, multiqueue feature I have not implemented it yet.
Thanks,
Yiwen.
.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Michael S. Tsirkin

maintainer

2018-12-04 04:08 UTC #31 lore raw

Quoted 79 lines · jiangyiwen

Quoted 56 lines · Michael S. Tsirkin
Quoted 55 lines · jiangyiwen
Quoted 35 lines · Michael S. Tsirkin
Quoted 34 lines · Jason Wang
Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.
Quoted 9 lines
Quoted 6 lines
- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks
Which code is duplicated between virtio vsock and virtio net right now?
Hi Michael,
AFAIK, there is almost no duplicate code between virtio vsock and virtio net now.
But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, it has some duplicate codes from virtio net. Based on it, we both think vsock may use virtio net as a transport channel, in this way, vsock can use some of virtio net great features.
Quoted 7 lines
Quoted 2 lines
Thanks,
Yiwen.
What I would do is just copy some code and show a performance benefit. If that works out it will be clearer which code should be shared.
Hi Michael,
I have already sent a series of patches (VSOCK: support mergeable rx buffer in vhost-vsock) a month ago, and the performance as follows:
I write a tool to test the vhost-vsock performance, mainly send big packet(64K) included guest->Host and Host->Guest. The result as follows:
Before performance:
              Single socket            Multiple sockets(Max Bandwidth)
Guest->Host   ~400MB/s                 ~480MB/s
Host->Guest   ~1450MB/s                ~1600MB/s

Quoted 7 lines

After performance:
              Single socket            Multiple sockets(Max Bandwidth)
Guest->Host   ~1700MB/s                ~2900MB/s
Host->Guest   ~1700MB/s                ~2900MB/s
Quoted 1 line
From the test results, the performance is improved obviously, and guest
memory will not be wasted.

Oh I didn't see that one. Pls CC me in the future.

Looking at it I agree zero page allocation looks like an issue but besides that, I think we can merge something similar and look at refactoring and future extensions later.

However, any interface change (e.g. a new feature) must be CC'd to one of virtio lists (subscriber-only).

Quoted 5 lines

In addition, multiqueue feature I have not implemented it yet.
Thanks,
Yiwen.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

jiangyiwen

author

2018-12-04 02:21 UTC #32 lore raw

Quoted 56 lines · Michael S. Tsirkin

Quoted 55 lines · jiangyiwen
Quoted 35 lines · Michael S. Tsirkin
Quoted 34 lines · Jason Wang
Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.
Quoted 9 lines
Quoted 6 lines
- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks
Which code is duplicated between virtio vsock and virtio net right now?
Hi Michael,
AFAIK, there is almost no duplicate code between virtio vsock and virtio net now.
But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, it has some duplicate codes from virtio net. Based on it, we both think vsock may use virtio net as a transport channel, in this way, vsock can use some of virtio net great features.

Quoted 7 lines

Quoted 2 lines
Thanks,
Yiwen.
What I would do is just copy some code and show a performance benefit. If that works out it will be clearer which code should be shared.

Hi Michael,

I have already sent a series of patches (VSOCK: support mergeable rx buffer in vhost-vsock) a month ago, and the performance as follows:

I write a tool to test the vhost-vsock performance, mainly send big packet(64K) included guest->Host and Host->Guest. The result as follows:

Before performance:

              Single socket            Multiple sockets(Max Bandwidth)
Guest->Host   ~400MB/s                 ~480MB/s
Host->Guest   ~1450MB/s                ~1600MB/s

After performance:

              Single socket            Multiple sockets(Max Bandwidth)
Guest->Host   ~1700MB/s                ~2900MB/s
Host->Guest   ~1700MB/s                ~2900MB/s

From the test results, the performance is improved obviously, and guest memory will not be wasted.

In addition, multiqueue feature I have not implemented it yet.

Thanks,
Yiwen.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Michael S. Tsirkin

maintainer

2018-12-04 01:31 UTC #33 lore raw

Quoted 50 lines · jiangyiwen

Quoted 35 lines · Michael S. Tsirkin
Quoted 34 lines · Jason Wang
Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.
Quoted 9 lines
Quoted 6 lines
- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks
Which code is duplicated between virtio vsock and virtio net right now?
Hi Michael,
AFAIK, there is almost no duplicate code between virtio vsock and virtio net now.

Quoted 7 lines

But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, it has some duplicate codes from virtio net. Based on it, we both think vsock may use virtio net as a transport channel, in this way, vsock can use some of virtio net great features.
Thanks,
Yiwen.

What I would do is just copy some code and show a performance benefit. If that works out it will be clearer which code should be shared.

Signature · 5 lines

MST
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

jiangyiwen

author

2018-12-03 03:10 UTC #34 lore raw

Quoted 35 lines · Michael S. Tsirkin

Quoted 34 lines · Jason Wang
Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.

Quoted 9 lines

Quoted 6 lines
- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks
Which code is duplicated between virtio vsock and virtio net right now?

Hi Michael,

AFAIK, there is almost no duplicate code between virtio vsock and virtio net now.

Thanks,
Yiwen.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Michael S. Tsirkin

maintainer

2018-11-30 13:40 UTC #35 lore raw

Quoted 34 lines · Jason Wang

Quoted 16 lines · Jason Wang
Quoted 14 lines · Michael S. Tsirkin
Quoted 14 lines
Quoted 8 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between
virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment.
So they are using timers for reliability, and receiver is always free to
discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device
itself. I still don't get why not using virtio-net instead.
Thanks
Quoted 9 lines
Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks
Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:
- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.

Quoted 6 lines

- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).
The change should be totally transparent to userspace applications.
Thanks

Which code is duplicated between virtio vsock and virtio net right now?

Signature · 5 lines

MST
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Jason Wang

2018-11-30 13:10 UTC #36 lore raw

Quoted 12 lines · Jason Wang

Quoted 10 lines · Michael S. Tsirkin
Quoted 10 lines
Quoted 5 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device itself. I still don't get why not using virtio-net instead.
Thanks

Quoted 9 lines

Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks

Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like:

- Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes.

- A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent).

The change should be totally transparent to userspace applications.

Thanks

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Michael S. Tsirkin

maintainer

2018-11-30 13:34 UTC #37 lore raw

Quoted 12 lines · Jason Wang

Quoted 10 lines · Michael S. Tsirkin
Quoted 10 lines
Quoted 5 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device itself. I still don't get why not using virtio-net instead.
Thanks

Quoted 8 lines

Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks

Because apps need reliability, multiplexing and flow control and virtio-net does not provide it.

Signature · 1 line

MST

Michael S. Tsirkin

maintainer

2018-11-30 13:34 UTC #38 lore raw

Quoted 12 lines · Jason Wang

Quoted 10 lines · Michael S. Tsirkin
Quoted 10 lines
Quoted 5 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.
Virtio-net knows nothing above L2, so they are totally transparent to device itself. I still don't get why not using virtio-net instead.
Thanks

Quoted 8 lines

Quoted 4 lines
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?
No, my question is why not do vsock through virtio-net.
Thanks

Because apps need reliability, multiplexing and flow control and virtio-net does not provide it.

Signature · 5 lines

MST
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Jason Wang

2018-11-30 12:55 UTC #39 lore raw

Quoted 5 lines · Michael S. Tsirkin

Quoted 5 lines
Quoted 5 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.

Quoted 9 lines

Quoted 5 lines
Virtio-net knows nothing above L2, so they are totally transparent to device itself. I still don't get why not using virtio-net instead.
Thanks
Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?

No, my question is why not do vsock through virtio-net.

Thanks

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Michael S. Tsirkin

maintainer

2018-11-30 12:52 UTC #40 lore raw

Quoted 48 lines · Jason Wang

Quoted 14 lines · Michael S. Tsirkin
Quoted 13 lines · Jason Wang
Quoted 11 lines · Michael S. Tsirkin
Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock?
Quoted 7 lines
Quoted 1 line
Vsock is just a L2 (and above) protocol from the view of the device.
I don't believe so. I think virtio-vsock operates at a transport level. There is in theory a bit of network level but we don't really implement it as it's only host to guest. I am not aware of any data link functionality n virtio-vsock. virtio-vsock provides services such as connection-oriented communication, reliability, flow control and multiplexing.
Ok, consider it doesn't implement L2, it's pretty fit for virtio-net I believe?
Quoted 6 lines
Quoted 5 lines
So I
think we should answer the question why we need two different paths for networking traffic? Or what is the fundamental reason that makes vsock does not go for virtio-net?
So virtio-vsock ensures reliability.
It's done at the level of protocol instead of virtio transport or virtio device.
Quoted 5 lines
If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.

Quoted 5 lines

Virtio-net knows nothing above L2, so they are totally transparent to device itself. I still don't get why not using virtio-net instead.
Thanks

Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net?

Quoted 6 lines

Quoted 6 lines
Quoted 6 lines
I agree they could be different type of devices but codes could be shared in both guest and host (or even qemu) for not duplicating features(bugs).

Quoted 7 lines

Quoted 7 lines
Quoted 7 lines
Thanks
Quoted 4 lines
The ones that you mention
all seem to be mostly of use to the networking stack.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Jason Wang

2018-11-30 12:45 UTC #41 lore raw

Quoted 14 lines · Michael S. Tsirkin

Quoted 13 lines · Jason Wang
Quoted 11 lines · Michael S. Tsirkin
Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock?

Quoted 7 lines

Quoted 1 line
Vsock is just a L2 (and above) protocol from the view of the device.
I don't believe so. I think virtio-vsock operates at a transport level. There is in theory a bit of network level but we don't really implement it as it's only host to guest. I am not aware of any data link functionality n virtio-vsock. virtio-vsock provides services such as connection-oriented communication, reliability, flow control and multiplexing.

Ok, consider it doesn't implement L2, it's pretty fit for virtio-net I believe?

Quoted 6 lines

Quoted 5 lines
So I
think we should answer the question why we need two different paths for networking traffic? Or what is the fundamental reason that makes vsock does not go for virtio-net?
So virtio-vsock ensures reliability.

It's done at the level of protocol instead of virtio transport or virtio device.

Quoted 5 lines

If you want to compare it with
something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data.

Virtio-net knows nothing above L2, so they are totally transparent to device itself. I still don't get why not using virtio-net instead.

Thanks

Quoted 5 lines

Quoted 5 lines
I agree they could be different type of devices but codes could be shared in both guest and host (or even qemu) for not duplicating features(bugs).

Quoted 7 lines

Quoted 7 lines
Thanks
Quoted 4 lines
The ones that you mention
all seem to be mostly of use to the networking stack.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Michael S. Tsirkin

maintainer

2018-11-29 14:00 UTC #42 lore raw

Quoted 11 lines · Jason Wang

Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.

Quoted 5 lines

Quoted 2 lines
I'm not sure I get the motivation. Which features of virtio net are relevant to vsock?
Vsock is just a L2 (and above) protocol from the view of the device.

Quoted 4 lines

So I
think we should answer the question why we need two different paths for networking traffic? Or what is the fundamental reason that makes vsock does not go for virtio-net?

Quoted 10 lines

I agree they could be different type of devices but codes could be shared in both guest and host (or even qemu) for not duplicating features(bugs).
Thanks
Quoted 4 lines
The ones that you mention
all seem to be mostly of use to the networking stack.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Michael S. Tsirkin

maintainer

2018-11-15 07:04 UTC #43 lore raw

Quoted 8 lines · jiangyiwen

Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.

I'm not sure I get the motivation. Which features of virtio net are relevant to vsock? The ones that you mention all seem to be mostly of use to the networking stack.

Quoted 7 lines

Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.
After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(

Quoted 7 lines

-------------------------Simple idea------------------------------
1. The packet layout will become as follows:
+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|

Which fields in virtio_net_hdr_mrg_rxbuf are of interest to vsock?

Quoted 7 lines

+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+

Thanks,

Signature · 1 line

MST

Yan Vugenfirer

2018-11-15 09:36 UTC #44 lore raw

In reply to Yan Vugenfirer

Quoted 12 lines · jiangyiwen

Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.

I would like to add that from Windows guest support perspective it makes more sense.
To support vsock we most probably need NDIS protocol driver and if it will be binded to NDIS miniport driver (the driver that currently handles virtio-net in Windows) it will make our life much easier. Otherwise we need to “plug” into NIDS on one hand, and handle transport outside of NDIS on other hand or in some strange way by creating mini port driver that is not really NIC miniport driver.

Best regards,
Yan.

Quoted 73 lines

Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.

After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(

-------------------------Simple idea------------------------------

1. The packet layout will become as follows:

+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+

2. The Guest->Host basic code flow as follow:

                           +------------+
                           |   Client   |
                           +------------+
                                 |
                                 |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|ops->sendmsg; (vsock_stream_sendmsg)                              |
|  -> alloc_skb; /* it will packet a skb buffer, and include vsock |
|                 * hdr and payload */                             |
|  -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */|
|vsock hdr and payload, and then call                              |
+------------------------------------------------------------------+
                                 |
                                 |
+------------------------------------------------------------------+
|Virtio-net Module                                                 |
|start_xmit                                                        |
|  -> add virtio_net_hdr and pack sg in ring desc, notify Host     |
+------------------------------------------------------------------+
                                 |
                                 |
+------------------------------------------------------------------+
|Vhost-net Module                                                  |
|handle_tx                                                         |
|  -> get tx buffer, skip virtio_net_hdr and call Vsock function.  |
| /* This point has some differences, vhost-net use ->sendmsg to   |
|  * forward information, however vsock only need to notify server |
|  * that data ready. */                                           |
+------------------------------------------------------------------+
                                 |
                                 |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|alloc_pkt, copy skb data to pkt.                                  |
|add pkt to rx_queue and notify server to get data.                |
+------------------------------------------------------------------+

3. To Host->Guest
I have a problem and difficult, mainly I know about virtio-net a little), because I have been doing work related with storage and file system.

jiangyiwen

author

2018-11-16 02:37 UTC #45 lore raw

Quoted 13 lines · Yan Vugenfirer

Quoted 12 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.

Quoted 5 lines

I would like to add that from Windows guest support perspective it makes more sense.
To support vsock we most probably need NDIS protocol driver and if it will be binded to NDIS miniport driver (the driver that currently handles virtio-net in Windows) it will make our life much easier. Otherwise we need to “plug” into NIDS on one hand, and handle transport outside of NDIS on other hand or in some strange way by creating mini port driver that is not really NIC miniport driver.
Best regards,
Yan.

Sorry, I am not familiar with windows vsock driver, if vsock is integrated in virtio-net, it can make windows vsock more easier, then it should be a more suitable solution.

Thanks.

Quoted 77 lines

Quoted 66 lines

Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.

After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(

-------------------------Simple idea------------------------------

1. The packet layout will become as follows:

+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+

2. The Guest->Host basic code flow as follow:

                           +------------+
                           |   Client   |
                           +------------+
                                 |
                                 |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|ops->sendmsg; (vsock_stream_sendmsg)                              |
|  -> alloc_skb; /* it will packet a skb buffer, and include vsock |
|                 * hdr and payload */                             |
|  -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */|
|vsock hdr and payload, and then call                              |
+------------------------------------------------------------------+
                                 |
                                 |
+------------------------------------------------------------------+
|Virtio-net Module                                                 |
|start_xmit                                                        |
|  -> add virtio_net_hdr and pack sg in ring desc, notify Host     |
+------------------------------------------------------------------+
                                 |
                                 |
+------------------------------------------------------------------+
|Vhost-net Module                                                  |
|handle_tx                                                         |
|  -> get tx buffer, skip virtio_net_hdr and call Vsock function.  |
| /* This point has some differences, vhost-net use ->sendmsg to   |
|  * forward information, however vsock only need to notify server |
|  * that data ready. */                                           |
+------------------------------------------------------------------+
                                 |
                                 |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|alloc_pkt, copy skb data to pkt.                                  |
|add pkt to rx_queue and notify server to get data.                |
+------------------------------------------------------------------+

3. To Host->Guest
I have a problem and difficult, mainly I know about virtio-net a little), because I have been doing work related with storage and file system.

Quoted 7 lines

The problem as follows:
we should monitor all of socket of vsock in handle_rx, when there are data coming, and copy data to vq desc. Vhost-net use ->recvmsg to get data, it is different with socket. To vsock, I think host will not call ->recvmsg when it need to send message to guest. To net, vhost-net only as forwarding layer.

jiangyiwen

author

2018-11-16 02:37 UTC #46 lore raw

Quoted 13 lines · Yan Vugenfirer

Quoted 12 lines · jiangyiwen
Hi Stefan, Michael, Jason and everyone,
Several days ago, I discussed with jason about "Vsock over Virtio-net". This idea has two advantages:
First, it can use many great features of virtio-net, like batching, mergeable rx buffer and multiqueue, etc.
Second, it can reduce many duplicate codes and make it easy to be maintained.

Quoted 5 lines

I would like to add that from Windows guest support perspective it makes more sense.
To support vsock we most probably need NDIS protocol driver and if it will be binded to NDIS miniport driver (the driver that currently handles virtio-net in Windows) it will make our life much easier. Otherwise we need to “plug” into NIDS on one hand, and handle transport outside of NDIS on other hand or in some strange way by creating mini port driver that is not really NIC miniport driver.
Best regards,
Yan.

Sorry, I am not familiar with windows vsock driver, if vsock is integrated in virtio-net, it can make windows vsock more easier, then it should be a more suitable solution.

Thanks.

Quoted 75 lines

Quoted 66 lines

Before the implement, I want to discuss with everyone again, and want to know everyone's suggestions.

After the discussion, based on this point I will try to implement this idea, but I am not familiar with the virtio-net, that is a pity. :(

-------------------------Simple idea------------------------------

1. The packet layout will become as follows:

+---------------------------------+
|        Virtio-net header        |
|(struct virtio_net_hdr_mrg_rxbuf)|
+---------------------------------+
|          Vsock header           |
|    (struct virtio_vsock_hdr)    |
+---------------------------------+
|             payload             |
|      (until end of packet)      |
+---------------------------------+

2. The Guest->Host basic code flow as follow:

                           +------------+
                           |   Client   |
                           +------------+
                                 |
                                 |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|ops->sendmsg; (vsock_stream_sendmsg)                              |
|  -> alloc_skb; /* it will packet a skb buffer, and include vsock |
|                 * hdr and payload */                             |
|  -> dev_queue_xmit(); /* it will call start_xmit(virtio-net.c) */|
|vsock hdr and payload, and then call                              |
+------------------------------------------------------------------+
                                 |
                                 |
+------------------------------------------------------------------+
|Virtio-net Module                                                 |
|start_xmit                                                        |
|  -> add virtio_net_hdr and pack sg in ring desc, notify Host     |
+------------------------------------------------------------------+
                                 |
                                 |
+------------------------------------------------------------------+
|Vhost-net Module                                                  |
|handle_tx                                                         |
|  -> get tx buffer, skip virtio_net_hdr and call Vsock function.  |
| /* This point has some differences, vhost-net use ->sendmsg to   |
|  * forward information, however vsock only need to notify server |
|  * that data ready. */                                           |
+------------------------------------------------------------------+
                                 |
                                 |
+------------------------------------------------------------------+
|VSOCK Core Module                                                 |
|alloc_pkt, copy skb data to pkt.                                  |
|add pkt to rx_queue and notify server to get data.                |
+------------------------------------------------------------------+

3. To Host->Guest
I have a problem and difficult, mainly I know about virtio-net a little), because I have been doing work related with storage and file system.

Quoted 7 lines

The problem as follows:
we should monitor all of socket of vsock in handle_rx, when there are data coming, and copy data to vq desc. Vhost-net use ->recvmsg to get data, it is different with socket. To vsock, I think host will not call ->recvmsg when it need to send message to guest. To net, vhost-net only as forwarding layer.

Quoted 2 lines

.

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Yan Vugenfirer

2018-11-15 09:36 UTC #47 lore raw