Is fallback vhost_net to qemu for live migrate available？

Hi all

I am participating in a project which try to port vhost_net on Xen。

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。TCP receive throughput of single vnic from 2.77Gbps up to 6Gps。In VM receive side，I instead grant_copy with grant_map + memcopy，it efficiently reduce the cost of grant_table spin_lock of dom0，So the hole server TCP performance from 5.33Gps up to 9.5Gps。

Now I am consider the live migrate of vhost_net on Xen，vhost_net use vhost_log for live migrate on Kvm，but qemu on Xen havn't manage the hole memory of VM，So I am trying to fallback datapath from vhost_net to qemu when doing live migrate ，and fallback datapath from qemu to
vhost_net again after vm migrate to new server。

My question is：
why didn't vhost_net do the same fallback operation for live
migrate on KVM，but use vhost_log to mark the dirty page？
Is there any mechanism fault for the idea of fallback datapath from
vhost_net to qemu for live migrate？

any question about the detail of vhost_net on Xen is welcome。

Thanks

Quoted 24 lines · Qin Chuanyu

Hi all

I am participating in a project which try to port vhost_net on Xen。

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。TCP receive throughput of single vnic from 2.77Gbps up to 6Gps。In VM receive side，I instead grant_copy with grant_map + memcopy，it efficiently reduce the cost of grant_table spin_lock of dom0，So the hole server TCP performance from 5.33Gps up to 9.5Gps。

Now I am consider the live migrate of vhost_net on Xen，vhost_net use vhost_log for live migrate on Kvm，but qemu on Xen havn't manage the hole memory of VM，So I am trying to fallback datapath from vhost_net to qemu when doing live migrate ，and fallback datapath from qemu to
vhost_net again after vm migrate to new server。

My question is：
why didn't vhost_net do the same fallback operation for live
migrate on KVM，but use vhost_log to mark the dirty page？
Is there any mechanism fault for the idea of fallback datapath from
vhost_net to qemu for live migrate？

Quoted 4 lines

any question about the detail of vhost_net on Xen is welcome。

Thanks

It should work, in practice.

However, one issue with this approach that I see is that you are running two instances of virtio-net on the host:
qemu and vhost-net, doubling your security surface for guest to host attack.

I don't exactly see why does it matter that qemu doesn't manage the whole memory of a VM - vhost only needs to log memory writes that it performs.

.

Quoted 37 lines · Michael S. Tsirkin

Quoted 24 lines · Qin Chuanyu

Hi all

I am participating in a project which try to port vhost_net on Xen。

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。TCP receive throughput of single vnic from 2.77Gbps up to 6Gps。In VM receive side，I instead grant_copy with grant_map + memcopy，it efficiently reduce the cost of grant_table spin_lock of dom0，So the hole server TCP performance from 5.33Gps up to 9.5Gps。

Now I am consider the live migrate of vhost_net on Xen，vhost_net use vhost_log for live migrate on Kvm，but qemu on Xen havn't manage the hole memory of VM，So I am trying to fallback datapath from vhost_net to qemu when doing live migrate ，and fallback datapath from qemu to
vhost_net again after vm migrate to new server。

My question is：
why didn't vhost_net do the same fallback operation for live
migrate on KVM，but use vhost_log to mark the dirty page？
Is there any mechanism fault for the idea of fallback datapath from
vhost_net to qemu for live migrate？

Quoted 4 lines

any question about the detail of vhost_net on Xen is welcome。

Thanks

It should work, in practice.

However, one issue with this approach that I see is that you are running two instances of virtio-net on the host:
qemu and vhost-net, doubling your security surface for guest to host attack.

Quoted 7 lines

I don't exactly see why does it matter that qemu doesn't manage the whole memory of a VM - vhost only needs to log memory writes that it performs.

.

Thanks for your reply.

In fact，I am not sure that whether vhost_log could work on Xen live migrate or not. Yes，vhost_sync_dirty_bitmap work well on Kvm，but vhost_net havn't run on Xen before，although the method vhost_dev_sync_region -> memory_region_set_dirty -> xen_modified_memory call sequence exist.

Have you considered the scene that vhost_migration_log code running on Xen？ If yes，it sames much easier then fallback datapath from vhost_net to qemu for live migrate.

Quoted 53 lines · Qin Chuanyu

Quoted 37 lines · Michael S. Tsirkin

Quoted 24 lines · Qin Chuanyu

Hi all

I am participating in a project which try to port vhost_net on Xen。

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。TCP receive throughput of single vnic from 2.77Gbps up to 6Gps。In VM receive side，I instead grant_copy with grant_map + memcopy，it efficiently reduce the cost of grant_table spin_lock of dom0，So the hole server TCP performance from 5.33Gps up to 9.5Gps。

Now I am consider the live migrate of vhost_net on Xen，vhost_net use vhost_log for live migrate on Kvm，but qemu on Xen havn't manage the hole memory of VM，So I am trying to fallback datapath from vhost_net to qemu when doing live migrate ，and fallback datapath from qemu to
vhost_net again after vm migrate to new server。

My question is：
why didn't vhost_net do the same fallback operation for live
migrate on KVM，but use vhost_log to mark the dirty page？
Is there any mechanism fault for the idea of fallback datapath from
vhost_net to qemu for live migrate？

Quoted 4 lines

any question about the detail of vhost_net on Xen is welcome。

Thanks

It should work, in practice.

However, one issue with this approach that I see is that you are running two instances of virtio-net on the host:
qemu and vhost-net, doubling your security surface for guest to host attack.

Quoted 7 lines

I don't exactly see why does it matter that qemu doesn't manage the whole memory of a VM - vhost only needs to log memory writes that it performs.

.

Thanks for your reply.

In fact，I am not sure that whether vhost_log could work on Xen live migrate or not. Yes，vhost_sync_dirty_bitmap work well on Kvm，but vhost_net havn't run on Xen before，although the method vhost_dev_sync_region -> memory_region_set_dirty -> xen_modified_memory call sequence exist.

Quoted 4 lines

Have you considered the scene that vhost_migration_log code running on Xen？ If yes，it sames much easier then fallback datapath from vhost_net to qemu for live migrate.

I never looked at how Xen live migration works with QEMU so I don't know.

.

Quoted 2 lines

.

Thanks for your reply.

In fact，I am not sure that whether vhost_log could work on Xen live migrate or not. Yes，vhost_sync_dirty_bitmap work well on Kvm，but vhost_net havn't run on Xen before，although the method vhost_dev_sync_region -> memory_region_set_dirty -> xen_modified_memory call sequence exist.

Have you considered the scene that vhost_migration_log code running on Xen？ If yes，it sames much easier then fallback datapath from vhost_net to qemu for live migrate.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi all

I am participating in a project which try to port vhost_net on Xen。

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

TCP receive throughput of
single vnic from 2.77Gbps up to 6Gps。In VM receive side，I instead grant_copy with grant_map + memcopy，it efficiently reduce the cost of grant_table spin_lock of dom0，So the hole server TCP performance from 5.33Gps up to 9.5Gps。

Now I am consider the live migrate of vhost_net on Xen，vhost_net use vhost_log for live migrate on Kvm，but qemu on Xen havn't manage the hole memory of VM，So I am trying to fallback datapath from vhost_net to qemu when doing live migrate ，and fallback datapath from qemu to
vhost_net again after vm migrate to new server。

My question is：
why didn't vhost_net do the same fallback operation for live migrate
on KVM，but use vhost_log to mark the dirty page？
Is there any mechanism fault for the idea of fallback datapath from
vhost_net to qemu for live migrate？

any question about the detail of vhost_net on Xen is welcome。

Thanks

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi all

I am participating in a project which try to port vhost_net on Xen。

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

TCP receive throughput of
single vnic from 2.77Gbps up to 6Gps。In VM receive side，I instead grant_copy with grant_map + memcopy，it efficiently reduce the cost of grant_table spin_lock of dom0，So the hole server TCP performance from 5.33Gps up to 9.5Gps。

Now I am consider the live migrate of vhost_net on Xen，vhost_net use vhost_log for live migrate on Kvm，but qemu on Xen havn't manage the hole memory of VM，So I am trying to fallback datapath from vhost_net to qemu when doing live migrate ，and fallback datapath from qemu to
vhost_net again after vm migrate to new server。

My question is：
why didn't vhost_net do the same fallback operation for live migrate
on KVM，but use vhost_log to mark the dirty page？
Is there any mechanism fault for the idea of fallback datapath from
vhost_net to qemu for live migrate？

any question about the detail of vhost_net on Xen is welcome。

Thanks

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

KVM and Xen represent memory in a very different way. KVM can only track when guest mode code dirties memory. It relies on QEMU to track when guest memory is dirtied by QEMU. Since vhost is running outside of QEMU, vhost also needs to tell QEMU when it has dirtied memory.

I don't think this is a problem with Xen though. I believe (although could be wrong) that Xen is able to track when either the domain or dom0 dirties memory.

So I think you can simply ignore the dirty logging with vhost and it should Just Work.

No, we don't have a mechanism to fallback to QEMU for the datapath. It would be possible but I think it's a bad idea to mix and match the two.

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

KVM and Xen represent memory in a very different way. KVM can only track when guest mode code dirties memory. It relies on QEMU to track when guest memory is dirtied by QEMU. Since vhost is running outside of QEMU, vhost also needs to tell QEMU when it has dirtied memory.

I don't think this is a problem with Xen though. I believe (although could be wrong) that Xen is able to track when either the domain or dom0 dirties memory.

So I think you can simply ignore the dirty logging with vhost and it should Just Work.

No, we don't have a mechanism to fallback to QEMU for the datapath. It would be possible but I think it's a bad idea to mix and match the two.

Quoted 11 lines · Anthony Liguori

Hi Qin,

Quoted 3 lines

By change the memory copy and notify mechanism ，currently virtio-net with
vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

Quoted 5 lines

KVM and Xen represent memory in a very different way. KVM can only track when guest mode code dirties memory. It relies on QEMU to track when guest memory is dirtied by QEMU. Since vhost is running outside of QEMU, vhost also needs to tell QEMU when it has dirtied memory.

Quoted 7 lines

I don't think this is a problem with Xen though. I believe (although could be wrong) that Xen is able to track when either the domain or dom0 dirties memory.

So I think you can simply ignore the dirty logging with vhost and it should Just Work.

Thanks for your advice, I have tried it, without ping, it could migrate successfully, but if there has skb been received, domU would crash. I guess that because though Xen track domU memory, but it could only track memory that changed in DomU. memory changed by Dom0 is out of control.

Quoted 5 lines

No, we don't have a mechanism to fallback to QEMU for the datapath. It would be possible but I think it's a bad idea to mix and match the two.

Next I would try to fallback datapath to qemu for three reason:
1: memory translate mechanism has been changed for vhost_net on Xen，so there would be some necessary changed needed for vhost_log in kernel.

2: I also maped IOREQ_PFN page(which is used for communication between qemu and Xen) in kernel notify module, so it also needed been marked as dirty when tx/rx exist in migrate period.

3: Most important of all， Michael S. Tsirkin said that he hadn't considered about vhost_net migrate on Xen，so there would be some changed needed in vhost_log for qemu.

fallback to qemu seems to much easier, isn't it.

Regards
Qin chuanyu

Quoted 11 lines · Anthony Liguori

Hi Qin,

Quoted 3 lines

By change the memory copy and notify mechanism ，currently virtio-net with
vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

Quoted 5 lines

KVM and Xen represent memory in a very different way. KVM can only track when guest mode code dirties memory. It relies on QEMU to track when guest memory is dirtied by QEMU. Since vhost is running outside of QEMU, vhost also needs to tell QEMU when it has dirtied memory.

Quoted 7 lines

I don't think this is a problem with Xen though. I believe (although could be wrong) that Xen is able to track when either the domain or dom0 dirties memory.

So I think you can simply ignore the dirty logging with vhost and it should Just Work.

Thanks for your advice, I have tried it, without ping, it could migrate successfully, but if there has skb been received, domU would crash. I guess that because though Xen track domU memory, but it could only track memory that changed in DomU. memory changed by Dom0 is out of control.

Quoted 5 lines

No, we don't have a mechanism to fallback to QEMU for the datapath. It would be possible but I think it's a bad idea to mix and match the two.

Next I would try to fallback datapath to qemu for three reason:
1: memory translate mechanism has been changed for vhost_net on Xen，so there would be some necessary changed needed for vhost_log in kernel.

2: I also maped IOREQ_PFN page(which is used for communication between qemu and Xen) in kernel notify module, so it also needed been marked as dirty when tx/rx exist in migrate period.

3: Most important of all， Michael S. Tsirkin said that he hadn't considered about vhost_net migrate on Xen，so there would be some changed needed in vhost_log for qemu.

fallback to qemu seems to much easier, isn't it.

Regards
Qin chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

Quoted 15 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 7 lines

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

Wei.

Quoted 15 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 7 lines

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

Wei.

Quoted 15 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 7 lines

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

Quoted 15 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 7 lines

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

Quoted 24 lines · Wei Liu

Quoted 22 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

Quoted 5 lines

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

In fact, I did what virtio-net have done in Qemu. I think security is a pseudo question because Dom0 is under control.

Host could access memory of guest in KVM much easier than Xen, but I hadn't heard someone said KVM is un-secret.

Regards
Qin chuanyu

--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 24 lines · Wei Liu

Quoted 22 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

Quoted 5 lines

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

In fact, I did what virtio-net have done in Qemu. I think security is a pseudo question because Dom0 is under control.

Host could access memory of guest in KVM much easier than Xen, but I hadn't heard someone said KVM is un-secret.

Regards
Qin chuanyu

--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 30 lines · Qin Chuanyu

Quoted 29 lines · Wei Liu

Quoted 15 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 7 lines

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

Quoted 3 lines

In fact, I did what virtio-net have done in Qemu. I think security is a pseudo question because Dom0 is under control.

Consider that you might have driver domains. Not every domain is under control or trusted.

Also consider that security model like XSM can be
used to audit operations to enhance security so your foreign mapping approach might not always work.

In short term foreign mapping can save you some time implementing the prototype.
In long term using grant table is the proper way to go. And IMHO the benifit outweights the cost.

Wei.

Quoted 11 lines

Host could access memory of guest in KVM much easier than Xen, but I hadn't heard someone said KVM is un-secret.

Regards
Qin chuanyu

--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 30 lines · Qin Chuanyu

Quoted 29 lines · Wei Liu

Quoted 15 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 7 lines

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

Quoted 3 lines

In fact, I did what virtio-net have done in Qemu. I think security is a pseudo question because Dom0 is under control.

Consider that you might have driver domains. Not every domain is under control or trusted.

Also consider that security model like XSM can be
used to audit operations to enhance security so your foreign mapping approach might not always work.

In short term foreign mapping can save you some time implementing the prototype.
In long term using grant table is the proper way to go. And IMHO the benifit outweights the cost.

Wei.

Quoted 11 lines

Host could access memory of guest in KVM much easier than Xen, but I hadn't heard someone said KVM is un-secret.

Regards
Qin chuanyu

--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 31 lines · Wei Liu

Quoted 30 lines · Qin Chuanyu

Quoted 29 lines · Wei Liu

Quoted 15 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 7 lines

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

Quoted 6 lines

Quoted 3 lines

In fact, I did what virtio-net have done in Qemu. I think security is a pseudo question because Dom0 is under control.

Consider that you might have driver domains. Not every domain is under control or trusted.

I don't see anything that will prevent using driver domains here.

Quoted 3 lines

Also consider that security model like XSM can be
used to audit operations to enhance security so your foreign mapping approach might not always work.

It could be nice to have as an option, sure.
XSM is disabled by default though so I don't think lack of support for that makes it a prototype.

Quoted 6 lines

In short term foreign mapping can save you some time implementing the prototype.
In long term using grant table is the proper way to go. And IMHO the benifit outweights the cost.

Wei.

I'm guessing direct access could be quite a bit faster. But someone would have to implement your idea in order to do a cost/benefit analysis.

Quoted 11 lines

Quoted 11 lines

Host could access memory of guest in KVM much easier than Xen, but I hadn't heard someone said KVM is un-secret.

Regards
Qin chuanyu

--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 31 lines · Wei Liu

Quoted 30 lines · Qin Chuanyu

Quoted 29 lines · Wei Liu

Quoted 15 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 7 lines

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

Quoted 6 lines

Quoted 3 lines

In fact, I did what virtio-net have done in Qemu. I think security is a pseudo question because Dom0 is under control.

Consider that you might have driver domains. Not every domain is under control or trusted.

I don't see anything that will prevent using driver domains here.

Quoted 3 lines

Also consider that security model like XSM can be
used to audit operations to enhance security so your foreign mapping approach might not always work.

It could be nice to have as an option, sure.
XSM is disabled by default though so I don't think lack of support for that makes it a prototype.

Quoted 6 lines

In short term foreign mapping can save you some time implementing the prototype.
In long term using grant table is the proper way to go. And IMHO the benifit outweights the cost.

Wei.

I'm guessing direct access could be quite a bit faster. But someone would have to implement your idea in order to do a cost/benefit analysis.

Quoted 11 lines

Quoted 11 lines

Host could access memory of guest in KVM much easier than Xen, but I hadn't heard someone said KVM is un-secret.

Regards
Qin chuanyu

--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 26 lines · Wei Liu

Quoted 25 lines · Qin Chuanyu

Quoted 24 lines · Wei Liu

Quoted 22 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

Quoted 7 lines

Quoted 7 lines

Quoted 5 lines

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

In fact, I did what virtio-net have done in Qemu. I think security is a pseudo question because Dom0 is under control.

Quoted 2 lines

Consider that you might have driver domains. Not every domain is under control or trusted.

I don't see anything that will prevent using driver domains here.

Quoted 3 lines

Also consider that security model like XSM can be
used to audit operations to enhance security so your foreign mapping approach might not always work.

It could be nice to have as an option, sure.
XSM is disabled by default though so I don't think lack of support for that makes it a prototype.

Quoted 26 lines · Wei Liu

Quoted 25 lines · Qin Chuanyu

Quoted 24 lines · Wei Liu

Quoted 22 lines · Qin Chuanyu

Quoted 10 lines · Anthony Liguori

Hi Qin,

Quoted 2 lines

By change the memory copy and notify mechanism ，currently virtio-net with vhost_net could run on Xen with good performance。

I think the key in doing this would be to implement a property ioeventfd and irqfd interface in the driver domain kernel. Just hacking vhost_net with Xen specific knowledge would be pretty nasty IMHO.

Yes, I add a kernel module which persist virtio-net pio_addr and msix address as what kvm module did. Guest wake up vhost thread by adding a hook func in evtchn_interrupt.

Quoted 3 lines

Did you modify the front end driver to do grant table mapping or is this all being done by mapping the domain's memory?

There is nothing changed in front end driver. Currently I use alloc_vm_area to get address space， and map the domain's memory as what what qemu did.

Quoted 7 lines

Quoted 7 lines

Quoted 5 lines

You mean you're using xc_map_foreign_range and friends in the backend to map guest memory? That's not very desirable as it violates Xen's security model. It would not be too hard to pass grant references instead of guest physical memory address IMHO.

In fact, I did what virtio-net have done in Qemu. I think security is a pseudo question because Dom0 is under control.

Quoted 2 lines

Consider that you might have driver domains. Not every domain is under control or trusted.

I don't see anything that will prevent using driver domains here.

Quoted 3 lines

Also consider that security model like XSM can be
used to audit operations to enhance security so your foreign mapping approach might not always work.

It could be nice to have as an option, sure.
XSM is disabled by default though so I don't think lack of support for that makes it a prototype.

Hi Qin,

KVM and Xen represent memory in a very different way. KVM can only track when guest mode code dirties memory. It relies on QEMU to track when guest memory is dirtied by QEMU. Since vhost is running outside of QEMU, vhost also needs to tell QEMU when it has dirtied memory.

I don't think this is a problem with Xen though. I believe (although could be wrong) that Xen is able to track when either the domain or dom0 dirties memory.

So I think you can simply ignore the dirty logging with vhost and it should Just Work.

Hi Qin,

KVM and Xen represent memory in a very different way. KVM can only track when guest mode code dirties memory. It relies on QEMU to track when guest memory is dirtied by QEMU. Since vhost is running outside of QEMU, vhost also needs to tell QEMU when it has dirtied memory.

I don't think this is a problem with Xen though. I believe (although could be wrong) that Xen is able to track when either the domain or dom0 dirties memory.

So I think you can simply ignore the dirty logging with vhost and it should Just Work.