Skip to contentSkip to conversation

vhost_net: a kernel-level virtio server

PATCH In review Rusty Russell 1 participant 1 message Dec 8, 2009

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

PATCH 3/3 vhost_net: a kernel-level virtio server

From: "Michael S. Tsirkin" <[email protected]>

What it is: vhost net is a character device that can be used to reduce the number of system calls involved in virtio networking. Existing virtio net code is used in the guest without modification.

There's similarity with vringfd, with some differences and reduced scope
- uses eventfd for signalling
- structures can be moved around in memory at any time (good for migration, bug work-arounds in userspace)
- write logging is supported (good for migration)
- support memory table and not just an offset (needed for kvm)

common virtio related code has been put in a separate file vhost.c and can be made into a separate module if/when more backends appear. I used Rusty's lguest.c as the source for developing this part : this supplied me with witty comments I wouldn't be able to write myself.

What it is not: vhost net is not a bus, and not a generic new system call. No assumptions are made on how guest performs hypercalls. Userspace hypervisors are supported as well as kvm.

How it works: Basically, we connect virtio frontend (configured by userspace) to a backend. The backend could be a network device, or a tap device. Backend is also configured by userspace, including vlan/mac etc.

Status: This works for me, and I haven't see any crashes. Compared to userspace, people reported improved latency (as I save up to 4 system calls per packet), as well as better bandwidth and CPU utilization.

Features that I plan to look at in the future:
- mergeable buffers
- zero copy
- scalability tuning: figure out the best threading model to use

Note on RCU usage (this is also documented in vhost.h, near private_pointer which is the value protected by this variant of RCU):
what is happening is that the rcu_dereference() is being used in a workqueue item. The role of rcu_read_lock() is taken on by the start of execution of the workqueue item, of rcu_read_unlock() by the end of execution of the workqueue item, and of synchronize_rcu() by flush_workqueue()/flush_work(). In the future we might need to apply some gcc attribute or sparse annotation to the function passed to INIT_WORK(). Paul's ack below is for this RCU usage.

(Includes fixes by Alan Cox <[email protected]>)

---
 MAINTAINERS                |    9 
 arch/x86/kvm/Kconfig       |    1 
 drivers/Makefile           |    1 
 drivers/vhost/Kconfig      |   11 
 drivers/vhost/Makefile     |    2 
 drivers/vhost/net.c        |  648 ++++++++++++++++++++++++++++++
 drivers/vhost/vhost.c      |  968 +++++++++++++++++++++++++++++++++++++++++++++
 drivers/vhost/vhost.h      |  159 +++++++
 include/linux/Kbuild       |    1 
 include/linux/miscdevice.h |    1 
 include/linux/vhost.h      |  130 ++++++
 11 files changed, 1931 insertions(+)
MAINTAINERS +9 -0
@@ -5665,6 +5665,15 @@ S:	Maintained
 F:	Documentation/filesystems/vfat.txt
 F:	fs/fat/
 
+VIRTIO HOST (VHOST)
+M:	"Michael S. Tsirkin" <[email protected]>
+L:	[email protected]
+L:	[email protected]
+L:	[email protected]
+S:	Maintained
+F:	drivers/vhost/
+F:	include/linux/vhost.h
+
 VIA RHINE NETWORK DRIVER
 M:	Roger Luethi <[email protected]>
 S:	Maintained
arch/x86/kvm/Kconfig +1 -0
@@ -64,6 +64,7 @@ config KVM_AMD
 
 # OK, it's a little counter-intuitive to do this, but it puts it neatly under
 # the virtualization menu.
+source drivers/vhost/Kconfig
 source drivers/lguest/Kconfig
 source drivers/virtio/Kconfig
 
drivers/Makefile +1 -0
@@ -106,6 +106,7 @@ obj-$(CONFIG_HID)		+= hid/
 obj-$(CONFIG_PPC_PS3)		+= ps3/
 obj-$(CONFIG_OF)		+= of/
 obj-$(CONFIG_SSB)		+= ssb/
+obj-$(CONFIG_VHOST_NET)		+= vhost/
 obj-$(CONFIG_VIRTIO)		+= virtio/
 obj-$(CONFIG_VLYNQ)		+= vlynq/
 obj-$(CONFIG_STAGING)		+= staging/
drivers/vhost/Kconfig new file +11 -0
new file mode 100644
@@ -0,0 +1,11 @@
+config VHOST_NET
+	tristate "Host kernel accelerator for virtio net (EXPERIMENTAL)"
+	depends on NET && EVENTFD && (TUN || !TUN) && EXPERIMENTAL
+	---help---
+	  This kernel module can be loaded in host kernel to accelerate
+	  guest networking with virtio_net. Not to be confused with virtio_net
+	  module itself which needs to be loaded in guest kernel.
+
+	  To compile this driver as a module, choose M here: the module will
+	  be called vhost_net.
+
drivers/vhost/Makefile new file +2 -0
new file mode 100644
@@ -0,0 +1,2 @@
+obj-$(CONFIG_VHOST_NET) += vhost_net.o
+vhost_net-y := vhost.o net.o
drivers/vhost/net.c new file +648 -0

648 lines not on this page · raw message

drivers/vhost/vhost.c new file +968 -0

968 lines not on this page · raw message

drivers/vhost/vhost.h new file +159 -0

159 lines not on this page · raw message

include/linux/Kbuild +1 -0
@@ -362,6 +362,7 @@ unifdef-y += uio.h
 unifdef-y += unistd.h
 unifdef-y += usbdevice_fs.h
 unifdef-y += utsname.h
+unifdef-y += vhost.h
 unifdef-y += videodev2.h
 unifdef-y += videodev.h
 unifdef-y += virtio_config.h
include/linux/miscdevice.h +1 -0
@@ -30,6 +30,7 @@
 #define HPET_MINOR		228
 #define FUSE_MINOR		229
 #define KVM_MINOR		232
+#define VHOST_NET_MINOR		233
 #define MISC_DYNAMIC_MINOR	255
 
 struct device;
include/linux/vhost.h new file +130 -0

130 lines not on this page · raw message