VM Sockets for Linux upstreaming

From: Andy King <[email protected]>

VM Sockets allows communication between virtual machines and the hypervisor. User level applications both in a virtual machine and on the host can use the VM Sockets API, which facilitates fast and efficient communication between guest virtual machines and their host. A socket address family, designed to be compatible with UDP and TCP at the interface level, is provided.

Today, VM Sockets is used by various VMware Tools components inside the guest for zero-config, network-less access to VMware host services. In addition to this, VMware's users are using VM Sockets for various applications, where network access of the virtual machine is restricted or non-existent. Examples of this are VMs communicating with device proxies for proprietary hardware running as host applications and automated testing of applications running within virtual machines.

The VMware VM Sockets are similar to other socket types, like Berkeley UNIX socket interface. The VM Sockets module supports both connection-oriented stream sockets like TCP, and connectionless datagram sockets like UDP. The VM Sockets protocol family is defined as "AF_VSOCK" and the socket operations split for SOCK_DGRAM and SOCK_STREAM.

For additional information about the use of VM Sockets, please refer to the VM Sockets Programming Guide available at:

https://www.vmware.com/support/developer/vmci-sdk/

Signed-off-by George Zhang <[email protected]>

Signed-off-by Dmitry Torokhov <[email protected]>

Signed-off-by Andy king <[email protected]>

index 0000000..95e2568

net/vmw_vsock/Kconfig line 7

Quoted 4 lines · On Fri, Jan 25, 2013 at 09:37:50AM -0800, [email protected] wrote:

net/vmw_vsock/Kconfig new file +4 -0

+
+config VMWARE_VSOCK
+	tristate "Virtual Socket protocol"
+	depends on VMWARE_VMCI

What is CONFIG_VMWARE_VMCI? I don't find that in any Kconfig in the tree?

I''m still looking over the rest, but I get build issues if I just remove the dependency.

net/vmw_vsock/Kconfig line 7

Quoted 4 lines · On Fri, Jan 25, 2013 at 09:37:50AM -0800, [email protected] wrote:

net/vmw_vsock/Kconfig new file +4 -0

+
+config VMWARE_VSOCK
+	tristate "Virtual Socket protocol"
+	depends on VMWARE_VMCI

What is CONFIG_VMWARE_VMCI? I don't find that in any Kconfig in the tree?

I''m still looking over the rest, but I get build issues if I just remove the dependency.

Hi Neil,

Quoted 8 lines · Neil Horman

net/vmw_vsock/Kconfig line 7

Quoted 4 lines · On Fri, Jan 25, 2013 at 09:37:50AM -0800, [email protected] wrote:

net/vmw_vsock/Kconfig new file +4 -0

+
+config VMWARE_VSOCK
+	tristate "Virtual Socket protocol"
+	depends on VMWARE_VMCI

What is CONFIG_VMWARE_VMCI? I don't find that in any Kconfig in the tree?

Quoted 2 lines

I''m still looking over the rest, but I get build issues if I just remove the dependency.

VMCI is in linux-next at the moment.

From the char-misc-next git tree to be specific.

Hi Neil,

Quoted 8 lines · Neil Horman

net/vmw_vsock/Kconfig line 7

Quoted 4 lines · On Fri, Jan 25, 2013 at 09:37:50AM -0800, [email protected] wrote:

net/vmw_vsock/Kconfig new file +4 -0

+
+config VMWARE_VSOCK
+	tristate "Virtual Socket protocol"
+	depends on VMWARE_VMCI

What is CONFIG_VMWARE_VMCI? I don't find that in any Kconfig in the tree?

Quoted 2 lines

I''m still looking over the rest, but I get build issues if I just remove the dependency.

VMCI is in linux-next at the moment.

Thanks,
Dmitry

From: Andy King <[email protected]>

VM Sockets allows communication between virtual machines and the hypervisor. User level applications both in a virtual machine and on the host can use the VM Sockets API, which facilitates fast and efficient communication between guest virtual machines and their host. A socket address family, designed to be compatible with UDP and TCP at the interface level, is provided.

Today, VM Sockets is used by various VMware Tools components inside the guest for zero-config, network-less access to VMware host services. In addition to this, VMware's users are using VM Sockets for various applications, where network access of the virtual machine is restricted or non-existent. Examples of this are VMs communicating with device proxies for proprietary hardware running as host applications and automated testing of applications running within virtual machines.

The VMware VM Sockets are similar to other socket types, like Berkeley UNIX socket interface. The VM Sockets module supports both connection-oriented stream sockets like TCP, and connectionless datagram sockets like UDP. The VM Sockets protocol family is defined as "AF_VSOCK" and the socket operations split for SOCK_DGRAM and SOCK_STREAM.

For additional information about the use of VM Sockets, please refer to the VM Sockets Programming Guide available at:

https://www.vmware.com/support/developer/vmci-sdk/

Signed-off-by George Zhang <[email protected]>

Signed-off-by Dmitry Torokhov <[email protected]>

Signed-off-by Andy king <[email protected]>

index 0000000..95e2568

net/vmw_vsock/Kconfig new file +4 -0

+static long vsock_dev_do_ioctl(struct file *filp,
+			       unsigned int cmd, void __user *ptr)
+{
+	static const u16 parts[4] = VSOCK_DRIVER_VERSION_PARTS;

net/vmw_vsock/Kconfig new file +7 -0

+static int __init vsock_init(void)
+{
+	int err;
+
+	vsock_init_tables();
+
+	err = misc_register(&vsock_device);

net/vmw_vsock/Kconfig new file +9 -0

+/**** TRANSPORT ****/
+
+struct vsock_transport {
+	void (*init)(struct vsock_sock *, struct vsock_sock *);
+	void (*destruct)(struct vsock_sock *);
+	void (*release)(struct vsock_sock *);
+	int (*connect)(struct vsock_sock *);
+	int (*bind_dgram)(struct vsock_sock *, struct sockaddr_vm *);
+	int (*send_dgram)(struct vsock_sock *, struct sockaddr_vm *,

net/vmw_vsock/Kconfig new file +4 -0

+static long vsock_dev_do_ioctl(struct file *filp,
+			       unsigned int cmd, void __user *ptr)
+{
+	static const u16 parts[4] = VSOCK_DRIVER_VERSION_PARTS;

net/vmw_vsock/Kconfig new file +7 -0

+static int __init vsock_init(void)
+{
+	int err;
+
+	vsock_init_tables();
+
+	err = misc_register(&vsock_device);

net/vmw_vsock/Kconfig new file +9 -0

+/**** TRANSPORT ****/
+
+struct vsock_transport {
+	void (*init)(struct vsock_sock *, struct vsock_sock *);
+	void (*destruct)(struct vsock_sock *);
+	void (*release)(struct vsock_sock *);
+	int (*connect)(struct vsock_sock *);
+	int (*bind_dgram)(struct vsock_sock *, struct sockaddr_vm *);
+	int (*send_dgram)(struct vsock_sock *, struct sockaddr_vm *,

net/vmw_vsock/Kconfig line 7

Quoted 3 lines

net/vmw_vsock/Kconfig new file +3 -0

+config VMWARE_VSOCK
+	tristate "Virtual Socket protocol"
+	depends on VMWARE_VMCI

I guess this is temporary? Cover letter says *mostly* separated ...

net/vmw_vsock/Makefile line 4

Quoted 3 lines

+vmw_vsock-y += af_vsock.o vmci_transport.o vmci_transport_notify.o \

net/vmw_vsock/Makefile new file +1 -0

+	vmci_transport_notify_qstate.o vsock_addr.o

Likewise, I expect with the final version vmci_transport is a separate module (or moves into the vmci driver), correct?

net/vmw_vsock/af_vsock.c line 1875

Quoted 6 lines

net/vmw_vsock/af_vsock.c new file +6 -0

+	switch (cmd) {
+	case IOCTL_VMCI_SOCKETS_VERSION:
+		version = VMCI_SOCKETS_MAKE_VERSION(parts);
+		if (put_user(version, p) != 0)
+			retval = -EFAULT;
+		break;

Still needed?

net/vmw_vsock/af_vsock.c line 1881

Quoted 5 lines

net/vmw_vsock/af_vsock.c new file +5 -0

+	case IOCTL_VMCI_SOCKETS_GET_AF_VALUE:
+		if (put_user(AF_VSOCK, p) != 0)
+			retval = -EFAULT;
+
+		break;

That can go away, with the upstream merge vsock will get a fixed AF_VSOCK.

net/vmw_vsock/af_vsock.c line 1885

Quoted 3 lines

net/vmw_vsock/af_vsock.c new file +3 -0

+	case IOCTL_VMCI_SOCKETS_GET_LOCAL_CID:
+		if (put_user(vmci_get_context_id(), p) != 0)
+			retval = -EFAULT;

What is this?

net/vmw_vsock/af_vsock.c line 1942

Quoted 5 lines

net/vmw_vsock/af_vsock.c new file +5 -0

+	err = vmci_transport_register(&transport);
+	if (err) {
+		pr_err("Cannot register with VMCI device\n");
+		goto err_misc_deregister;
+	}

Hmm? There should be a vsock_(un)register_transport which the vmci transport code can call (and likewise virtio transport some day).

net/vmw_vsock/af_vsock.h line 72

Quoted 3 lines

net/vmw_vsock/af_vsock.h new file +3 -0

+	struct {
+		/* For DGRAMs. */
+		struct vmci_handle dg_handle;

Yep, should be a pointer where transports can hook in their private data.

Quoted 7 lines

net/vmw_vsock/af_vsock.h new file +7 -0

+struct vsock_transport {
+	void (*init)(struct vsock_sock *, struct vsock_sock *);
+	void (*destruct)(struct vsock_sock *);
+	void (*release)(struct vsock_sock *);
+	int (*connect)(struct vsock_sock *);
+	int (*bind_dgram)(struct vsock_sock *, struct sockaddr_vm *);
+	int (*send_dgram)(struct vsock_sock *, struct sockaddr_vm *,

net/vmw_vsock/af_vsock.h line 114

Quoted 10 lines

net/vmw_vsock/af_vsock.h new file +10 -0

+			  struct iovec *, size_t len);
+	ssize_t (*recv_stream)(struct vsock_sock *, struct iovec *,
+			       size_t len, int flags);
+	ssize_t (*send_stream)(struct vsock_sock *, struct iovec *,
+			       size_t len);
+	s64 (*stream_has_data)(struct vsock_sock *);
+	s64 (*stream_has_space)(struct vsock_sock *);
+	int (*send_shutdown)(struct sock *sk, int mode);
+	void (*unregister)(void);
+};

So that is the interface transports have to implement. Looks reasonable to me. Some documentation would be nice, although most of it is self-explaining.

Where is recv_dgram?

Also why bind_dgram? I guess binding stream sockets doesn't make sense for the vsock family?

I'd make the naming a bit more consistent, some stream callbacks are prefixed and some postfixed with "stream".

I'd also name send_shutdown just shutdown (same name the system call has).

What does unregister?

Quoted 2 lines

Likewise, I expect with the final version vmci_transport is a separate module (or moves into the vmci driver), correct?

When you say final, do you mean something that we have to do before acceptance into mainline or something we can refine over time?

From my side the minimum requirement is to have

Quoted 5 lines

net/vmw_vsock/af_vsock.c line 1885

Quoted 3 lines

net/vmw_vsock/af_vsock.c new file +3 -0

+	case IOCTL_VMCI_SOCKETS_GET_LOCAL_CID:
+		if (put_user(vmci_get_context_id(), p) != 0)
+			retval = -EFAULT;

What is this?

A CID, or "context ID" is how we identify a VM. It's also in the address structure (svm_cid). If you look at vm_sockets.h, you'll see that we have definitions for various endpoints (the host, anonymous and so forth). It's sometimes useful for VMs to be able to query their own ID, for example, to be able to pass it out-of-band via INET to a peer. So we'd like to keep this, although I guess it should be transport-defined, i.e., we should ask the transport for this value.

Quoted 6 lines

net/vmw_vsock/af_vsock.h line 72

Quoted 3 lines

net/vmw_vsock/af_vsock.h new file +3 -0

+	struct {
+		/* For DGRAMs. */
+		struct vmci_handle dg_handle;

Yep, should be a pointer where transports can hook in their private data.

I'm fixing this.

Quoted 1 line

Where is recv_dgram?

The transport just enqueues sk_buffs in the socket's buffer, so the core socket code can just pull them off. So there's no explicit recv_dgram.

Quoted 2 lines

Also why bind_dgram? I guess binding stream sockets doesn't make sense for the vsock family?

Ah, for our transport, there's nothing special involved in binding a STREAM, everything is handled by the core socket code. So I didn't add a transport callback. This is something we can add when it becomes necessary, if that's okay?

From my side the minimum requirement is to have vsock_(un)register_transport calls available, so it is possible to write a virtio transport module without having to patch vsock code to hook it up.

Quoted 9 lines

Quoted 7 lines

net/vmw_vsock/af_vsock.h line 72

Quoted 3 lines

net/vmw_vsock/af_vsock.h new file +3 -0

+	struct {
+		/* For DGRAMs. */
+		struct vmci_handle dg_handle;

Yep, should be a pointer where transports can hook in their private
data.

I'm fixing this.

Yes, please, that is needed too to get started with virtio support.