Using VMChannel to communicate with the host.

Hi,

I'm trying to get a better understanding of VM-to-host communication that doesn't involve going over virtual networks. I understand there are a couple of developments underway, but I just want to play around a
better sense of things. I think the current hypercall mechanism is the
current way to do this using the -vmchannel option. But, I'm not very experienced with PCI devices and their drivers so I need some help. All I want to do is to communicate across a unix domain socket on the host.
Besides loading the hypercall module in the guest how do things needs
to be configured on the host?

Is there a simple tutorial or just a webpage with examples for how to do this?

Quoted 12 lines · Cam Macdonell

Hi,

I'm trying to get a better understanding of VM-to-host communication that doesn't involve going over virtual networks. I understand there are a couple of developments underway, but I just want to play around a
better sense of things. I think the current hypercall mechanism is the
current way to do this using the -vmchannel option. But, I'm not very experienced with PCI devices and their drivers so I need some help. All I want to do is to communicate across a unix domain socket on the host.
Besides loading the hypercall module in the guest how do things needs
to be configured on the host?

Quoted 3 lines

Is there a simple tutorial or just a webpage with examples for how to do this?

I'm going to suggest that a better way to do VM-to-host or VM-to-VM communication would be to write an OpenFabrics (www.openfabrics.org) driver that uses hypercalls. There is probably some gotcha I am not aware of, but I think this would allow direct userspace to userspace communication between a process running on a host, and a process running on a VM, while still maintaining isolation.

Hi,

I'm trying to get a better understanding of VM-to-host communication that doesn't involve going over virtual networks. I understand there are a couple of developments underway, but I just want to play around a
better sense of things. I think the current hypercall mechanism is the
current way to do this using the -vmchannel option. But, I'm not very experienced with PCI devices and their drivers so I need some help. All I want to do is to communicate across a unix domain socket on the host.
Besides loading the hypercall module in the guest how do things needs
to be configured on the host?

Is there a simple tutorial or just a webpage with examples for how to do this?

Thanks,
Cam

I just tested it, it got a little deprecated over time since it call pci_module_init instead of pci_register_* and some more irq function deprecation.The sysfs interface in not complete and you can't read/write from the device.
The qemu parameters are -vmchannel di:2258,unix:/tmp/file,server 2258 is the vendor id for the device, you can use unix sockets ot even tcp for the host end.

Quoted 8 lines · Dor Laor

I just tested it, it got a little deprecated over time since it call pci_module_init instead of pci_register_* and some more irq function deprecation.The sysfs interface in not complete and you can't read/write from the device.
The qemu parameters are -vmchannel di:2258,unix:/tmp/file,server 2258 is the vendor id for the device, you can use unix sockets ot even tcp for the host end.

Hi Dor,

Thanks, that got it up and running. But in the hypercall.c code (from the git repo) there is a "hello host" message that looks like it should be written to the host. How do I trigger that to be sent so my very simple "listening" program can read it?

Thanks,
Cam

Quoted 24 lines · Cam Macdonell

Quoted 1 line · Dor Laor

I just tested it, it got a little deprecated over time since it

call

Quoted 1 line

pci_module_init instead of pci_register_* and some more irq

function

Quoted 1 line

deprecation.The sysfs interface in not complete and you can't

read/write

Quoted 3 lines

from the device.
The qemu parameters are -vmchannel di:2258,unix:/tmp/file,server 2258 is the vendor id for the device, you can use unix sockets

ot even

Quoted 2 lines

tcp for the host end.

Hi Dor,

Thanks, that got it up and running. But in the hypercall.c code (from
the git repo) there is a "hello host" message that looks like it should
be written to the host. How do I trigger that to be sent so my very simple "listening" program can read it?

Quoted 3 lines

Thanks,
Cam

You need to open the unix socket you passed to the vmchannel parameter.
An easier alternative is to use -vmchannel di:2258,tcp:// 0:4444,server.
Before the guest loads you'll need to telnet the 4444 port and then you should receive the
hello world output once the driver is up.
-Dor

Quoted 8 lines

You need to open the unix socket you passed to the vmchannel parameter.
An easier alternative is to use -vmchannel di:2258,tcp:// 0:4444,server.
Before the guest loads you'll need to telnet the 4444 port and then you should receive the
hello world output once the driver is up.
-Dor

I tried having a program listening on the unix domain socket. Actually, the VM won't even start until a program connects to the socket. I didn't get the message with my listening program, but I'll try the telnet method as I haven't programmed a socket in a while so I may have missed a step.

While getting this working is novel to me, it seems from your email that not much can be done with the hypercall interface in terms of host-VM or VM-to-VM communication, correct? If reading and writing don't work, how can one exchange info between VMs? I'll look forward to the virtIO implementation.

Quoted 8 lines

Quoted 8 lines

You need to open the unix socket you passed to the vmchannel parameter.
An easier alternative is to use -vmchannel di:2258,tcp:// 0:4444,server.
Before the guest loads you'll need to telnet the 4444 port and then you should receive the
hello world output once the driver is up.
-Dor

Quoted 5 lines

I tried having a program listening on the unix domain socket. Actually, the VM won't even start until a program connects to the socket. I didn't get the message with my listening program, but I'll try the telnet method as I haven't programmed a socket in a while so I may have missed a step.

Go for it, its 1 minute effort.

Quoted 7 lines

While getting this working is novel to me, it seems from your email that not much can be done with the hypercall interface in terms of host-VM or VM-to-VM communication, correct? If reading and writing don't work, how can one exchange info between VMs? I'll look forward to the virtIO implementation.

The vmchannel was not intended to VM-to-VM networking, although it can work.
Next version will be better. Anyway, it should have similar performance to pv network driver.

Quoted 8 lines · Dor Laor

Quoted 8 lines

Quoted 8 lines

You need to open the unix socket you passed to the vmchannel parameter.
An easier alternative is to use -vmchannel di:2258,tcp:// 0:4444,server.
Before the guest loads you'll need to telnet the 4444 port and then you should receive the
hello world output once the driver is up.
-Dor

Quoted 7 lines

Quoted 5 lines

I tried having a program listening on the unix domain socket. Actually, the VM won't even start until a program connects to the socket. I didn't get the message with my listening program, but I'll try the telnet method as I haven't programmed a socket in a while so I may have missed a step.

Go for it, its 1 minute effort.

Hi Dor,

It didn't work. I used the following option: -vmchannel di:2258,tcp:0:4444,server (the // confused kvm) and when the VM booted, I connected with "telnet localhost 4444" which allowed the boot to proceed. But, I didn't get the hello host message when I loaded the hypercall module. dmesg did show that the module loaded successfully. I'll dig around with it and see what else I can figure ou

Quoted 9 lines · Cam Macdonell

It didn't work. I used the following option: -vmchannel di:2258,tcp:0:4444,server (the // confused kvm) and when the VM booted,
I connected with "telnet localhost 4444" which allowed the boot to proceed. But, I didn't get the hello host message when I loaded the hypercall module. dmesg did show that the module loaded successfully.
I'll dig around with it and see what else I can figure ou

I just tried it and it worked. I used fc7 2.6.21 kernel, 64 bits for the guest.
Can you send the relevant dmesg?

Quoted 10 lines · Dor Laor

Quoted 9 lines · Cam Macdonell

It didn't work. I used the following option: -vmchannel di:2258,tcp:0:4444,server (the // confused kvm) and when the VM booted,
I connected with "telnet localhost 4444" which allowed the boot to proceed. But, I didn't get the hello host message when I loaded the hypercall module. dmesg did show that the module loaded successfully.
I'll dig around with it and see what else I can figure ou

Quoted 3 lines

I just tried it and it worked. I used fc7 2.6.21 kernel, 64 bits for the guest.
Can you send the relevant dmesg?

Here is the dmesg output within the guest. I am running KVM-39 on Ubuntu 2.6.20 kernel, 32 bit for both
host and guest.

[  540.144000] Qumranet_hypercall_driver
[  540.148000] pci dev 0000:00:04.0 (id 5002:2258 rev 00) is a guest 
hypercall device
[  540.148000] PCI: Found IRQ 9 for device 0000:00:04.0
[  540.148000] hypercall_init_board: PIO region size == 0x100
[  540.148000] hypercall_init_board: MMIO region size == 0x00
[  540.148000] name=: base_addr=0xc200, io_addr=0x1c200, IRQ=9

For what it's worth, I can compile the driver/hypercall.c file that comes with kvm-39. However, I can't compile your hypercall.c implementation, I get the following error:

make -C /lib/modules/2.6.20-16-generic/build M=pwd "$@" make[1]: Entering directory `/usr/src/linux-headers-2.6.20-16-generic'
CC [M] /home/kvm/kvm-userspace/drivers/hypercall.o
/home/kvm/kvm-userspace/drivers/hypercall.c: In function ‘hypercall_do_tasklet’:
/home/kvm/kvm-userspace/drivers/hypercall.c:226: error: invalid lvalue in unary ‘&’ /home/kvm/kvm-userspace/drivers/hypercall.c:226: error: invalid lvalue in unary ‘&’ /home/kvm/kvm-userspace/drivers/hypercall.c:231: warning: implicit declaration of function ‘hypercall_tx’ /home/kvm/kvm-userspace/drivers/hypercall.c: At top level:
/home/kvm/kvm-userspace/drivers/hypercall.c:236: error: static declaration of ‘hypercall_tx’ follows non-static declaration /home/kvm/kvm-userspace/drivers/hypercall.c:231: error: previous implicit declaration of ‘hypercall_tx’ was here /home/kvm/kvm-userspace/drivers/hypercall.c: In function ‘hypercall_interrupt’:
/home/kvm/kvm-userspace/drivers/hypercall.c:302: warning: passing argument 1 of ‘__constant_memcpy’ makes pointer from integer without a cast
/home/kvm/kvm-userspace/drivers/hypercall.c:302: warning: passing argument 1 of ‘__memcpy’ makes pointer from integer without a cast /home/kvm/kvm-userspace/drivers/hypercall.c: In function ‘hypercall_open’:
/home/kvm/kvm-userspace/drivers/hypercall.c:320: warning: passing argument 2 of ‘request_irq’ from incompatible pointer type /home/kvm/kvm-userspace/drivers/hypercall.c:326: warning: assignment makes integer from pointer without a cast /home/kvm/kvm-userspace/drivers/hypercall.c:328: error:
‘hypercall_task’ undeclared (first use in this function) /home/kvm/kvm-userspace/drivers/hypercall.c:328: error: (Each undeclared identifier is reported only once /home/kvm/kvm-userspace/drivers/hypercall.c:328: error: for each function it appears in.)
make[2]: *** [/home/kvm/kvm-userspace/drivers/hypercall.o] Error 1 make[1]: *** [_module_/home/kvm/kvm-userspace/drivers] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-2.6.20-16-generic'
make: *** [all] Error 2

Thanks,
Cam

-----------------------------------------------
A. Cameron Macdonell
Ph.D. Student
Department of Computing Science
University of Alberta
[email protected]

Quoted 1 line

Hmm, well it's an old code that Uri will rebase for virtio so just drop it. I just thought it might help.

Quoted 1 line

Quoted 5 lines · Dor Laor

Quoted 1 line

Hmm, well it's an old code that Uri will rebase for virtio so just drop it.
I just thought it might help.

Quoted 1 line

No worries. In terms of moving to virtIO I grabbed your tree previously to look at it. To test and play around with virtIO, do I need to use your kernel (or module) in the guest?