kvm: pci PORT IO MMIO and PV MMIO speed tests
Subsystems (3)
Michael S. Tsirkin
maintainerv2 cover kvm: pci PORT IO MMIO and PV MMIO speed tests
These patches add a test device, useful to measure speed of MMIO versus PIO, in different configurations. As I didn't want to reserve a hardcoded range of memory, I added pci device for this instead. Used together with the kvm unittest patches I posted on kvm mailing list.
To use, simply add the device on the pci bus.
Example test output:
vmcall 1519
....
outl_to_kernel 1745
....
mmio-no-eventfd:pci-mem 9075
mmio-wildcard-eventfd:pci-mem 3529
mmio-datamatch-eventfd:pci-mem 3509
mmio-pv-eventfd:pci-mem 1878
portio-no-eventfd:pci-io 5535
portio-wildcard-eventfd:pci-io 1846
portio-datamatch-eventfd:pci-io 1848
portio-pv-eventfd:pci-io 1842First interesting conclusion is that the overhead of MMIO exit to QEMU as compared to PIO is double that of
MMIO ioeventfd as compared to PIO eventfd. Is this a known fact?
Second is that PV MMIO is almost as fast as portio, so it looks like a viable replacement when we have a lot of devices such that we run out of IO space.
Third is that eventfd has small but measureable overhead, and that hypercalls are measureably faster than port io. The difference is probably in hardware. Whether this justifies considering hypercalls for e.g. virtio, is open for discussion.
I also had to extend kvm in a minor way, making all ioeventfd options accessible through the API. This actually needs less code than checking that users DTRT.
Patches 1-5 are useful in themselves.
Patches 5 and 6 add support for PV MMIO eventfds and add the test for it, tested against a kernel patch I'll send shortly. They can be applied separately when we are sure the kernel interface is going to be there.
Michael S. Tsirkin (6):
kvm: remove unused APIs
kvm: support any size for pio eventfd
kvm: support non datamatch ioeventfd
pci: add pci test device
kvm: add PV MMIO
pci-testdev: add pv mmio test
docs/specs/pci-testdev.txt | 26 ++++ hw/Makefile.objs | 1 + hw/dataplane/hostmem.c | 1 + hw/ivshmem.c | 2 + hw/pci-testdev.c | 311 +++++++++++++++++++++++++++++++++++++++++++++ hw/pci/pci.h | 1 + hw/vhost.c | 4 +- hw/virtio-pci.c | 4 +- include/exec/memory.h | 10 ++ include/sysemu/kvm.h | 4 - kvm-all.c | 142 +++++++++++---------- kvm-stub.c | 10 -- linux-headers/linux/kvm.h | 8 ++ memory.c | 9 +- 14 files changed, 448 insertions(+), 85 deletions(-) create mode 100644 docs/specs/pci-testdev.txt create mode 100644 hw/pci-testdev.c
Signature · 1 line
MST
Michael S. Tsirkin
maintainerPATCH v2 1/6 kvm: remove unused APIs
There are only used internally now, move them
out of header and out of stub.
Signed-off-by Michael S. Tsirkin <[email protected]>
--- include/sysemu/kvm.h | 4 -- kvm-all.c | 107 ++++++++++++++++++++++++++------------------------- kvm-stub.c | 10 ----- 3 files changed, 54 insertions(+), 67 deletions(-)
include/sysemu/kvm.h +0 -4
@@ -283,10 +283,6 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,#endif #endif -int kvm_set_ioeventfd_mmio (int fd,uint32_t adr,uint32_t val,bool assign, -uint32_t size); - -int kvm_set_ioeventfd_pio_word (int fd,uint16_t adr,uint16_t val,bool assign);int kvm_irqchip_add_msi_route (KVMState *s,MSIMessage msg);int kvm_irqchip_update_msi_route (KVMState *s,int virq,MSIMessage msg);
kvm-all.c +54 -53
119 lines not on this page · raw message
kvm-stub.c +0 -10
@@ -102,16 +102,6 @@ int kvm_set_signal_mask(CPUArchState *env, const sigset_t *sigset) }#endif -int kvm_set_ioeventfd_pio_word (int fd,uint16_t addr,uint16_t val,bool assign) -{ -return -ENOSYS ; -} - -int kvm_set_ioeventfd_mmio (int fd,uint32_t adr,uint32_t val,bool assign,uint32_t len) -{ -return -ENOSYS ; -} -int kvm_on_sigbus_vcpu (CPUState *cpu,int code,void *addr) {return 1 ;
Signature · 1 line
MST
Michael S. Tsirkin
maintainerPATCH v2 2/6 kvm: support any size for pio eventfd
In reply to Michael S. TsirkinSigned-off-by Michael S. Tsirkin <[email protected]>
--- kvm-all.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)
kvm-all.c +12 -12
@@ -500,8 +500,8 @@ int kvm_check_extension(KVMState *s, unsigned int extension)return ret; } -static int kvm_set_ioeventfd_mmio (int fd,uint32_t addr,uint32_t val,bool assign, -uint32_t size) +static int kvm_set_ioeventfd_mmio (int fd,uint32_t addr,uint32_t val, +bool assign,uint32_t size) {int ret;struct kvm_ioeventfd iofd;
@@ -529,13 +529,13 @@ static int kvm_set_ioeventfd_mmio(int fd, uint32_t addr, uint32_t val, bool assireturn 0 ; } -static int kvm_set_ioeventfd_pio_word (int fd,uint16_t addr,uint16_t val, -bool assign) +static int kvm_set_ioeventfd_pio (int fd,uint16_t addr,uint16_t val, +bool assign,uint32_t size) {struct kvm_ioeventfd kick = { .datamatch = val, .addr = addr, - .len =2 , + .len = size, .flags =KVM_IOEVENTFD_FLAG_DATAMATCH |KVM_IOEVENTFD_FLAG_PIO , .fd = fd, };
@@ -571,7 +571,7 @@ static int kvm_check_many_ioeventfds(void)if (ioeventfds[i] <0 ) {break ; } - ret =kvm_set_ioeventfd_pio_word (ioeventfds[i],0 , i, true); + ret =kvm_set_ioeventfd_pio (ioeventfds[i],0 , i, true,2 );if (ret <0 ) {close (ioeventfds[i]);break ;
@@ -582,7 +582,7 @@ static int kvm_check_many_ioeventfds(void) ret = i ==ARRAY_SIZE (ioeventfds);while (i-- >0 ) { -kvm_set_ioeventfd_pio_word (ioeventfds[i],0 , i, false); +kvm_set_ioeventfd_pio (ioeventfds[i],0 , i, false,2 );close (ioeventfds[i]); }return ret;
@@ -834,10 +834,10 @@ static void kvm_io_ioeventfd_add(MemoryListener *listener,int fd =event_notifier_get_fd (e);int r; -assert (match_data && section->size ==2 ); +assert (match_data && section->size <=8 ); - r =kvm_set_ioeventfd_pio_word (fd, section->offset_within_address_space, - data, true); + r =kvm_set_ioeventfd_pio (fd, section->offset_within_address_space, + data, true, section->size);if (r <0 ) {abort (); }
@@ -852,8 +852,8 @@ static void kvm_io_ioeventfd_del(MemoryListener *listener,int fd =event_notifier_get_fd (e);int r; - r =kvm_set_ioeventfd_pio_word (fd, section->offset_within_address_space, - data, false); + r =kvm_set_ioeventfd_pio (fd, section->offset_within_address_space, + data, false, section->size);if (r <0 ) {abort (); }
Signature · 1 line
MST
Michael S. Tsirkin
maintainerPATCH v2 3/6 kvm: support non datamatch ioeventfd
In reply to Michael S. TsirkinAdding restrictions just adds code.
Signed-off-by Michael S. Tsirkin <[email protected]>
--- kvm-all.c | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-)
kvm-all.c +18 -16
@@ -501,21 +501,24 @@ int kvm_check_extension(KVMState *s, unsigned int extension) }static int kvm_set_ioeventfd_mmio (int fd,uint32_t addr,uint32_t val, -bool assign,uint32_t size) +bool assign,uint32_t size,bool datamatch) {int ret;struct kvm_ioeventfd iofd; - iofd.datamatch = val; + iofd.datamatch = datamatch ? val :0 ; iofd.addr = addr; iofd.len = size; - iofd.flags =KVM_IOEVENTFD_FLAG_DATAMATCH ; + iofd.flags =0 ; iofd.fd = fd;if (!kvm_enabled ()) {return -ENOSYS ; } +if (datamatch) { + iofd.flags |=KVM_IOEVENTFD_FLAG_DATAMATCH ; + }if (!assign) { iofd.flags |=KVM_IOEVENTFD_FLAG_DEASSIGN ; }
@@ -530,19 +533,22 @@ static int kvm_set_ioeventfd_mmio(int fd, uint32_t addr, uint32_t val, }static int kvm_set_ioeventfd_pio (int fd,uint16_t addr,uint16_t val, -bool assign,uint32_t size) +bool assign,uint32_t size,bool datamatch) {struct kvm_ioeventfd kick = { - .datamatch = val, + .datamatch = datamatch ? val :0 , .addr = addr, + .flags =KVM_IOEVENTFD_FLAG_PIO , .len = size, - .flags =KVM_IOEVENTFD_FLAG_DATAMATCH |KVM_IOEVENTFD_FLAG_PIO , .fd = fd, };int r;if (!kvm_enabled ()) {return -ENOSYS ; } +if (datamatch) { + kick.flags |=KVM_IOEVENTFD_FLAG_DATAMATCH ; + }if (!assign) { kick.flags |=KVM_IOEVENTFD_FLAG_DEASSIGN ; }
@@ -571,7 +577,7 @@ static int kvm_check_many_ioeventfds(void)if (ioeventfds[i] <0 ) {break ; } - ret =kvm_set_ioeventfd_pio (ioeventfds[i],0 , i, true,2 ); + ret =kvm_set_ioeventfd_pio (ioeventfds[i],0 , i, true,2 , true);if (ret <0 ) {close (ioeventfds[i]);break ;
@@ -582,7 +588,7 @@ static int kvm_check_many_ioeventfds(void) ret = i ==ARRAY_SIZE (ioeventfds);while (i-- >0 ) { -kvm_set_ioeventfd_pio (ioeventfds[i],0 , i, false,2 ); +kvm_set_ioeventfd_pio (ioeventfds[i],0 , i, false,2 , true);close (ioeventfds[i]); }return ret;
@@ -802,10 +808,8 @@ static void kvm_mem_ioeventfd_add(MemoryListener *listener,int fd =event_notifier_get_fd (e);int r; -assert (match_data && section->size <=8 ); - r =kvm_set_ioeventfd_mmio (fd, section->offset_within_address_space, - data, true, section->size); + data, true, section->size, match_data);if (r <0 ) {abort (); }
@@ -820,7 +824,7 @@ static void kvm_mem_ioeventfd_del(MemoryListener *listener,int r; r =kvm_set_ioeventfd_mmio (fd, section->offset_within_address_space, - data, false, section->size); + data, false, section->size, match_data);if (r <0 ) {abort (); }
@@ -834,10 +838,8 @@ static void kvm_io_ioeventfd_add(MemoryListener *listener,int fd =event_notifier_get_fd (e);int r; -assert (match_data && section->size <=8 ); - r =kvm_set_ioeventfd_pio (fd, section->offset_within_address_space, - data, true, section->size); + data, true, section->size, match_data);if (r <0 ) {abort (); }
@@ -853,7 +855,7 @@ static void kvm_io_ioeventfd_del(MemoryListener *listener,int r; r =kvm_set_ioeventfd_pio (fd, section->offset_within_address_space, - data, false, section->size); + data, false, section->size, match_data);if (r <0 ) {abort (); }
Signature · 1 line
MST
Michael S. Tsirkin
maintainerPATCH v2 4/6 pci: add pci test device
In reply to Michael S. TsirkinThis device is used for kvm unit tests, currently it supports testing performance of ioeventfd. Using updated kvm unittest, here's an example output:
mmio-no-eventfd:pci-mem 8796
mmio-wildcard-eventfd:pci-mem 3609
mmio-datamatch-eventfd:pci-mem 3685
portio-no-eventfd:pci-io 5287
portio-wildcard-eventfd:pci-io 1762
portio-datamatch-eventfd:pci-io 1777Signed-off-by Michael S. Tsirkin <[email protected]>
--- docs/specs/pci-testdev.txt | 26 ++++ hw/Makefile.objs | 1 + hw/pci-testdev.c | 306 +++++++++++++++++++++++++++++++++++++++++++++ hw/pci/pci.h | 1 + 4 files changed, 334 insertions(+) create mode 100644 docs/specs/pci-testdev.txt create mode 100644 hw/pci-testdev.c
docs/specs/pci-testdev.txt new file +26 -0
@@ -0,0 +1,26 @@ +pci-test is a device used for testing low level IO + +device implements up to two BARs: BAR0 and BAR1. +Each BAR can be memory or IO. Guests must detect +BAR type and act accordingly. + +Each BAR size is up to 4K bytes. +Each BAR starts with the following header: + +typedef struct PCITestDevHdr { + uint8_t test; <- write-only, starts a given test number + uint8_t width_type; <- read-only, type and width of access for a given test. + 1,2,4 for byte,word or long write. + any other value if test not supported on this BAR + uint8_t pad0[2]; + uint32_t offset; <- read-only, offset in this BAR for a given test + uint32_t data; <- read-only, data to use for a given test + uint32_t count; <- for debugging. number of writes detected. + uint8_t name[]; <- for debugging. 0-terminated ASCII string. +} PCITestDevHdr; + +All registers are little endian. + +device is expected to always implement tests 0 to N on each BAR, and to add new +tests with higher numbers. In this way a guest can scan test numbers until it +detects an access type that it does not support on this BAR, then stop.
hw/Makefile.objs +1 -0
@@ -12,6 +12,7 @@ common-obj-$(CONFIG_VIRTIO_PCI) += virtio-pci.o common-obj-$(CONFIG_VIRTIO) += virtio-bus.o common-obj-y += fw_cfg.o common-obj-$(CONFIG_PCI) += pci_bridge_dev.o +common-obj-$(CONFIG_PCI) += pci-testdev.o common-obj-$(CONFIG_PCI) += ioh3420.o xio3130_upstream.o xio3130_downstream.o common-obj-$(CONFIG_PCI) += i82801b11.o common-obj-y += watchdog.o
hw/pci-testdev.c new file +306 -0
306 lines not on this page · raw message
hw/pci/pci.h +1 -0
@@ -84,6 +84,7 @@#define PCI_DEVICE_ID_REDHAT_SERIAL 0x0002#define PCI_DEVICE_ID_REDHAT_SERIAL2 0x0003#define PCI_DEVICE_ID_REDHAT_SERIAL4 0x0004 +#define PCI_DEVICE_ID_REDHAT_TEST 0x0005#define PCI_DEVICE_ID_REDHAT_QXL 0x0100#define FMT_PCIBUS PRIx64
Signature · 1 line
MST
Michael S. Tsirkin
maintainerPATCH v2 5/6 kvm: add PV MMIO
In reply to Michael S. TsirkinAdd an option for users to specify PV eventfd
listeners, and utilize KVM's PV MMIO underneath.
Upodate all callers.
Signed-off-by Michael S. Tsirkin <[email protected]>
--- hw/dataplane/hostmem.c | 1 + hw/ivshmem.c | 2 ++ hw/pci-testdev.c | 2 ++ hw/vhost.c | 4 ++-- hw/virtio-pci.c | 4 ++-- include/exec/memory.h | 10 ++++++++++ kvm-all.c | 15 ++++++++++++--- linux-headers/linux/kvm.h | 8 ++++++++ memory.c | 9 +++++++-- 9 files changed, 46 insertions(+), 9 deletions(-)
hw/dataplane/hostmem.c +1 -0
@@ -126,6 +126,7 @@ static void hostmem_listener_section_dummy(MemoryListener *listener,static void hostmem_listener_eventfd_dummy (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {
hw/ivshmem.c +2 -0
@@ -352,6 +352,7 @@ static void ivshmem_add_eventfd(IVShmemState *s, int posn, int i)memory_region_add_eventfd (&s->ivshmem_mmio,DOORBELL ,4 , + false, true, (posn <<16 ) | i, &s->peers[posn].eventfds[i]);
@@ -362,6 +363,7 @@ static void ivshmem_del_eventfd(IVShmemState *s, int posn, int i)memory_region_del_eventfd (&s->ivshmem_mmio,DOORBELL ,4 , + false, true, (posn <<16 ) | i, &s->peers[posn].eventfds[i]);
hw/pci-testdev.c +2 -0
@@ -80,6 +80,7 @@ static int pci_testdev_start(IOTest *test)memory_region_add_eventfd (test->mr,le32_to_cpu (test->hdr->offset), test->size, + false, test->match_data, test->hdr->data, &test->notifier);
@@ -94,6 +95,7 @@ static void pci_testdev_stop(IOTest *test)memory_region_del_eventfd (test->mr,le32_to_cpu (test->hdr->offset), test->size, + false, test->match_data, test->hdr->data, &test->notifier);
hw/vhost.c +2 -2
@@ -750,13 +750,13 @@ static void vhost_virtqueue_stop(struct vhost_dev *dev, }static void vhost_eventfd_add (MemoryListener *listener, -MemoryRegionSection *section, +MemoryRegionSection *section,bool pv,bool match_data,uint64_t data,EventNotifier *e) { }static void vhost_eventfd_del (MemoryListener *listener, -MemoryRegionSection *section, +MemoryRegionSection *section,bool pv,bool match_data,uint64_t data,EventNotifier *e) { }
hw/virtio-pci.c +2 -2
@@ -189,10 +189,10 @@ static int virtio_pci_set_host_notifier_internal(VirtIOPCIProxy *proxy, }virtio_queue_set_host_notifier_fd_handler (vq, true, set_handler);memory_region_add_eventfd (&proxy->bar,VIRTIO_PCI_QUEUE_NOTIFY ,2 , - true, n, notifier); + false, true, n, notifier); }else {memory_region_del_eventfd (&proxy->bar,VIRTIO_PCI_QUEUE_NOTIFY ,2 , - true, n, notifier); + false, true, n, notifier);virtio_queue_set_host_notifier_fd_handler (vq, false, false);event_notifier_cleanup (notifier); }
include/exec/memory.h +10 -0
@@ -215,8 +215,10 @@ struct MemoryListener {void (*log_global_start)(MemoryListener *listener);void (*log_global_stop)(MemoryListener *listener);void (*eventfd_add)(MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e);void (*eventfd_del)(MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e);void (*coalesced_mmio_add)(MemoryListener *listener,MemoryRegionSection *section,hwaddr addr,hwaddr len);
@@ -594,6 +596,9 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr);* @mr: the memory region being updated. * @addr: the address within @mr that is to be monitored * @size: the size of the access to trigger the eventfd +* @pv: set if Guest can promise us that all accesses touching this address +* are writes of specified length, starting at the specified address. +* If not - it's a Guest bug. Might give faster access. * @match_data: whether to match against @data, instead of just @addr * @data: the data to match against the guest write * @fd: the eventfd to be triggered when @addr, @size, and @data all match.
@@ -601,6 +606,7 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr);void memory_region_add_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e);
@@ -614,6 +620,9 @@ void memory_region_add_eventfd(MemoryRegion *mr,* @mr: the memory region being updated. * @addr: the address within @mr that is to be monitored * @size: the size of the access to trigger the eventfd +* @pv: set if Guest can promise us that all accesses touching this address +* are writes of specified length, starting at the specified address. +* If not - it's a Guest bug. Might give faster access. * @match_data: whether to match against @data, instead of just @addr * @data: the data to match against the guest write * @fd: the eventfd to be triggered when @addr, @size, and @data all match.
@@ -621,6 +630,7 @@ void memory_region_add_eventfd(MemoryRegion *mr,void memory_region_del_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e);
kvm-all.c +12 -3
@@ -500,7 +500,7 @@ int kvm_check_extension(KVMState *s, unsigned int extension)return ret; } -static int kvm_set_ioeventfd_mmio (int fd,uint32_t addr,uint32_t val, +static int kvm_set_ioeventfd_mmio (int fd,uint32_t addr,uint32_t val,bool pv,bool assign,uint32_t size,bool datamatch) {int ret;
@@ -518,6 +518,11 @@ static int kvm_set_ioeventfd_mmio(int fd, uint32_t addr, uint32_t val,if (datamatch) { iofd.flags |=KVM_IOEVENTFD_FLAG_DATAMATCH ; +/* Kernel doesn't support data match with PV MMIO. */ + pv = false; + } +if (pv) { + iofd.flags |=KVM_IOEVENTFD_FLAG_PV_MMIO ; }if (!assign) { iofd.flags |=KVM_IOEVENTFD_FLAG_DEASSIGN ;
@@ -802,6 +807,7 @@ static void kvm_log_global_stop(struct MemoryListener *listener)static void kvm_mem_ioeventfd_add (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {
@@ -809,7 +815,7 @@ static void kvm_mem_ioeventfd_add(MemoryListener *listener,int r; r =kvm_set_ioeventfd_mmio (fd, section->offset_within_address_space, - data, true, section->size, match_data); + data, pv, true, section->size, match_data);if (r <0 ) {abort (); }
@@ -817,6 +823,7 @@ static void kvm_mem_ioeventfd_add(MemoryListener *listener,static void kvm_mem_ioeventfd_del (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {
@@ -824,7 +831,7 @@ static void kvm_mem_ioeventfd_del(MemoryListener *listener,int r; r =kvm_set_ioeventfd_mmio (fd, section->offset_within_address_space, - data, false, section->size, match_data); + data, pv, false, section->size, match_data);if (r <0 ) {abort (); }
@@ -832,6 +839,7 @@ static void kvm_mem_ioeventfd_del(MemoryListener *listener,static void kvm_io_ioeventfd_add (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {
@@ -847,6 +855,7 @@ static void kvm_io_ioeventfd_add(MemoryListener *listener,static void kvm_io_ioeventfd_del (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e)
linux-headers/linux/kvm.h +8 -0
@@ -449,11 +449,19 @@ enum { kvm_ioeventfd_flag_nr_datamatch, kvm_ioeventfd_flag_nr_pio, kvm_ioeventfd_flag_nr_deassign, + kvm_ioeventfd_flag_nr_pv_mmio, kvm_ioeventfd_flag_nr_max, };#define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch)#define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio) +/* +* PV_MMIO - Guest can promise us that all accesses touching this address +* are writes of specified length, starting at the specified address. +* If not - it's a Guest bug. +* Can not be used together with either PIO or DATAMATCH. +*/ +#define KVM_IOEVENTFD_FLAG_PV_MMIO (1 << kvm_ioeventfd_flag_nr_pv_mmio)#define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign)#define KVM_IOEVENTFD_VALID_FLAG_MASK ((1 << kvm_ioeventfd_flag_nr_max) - 1)
memory.c +7 -2
@@ -162,6 +162,7 @@ struct CoalescedMemoryRange {struct MemoryRegionIoeventfd {AddrRange addr; +bool pv;bool match_data;uint64_t data;EventNotifier *e;
@@ -600,7 +601,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, .offset_within_address_space = int128_get64(fd->addr.start), .size = int128_get64(fd->addr.size), }; -MEMORY_LISTENER_CALL (eventfd_del, Forward, §ion, +MEMORY_LISTENER_CALL (eventfd_del, Forward, §ion, fd->pv, fd->match_data, fd->data, fd->e); ++iold; } else if (inew < fds_new_nb
@@ -613,7 +614,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, .offset_within_address_space = int128_get64(fd->addr.start), .size = int128_get64(fd->addr.size), }; -MEMORY_LISTENER_CALL (eventfd_add, Reverse, §ion, +MEMORY_LISTENER_CALL (eventfd_add, Reverse, §ion, fd->pv, fd->match_data, fd->data, fd->e); ++inew; }else {
@@ -1243,6 +1244,7 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr)void memory_region_add_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e)
@@ -1250,6 +1252,7 @@ void memory_region_add_eventfd(MemoryRegion *mr,MemoryRegionIoeventfd mrfd = { .addr.start =int128_make64 (addr), .addr.size =int128_make64 (size), + .pv = pv, .match_data = match_data, .data = data, .e = e,
@@ -1276,6 +1279,7 @@ void memory_region_add_eventfd(MemoryRegion *mr,void memory_region_del_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e)
@@ -1283,6 +1287,7 @@ void memory_region_del_eventfd(MemoryRegion *mr,MemoryRegionIoeventfd mrfd = { .addr.start =int128_make64 (addr), .addr.size =int128_make64 (size), + .pv = pv, .match_data = match_data, .data = data, .e = e,
Signature · 1 line
MST
Paolo Bonzini
maintainerRe: [PATCH v2 5/6] kvm: add PV MMIO
Il 04/04/2013 12:40, Michael S. Tsirkin ha scritto:
Quoted 3 lines
Add an option for users to specify PV eventfd
listeners, and utilize KVM's PV MMIO underneath.
Upodate all callers.
I would like Avi to comment on this, because I think this is not the "memory-API approved" way of doing things. You need KVM to define its own AddressSpace, and make KVM's listener use memory_region_to_address_space to figure out if it is for PV MMIO.
To handle accesses from TCG, the PV AddressSpace can simply have just an alias to the actual MemoryRegion where the doorbell is.
Paolo
Quoted 315 lines
Signed-off-by Michael S. Tsirkin <[email protected]>--- hw/dataplane/hostmem.c | 1 + hw/ivshmem.c | 2 ++ hw/pci-testdev.c | 2 ++ hw/vhost.c | 4 ++-- hw/virtio-pci.c | 4 ++-- include/exec/memory.h | 10 ++++++++++ kvm-all.c | 15 ++++++++++++--- linux-headers/linux/kvm.h | 8 ++++++++ memory.c | 9 +++++++-- 9 files changed, 46 insertions(+), 9 deletions(-)hw/dataplane/hostmem.c +1
-0@@ -126,6 +126,7 @@ static void hostmem_listener_section_dummy(MemoryListener *listener,static void hostmem_listener_eventfd_dummy (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {hw/ivshmem.c +2
-0@@ -352,6 +352,7 @@ static void ivshmem_add_eventfd(IVShmemState *s, int posn, int i)memory_region_add_eventfd (&s->ivshmem_mmio,DOORBELL ,4 , + false, true, (posn <<16 ) | i, &s->peers[posn].eventfds[i]);@@ -362,6 +363,7 @@ static void ivshmem_del_eventfd(IVShmemState *s, int posn, int i)memory_region_del_eventfd (&s->ivshmem_mmio,DOORBELL ,4 , + false, true, (posn <<16 ) | i, &s->peers[posn].eventfds[i]);hw/pci-testdev.c +2
-0@@ -80,6 +80,7 @@ static int pci_testdev_start(IOTest *test)memory_region_add_eventfd (test->mr,le32_to_cpu (test->hdr->offset), test->size, + false, test->match_data, test->hdr->data, &test->notifier);@@ -94,6 +95,7 @@ static void pci_testdev_stop(IOTest *test)memory_region_del_eventfd (test->mr,le32_to_cpu (test->hdr->offset), test->size, + false, test->match_data, test->hdr->data, &test->notifier);hw/vhost.c +2
-2@@ -750,13 +750,13 @@ static void vhost_virtqueue_stop(struct vhost_dev *dev, }static void vhost_eventfd_add (MemoryListener *listener, -MemoryRegionSection *section, +MemoryRegionSection *section,bool pv,bool match_data,uint64_t data,EventNotifier *e) { }static void vhost_eventfd_del (MemoryListener *listener, -MemoryRegionSection *section, +MemoryRegionSection *section,bool pv,bool match_data,uint64_t data,EventNotifier *e) { }hw/virtio-pci.c +2
-2@@ -189,10 +189,10 @@ static int virtio_pci_set_host_notifier_internal(VirtIOPCIProxy *proxy, }virtio_queue_set_host_notifier_fd_handler (vq, true, set_handler);memory_region_add_eventfd (&proxy->bar,VIRTIO_PCI_QUEUE_NOTIFY ,2 , - true, n, notifier); + false, true, n, notifier); }else {memory_region_del_eventfd (&proxy->bar,VIRTIO_PCI_QUEUE_NOTIFY ,2 , - true, n, notifier); + false, true, n, notifier);virtio_queue_set_host_notifier_fd_handler (vq, false, false);event_notifier_cleanup (notifier); }include/exec/memory.h +10
-0@@ -215,8 +215,10 @@ struct MemoryListener {void (*log_global_start)(MemoryListener *listener);void (*log_global_stop)(MemoryListener *listener);void (*eventfd_add)(MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e);void (*eventfd_del)(MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e);void (*coalesced_mmio_add)(MemoryListener *listener,MemoryRegionSection *section,hwaddr addr,hwaddr len);@@ -594,6 +596,9 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr);* @mr: the memory region being updated. * @addr: the address within @mr that is to be monitored * @size: the size of the access to trigger the eventfd +* @pv: set if Guest can promise us that all accesses touching this address +* are writes of specified length, starting at the specified address. +* If not - it's a Guest bug. Might give faster access. * @match_data: whether to match against @data, instead of just @addr * @data: the data to match against the guest write * @fd: the eventfd to be triggered when @addr, @size, and @data all match. @@ -601,6 +606,7 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr);void memory_region_add_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e);@@ -614,6 +620,9 @@ void memory_region_add_eventfd(MemoryRegion *mr,* @mr: the memory region being updated. * @addr: the address within @mr that is to be monitored * @size: the size of the access to trigger the eventfd +* @pv: set if Guest can promise us that all accesses touching this address +* are writes of specified length, starting at the specified address. +* If not - it's a Guest bug. Might give faster access. * @match_data: whether to match against @data, instead of just @addr * @data: the data to match against the guest write * @fd: the eventfd to be triggered when @addr, @size, and @data all match. @@ -621,6 +630,7 @@ void memory_region_add_eventfd(MemoryRegion *mr,void memory_region_del_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e);kvm-all.c +12
-3@@ -500,7 +500,7 @@ int kvm_check_extension(KVMState *s, unsigned int extension)return ret; } -static int kvm_set_ioeventfd_mmio (int fd,uint32_t addr,uint32_t val, +static int kvm_set_ioeventfd_mmio (int fd,uint32_t addr,uint32_t val,bool pv,bool assign,uint32_t size,bool datamatch) {int ret;@@ -518,6 +518,11 @@ static int kvm_set_ioeventfd_mmio(int fd, uint32_t addr, uint32_t val,if (datamatch) { iofd.flags |=KVM_IOEVENTFD_FLAG_DATAMATCH ; +/* Kernel doesn't support data match with PV MMIO. */ + pv = false; + } +if (pv) { + iofd.flags |=KVM_IOEVENTFD_FLAG_PV_MMIO ; }if (!assign) { iofd.flags |=KVM_IOEVENTFD_FLAG_DEASSIGN ;@@ -802,6 +807,7 @@ static void kvm_log_global_stop(struct MemoryListener *listener)static void kvm_mem_ioeventfd_add (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {@@ -809,7 +815,7 @@ static void kvm_mem_ioeventfd_add(MemoryListener *listener,int r; r =kvm_set_ioeventfd_mmio (fd, section->offset_within_address_space, - data, true, section->size, match_data); + data, pv, true, section->size, match_data);if (r <0 ) {abort (); }@@ -817,6 +823,7 @@ static void kvm_mem_ioeventfd_add(MemoryListener *listener,static void kvm_mem_ioeventfd_del (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {@@ -824,7 +831,7 @@ static void kvm_mem_ioeventfd_del(MemoryListener *listener,int r; r =kvm_set_ioeventfd_mmio (fd, section->offset_within_address_space, - data, false, section->size, match_data); + data, pv, false, section->size, match_data);if (r <0 ) {abort (); }@@ -832,6 +839,7 @@ static void kvm_mem_ioeventfd_del(MemoryListener *listener,static void kvm_io_ioeventfd_add (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {@@ -847,6 +855,7 @@ static void kvm_io_ioeventfd_add(MemoryListener *listener,static void kvm_io_ioeventfd_del (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e)linux-headers/linux/kvm.h +8
-0@@ -449,11 +449,19 @@ enum { kvm_ioeventfd_flag_nr_datamatch, kvm_ioeventfd_flag_nr_pio, kvm_ioeventfd_flag_nr_deassign, + kvm_ioeventfd_flag_nr_pv_mmio, kvm_ioeventfd_flag_nr_max, };#define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch)#define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio) +/* +* PV_MMIO - Guest can promise us that all accesses touching this address +* are writes of specified length, starting at the specified address. +* If not - it's a Guest bug. +* Can not be used together with either PIO or DATAMATCH. +*/ +#define KVM_IOEVENTFD_FLAG_PV_MMIO (1 << kvm_ioeventfd_flag_nr_pv_mmio)#define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign)#define KVM_IOEVENTFD_VALID_FLAG_MASK ((1 << kvm_ioeventfd_flag_nr_max) - 1)memory.c +7
-2@@ -162,6 +162,7 @@ struct CoalescedMemoryRange {struct MemoryRegionIoeventfd {AddrRange addr; +bool pv;bool match_data;uint64_t data;EventNotifier *e;@@ -600,7 +601,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, .offset_within_address_space = int128_get64(fd->addr.start), .size = int128_get64(fd->addr.size), }; -MEMORY_LISTENER_CALL (eventfd_del, Forward, §ion, +MEMORY_LISTENER_CALL (eventfd_del, Forward, §ion, fd->pv, fd->match_data, fd->data, fd->e); ++iold; } else if (inew < fds_new_nb@@ -613,7 +614,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, .offset_within_address_space = int128_get64(fd->addr.start), .size = int128_get64(fd->addr.size), }; -MEMORY_LISTENER_CALL (eventfd_add, Reverse, §ion, +MEMORY_LISTENER_CALL (eventfd_add, Reverse, §ion, fd->pv, fd->match_data, fd->data, fd->e); ++inew; }else {@@ -1243,6 +1244,7 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr)void memory_region_add_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e)@@ -1250,6 +1252,7 @@ void memory_region_add_eventfd(MemoryRegion *mr,MemoryRegionIoeventfd mrfd = { .addr.start =int128_make64 (addr), .addr.size =int128_make64 (size), + .pv = pv, .match_data = match_data, .data = data, .e = e,@@ -1276,6 +1279,7 @@ void memory_region_add_eventfd(MemoryRegion *mr,void memory_region_del_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e)@@ -1283,6 +1287,7 @@ void memory_region_del_eventfd(MemoryRegion *mr,MemoryRegionIoeventfd mrfd = { .addr.start =int128_make64 (addr), .addr.size =int128_make64 (size), + .pv = pv, .match_data = match_data, .data = data, .e = e,
Michael S. Tsirkin
maintainerRe: [PATCH v2 5/6] kvm: add PV MMIO
Quoted 10 lines · Paolo Bonzini
Il 04/04/2013 12:40, Michael S. Tsirkin ha scritto:
Quoted 3 lines
Add an option for users to specify PV eventfd
listeners, and utilize KVM's PV MMIO underneath.
Upodate all callers.I would like Avi to comment on this, because I think this is not the "memory-API approved" way of doing things. You need KVM to define its own AddressSpace, and make KVM's listener use
memory_region_to_address_spaceto figure out if it is for PV MMIO.
Quoted 4 lines
To handle accesses from TCG, the PV AddressSpace can simply have just an alias to the actual MemoryRegion where the doorbell is.
Paolo
This is not really different from other eventfd flags like datamatch. Separate address space is not appropriate here.
This is regular memory space, KVM eventfd simply has a flag that says "guest is well behaved so please make eventfd go faster".
Quoted 315 lines
Quoted 315 lines
Signed-off-by Michael S. Tsirkin <[email protected]>--- hw/dataplane/hostmem.c | 1 + hw/ivshmem.c | 2 ++ hw/pci-testdev.c | 2 ++ hw/vhost.c | 4 ++-- hw/virtio-pci.c | 4 ++-- include/exec/memory.h | 10 ++++++++++ kvm-all.c | 15 ++++++++++++--- linux-headers/linux/kvm.h | 8 ++++++++ memory.c | 9 +++++++-- 9 files changed, 46 insertions(+), 9 deletions(-)hw/dataplane/hostmem.c +1
-0@@ -126,6 +126,7 @@ static void hostmem_listener_section_dummy(MemoryListener *listener,static void hostmem_listener_eventfd_dummy (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {hw/ivshmem.c +2
-0@@ -352,6 +352,7 @@ static void ivshmem_add_eventfd(IVShmemState *s, int posn, int i)memory_region_add_eventfd (&s->ivshmem_mmio,DOORBELL ,4 , + false, true, (posn <<16 ) | i, &s->peers[posn].eventfds[i]);@@ -362,6 +363,7 @@ static void ivshmem_del_eventfd(IVShmemState *s, int posn, int i)memory_region_del_eventfd (&s->ivshmem_mmio,DOORBELL ,4 , + false, true, (posn <<16 ) | i, &s->peers[posn].eventfds[i]);hw/pci-testdev.c +2
-0@@ -80,6 +80,7 @@ static int pci_testdev_start(IOTest *test)memory_region_add_eventfd (test->mr,le32_to_cpu (test->hdr->offset), test->size, + false, test->match_data, test->hdr->data, &test->notifier);@@ -94,6 +95,7 @@ static void pci_testdev_stop(IOTest *test)memory_region_del_eventfd (test->mr,le32_to_cpu (test->hdr->offset), test->size, + false, test->match_data, test->hdr->data, &test->notifier);hw/vhost.c +2
-2@@ -750,13 +750,13 @@ static void vhost_virtqueue_stop(struct vhost_dev *dev, }static void vhost_eventfd_add (MemoryListener *listener, -MemoryRegionSection *section, +MemoryRegionSection *section,bool pv,bool match_data,uint64_t data,EventNotifier *e) { }static void vhost_eventfd_del (MemoryListener *listener, -MemoryRegionSection *section, +MemoryRegionSection *section,bool pv,bool match_data,uint64_t data,EventNotifier *e) { }hw/virtio-pci.c +2
-2@@ -189,10 +189,10 @@ static int virtio_pci_set_host_notifier_internal(VirtIOPCIProxy *proxy, }virtio_queue_set_host_notifier_fd_handler (vq, true, set_handler);memory_region_add_eventfd (&proxy->bar,VIRTIO_PCI_QUEUE_NOTIFY ,2 , - true, n, notifier); + false, true, n, notifier); }else {memory_region_del_eventfd (&proxy->bar,VIRTIO_PCI_QUEUE_NOTIFY ,2 , - true, n, notifier); + false, true, n, notifier);virtio_queue_set_host_notifier_fd_handler (vq, false, false);event_notifier_cleanup (notifier); }include/exec/memory.h +10
-0@@ -215,8 +215,10 @@ struct MemoryListener {void (*log_global_start)(MemoryListener *listener);void (*log_global_stop)(MemoryListener *listener);void (*eventfd_add)(MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e);void (*eventfd_del)(MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e);void (*coalesced_mmio_add)(MemoryListener *listener,MemoryRegionSection *section,hwaddr addr,hwaddr len);@@ -594,6 +596,9 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr);* @mr: the memory region being updated. * @addr: the address within @mr that is to be monitored * @size: the size of the access to trigger the eventfd +* @pv: set if Guest can promise us that all accesses touching this address +* are writes of specified length, starting at the specified address. +* If not - it's a Guest bug. Might give faster access. * @match_data: whether to match against @data, instead of just @addr * @data: the data to match against the guest write * @fd: the eventfd to be triggered when @addr, @size, and @data all match. @@ -601,6 +606,7 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr);void memory_region_add_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e);@@ -614,6 +620,9 @@ void memory_region_add_eventfd(MemoryRegion *mr,* @mr: the memory region being updated. * @addr: the address within @mr that is to be monitored * @size: the size of the access to trigger the eventfd +* @pv: set if Guest can promise us that all accesses touching this address +* are writes of specified length, starting at the specified address. +* If not - it's a Guest bug. Might give faster access. * @match_data: whether to match against @data, instead of just @addr * @data: the data to match against the guest write * @fd: the eventfd to be triggered when @addr, @size, and @data all match. @@ -621,6 +630,7 @@ void memory_region_add_eventfd(MemoryRegion *mr,void memory_region_del_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e);kvm-all.c +12
-3@@ -500,7 +500,7 @@ int kvm_check_extension(KVMState *s, unsigned int extension)return ret; } -static int kvm_set_ioeventfd_mmio (int fd,uint32_t addr,uint32_t val, +static int kvm_set_ioeventfd_mmio (int fd,uint32_t addr,uint32_t val,bool pv,bool assign,uint32_t size,bool datamatch) {int ret;@@ -518,6 +518,11 @@ static int kvm_set_ioeventfd_mmio(int fd, uint32_t addr, uint32_t val,if (datamatch) { iofd.flags |=KVM_IOEVENTFD_FLAG_DATAMATCH ; +/* Kernel doesn't support data match with PV MMIO. */ + pv = false; + } +if (pv) { + iofd.flags |=KVM_IOEVENTFD_FLAG_PV_MMIO ; }if (!assign) { iofd.flags |=KVM_IOEVENTFD_FLAG_DEASSIGN ;@@ -802,6 +807,7 @@ static void kvm_log_global_stop(struct MemoryListener *listener)static void kvm_mem_ioeventfd_add (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {@@ -809,7 +815,7 @@ static void kvm_mem_ioeventfd_add(MemoryListener *listener,int r; r =kvm_set_ioeventfd_mmio (fd, section->offset_within_address_space, - data, true, section->size, match_data); + data, pv, true, section->size, match_data);if (r <0 ) {abort (); }@@ -817,6 +823,7 @@ static void kvm_mem_ioeventfd_add(MemoryListener *listener,static void kvm_mem_ioeventfd_del (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {@@ -824,7 +831,7 @@ static void kvm_mem_ioeventfd_del(MemoryListener *listener,int r; r =kvm_set_ioeventfd_mmio (fd, section->offset_within_address_space, - data, false, section->size, match_data); + data, pv, false, section->size, match_data);if (r <0 ) {abort (); }@@ -832,6 +839,7 @@ static void kvm_mem_ioeventfd_del(MemoryListener *listener,static void kvm_io_ioeventfd_add (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e) {@@ -847,6 +855,7 @@ static void kvm_io_ioeventfd_add(MemoryListener *listener,static void kvm_io_ioeventfd_del (MemoryListener *listener,MemoryRegionSection *section, +bool pv,bool match_data,uint64_t data,EventNotifier *e)linux-headers/linux/kvm.h +8
-0@@ -449,11 +449,19 @@ enum { kvm_ioeventfd_flag_nr_datamatch, kvm_ioeventfd_flag_nr_pio, kvm_ioeventfd_flag_nr_deassign, + kvm_ioeventfd_flag_nr_pv_mmio, kvm_ioeventfd_flag_nr_max, };#define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch)#define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio) +/* +* PV_MMIO - Guest can promise us that all accesses touching this address +* are writes of specified length, starting at the specified address. +* If not - it's a Guest bug. +* Can not be used together with either PIO or DATAMATCH. +*/ +#define KVM_IOEVENTFD_FLAG_PV_MMIO (1 << kvm_ioeventfd_flag_nr_pv_mmio)#define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign)#define KVM_IOEVENTFD_VALID_FLAG_MASK ((1 << kvm_ioeventfd_flag_nr_max) - 1)memory.c +7
-2@@ -162,6 +162,7 @@ struct CoalescedMemoryRange {struct MemoryRegionIoeventfd {AddrRange addr; +bool pv;bool match_data;uint64_t data;EventNotifier *e;@@ -600,7 +601,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, .offset_within_address_space = int128_get64(fd->addr.start), .size = int128_get64(fd->addr.size), }; -MEMORY_LISTENER_CALL (eventfd_del, Forward, §ion, +MEMORY_LISTENER_CALL (eventfd_del, Forward, §ion, fd->pv, fd->match_data, fd->data, fd->e); ++iold; } else if (inew < fds_new_nb@@ -613,7 +614,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, .offset_within_address_space = int128_get64(fd->addr.start), .size = int128_get64(fd->addr.size), }; -MEMORY_LISTENER_CALL (eventfd_add, Reverse, §ion, +MEMORY_LISTENER_CALL (eventfd_add, Reverse, §ion, fd->pv, fd->match_data, fd->data, fd->e); ++inew; }else {@@ -1243,6 +1244,7 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr)void memory_region_add_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e)@@ -1250,6 +1252,7 @@ void memory_region_add_eventfd(MemoryRegion *mr,MemoryRegionIoeventfd mrfd = { .addr.start =int128_make64 (addr), .addr.size =int128_make64 (size), + .pv = pv, .match_data = match_data, .data = data, .e = e,@@ -1276,6 +1279,7 @@ void memory_region_add_eventfd(MemoryRegion *mr,void memory_region_del_eventfd (MemoryRegion *mr,hwaddr addr,unsigned size, +bool pv,bool match_data,uint64_t data,EventNotifier *e)@@ -1283,6 +1287,7 @@ void memory_region_del_eventfd(MemoryRegion *mr,MemoryRegionIoeventfd mrfd = { .addr.start =int128_make64 (addr), .addr.size =int128_make64 (size), + .pv = pv, .match_data = match_data, .data = data, .e = e,
Paolo Bonzini
maintainerRe: [PATCH v2 5/6] kvm: add PV MMIO
Il 04/04/2013 15:10, Michael S. Tsirkin ha scritto:
Quoted 5 lines
Quoted 5 lines
I would like Avi to comment on this, because I think this is not the "memory-API approved" way of doing things. You need KVM to define its own AddressSpace, and make KVM's listener use
memory_region_to_address_spaceto figure out if it is for PV MMIO.
Quoted 7 lines
Quoted 2 lines
To handle accesses from TCG, the PV AddressSpace can simply have just an alias to the actual MemoryRegion where the doorbell is.
This is not really different from other eventfd flags like datamatch. Separate address space is not appropriate here.
This is regular memory space, KVM eventfd simply has a flag that says "guest is well behaved so please make eventfd go faster".
Having a separate address space would match what you do in the kernel though.
I just don't like functions with a dozen arguments... Can you just make datamatch and pv a single flags argument, as is the case with the KVM API?
Paolo
Michael S. Tsirkin
maintainerRe: [PATCH v2 5/6] kvm: add PV MMIO
Quoted 9 lines · Paolo Bonzini
Il 04/04/2013 15:10, Michael S. Tsirkin ha scritto:
Quoted 8 lines
Quoted 7 lines
I would like Avi to comment on this, because I think this is not the "memory-API approved" way of doing things. You need KVM to define its own AddressSpace, and make KVM's listener use
memory_region_to_address_spaceto figure out if it is for PV MMIO.To handle accesses from TCG, the PV AddressSpace can simply have just an alias to the actual MemoryRegion where the doorbell is.
Quoted 7 lines
Quoted 4 lines
This is not really different from other eventfd flags like datamatch. Separate address space is not appropriate here.
This is regular memory space, KVM eventfd simply has a flag that says "guest is well behaved so please make eventfd go faster".Having a separate address space would match what you do in the kernel though.
Not really, in kernel the device is as usual on the MMIO bus. The pv_mmio_bus is an extra datastructure to enable lookups where length is not known in an efficient way. Userspace does not need to know about these details.
Quoted 4 lines
I just don't like functions with a dozen arguments... Can you just make datamatch and pv a single flags argument, as is the case with the KVM API?
Paolo
Extra parameters are detected by compiler :) I don't really care, anyone else objects to changing datamatch to a flags argument?
Please let me know to avoid changing it back and forth.
Signature · 1 line
MST
Re: [Qemu-devel] [PATCH v2 5/6] kvm: add PV MMIO
In reply to Michael S. TsirkinQuoted 5 lines · Michael S. Tsirkin
Add an option for users to specify PV eventfd
listeners, and utilize KVM's PV MMIO underneath.
Upodate all callers.Signed-off-by Michael S. Tsirkin <[email protected]>
Quoted 10 lines
--- hw/dataplane/hostmem.c | 1 + hw/ivshmem.c | 2 ++ hw/pci-testdev.c | 2 ++ hw/vhost.c | 4 ++-- hw/virtio-pci.c | 4 ++-- include/exec/memory.h | 10 ++++++++++ kvm-all.c | 15 ++++++++++++--- linux-headers/linux/kvm.h | 8 ++++++++ memory.c | 9 +++++++--
Can we keep linux-headers updates in separate patches, please (with commit messages saying what kernel commit/version the sync is with)?
thanks
-- PMM
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCH v2 5/6] kvm: add PV MMIO
Quoted 17 lines · Peter Maydell
Quoted 5 lines · Michael S. Tsirkin
Add an option for users to specify PV eventfd
listeners, and utilize KVM's PV MMIO underneath.
Upodate all callers.Signed-off-by Michael S. Tsirkin <[email protected]>Quoted 10 lines
--- hw/dataplane/hostmem.c | 1 + hw/ivshmem.c | 2 ++ hw/pci-testdev.c | 2 ++ hw/vhost.c | 4 ++-- hw/virtio-pci.c | 4 ++-- include/exec/memory.h | 10 ++++++++++ kvm-all.c | 15 ++++++++++++--- linux-headers/linux/kvm.h | 8 ++++++++ memory.c | 9 +++++++--
Quoted 6 lines
Can we keep linux-headers updates in separate patches, please (with commit messages saying what kernel commit/version the sync is with)?
thanks
-- PMM
OK, first it needs to get into linux though :).
Signature · 1 line
MST
Re: [Qemu-devel] [PATCH v2 5/6] kvm: add PV MMIO
Quoted 6 lines · Michael S. Tsirkin
Quoted 3 lines · Peter Maydell
Can we keep linux-headers updates in separate patches, please (with commit messages saying what kernel commit/version the sync is with)?
OK, first it needs to get into linux though :).
In that case you need an RFC tag on your patches :-)
-- PMM
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCH v2 5/6] kvm: add PV MMIO
Quoted 11 lines · Peter Maydell
Quoted 6 lines · Michael S. Tsirkin
Quoted 3 lines · Peter Maydell
Can we keep linux-headers updates in separate patches, please (with commit messages saying what kernel commit/version the sync is with)?
OK, first it needs to get into linux though :).
In that case you need an RFC tag on your patches :-)
-- PMM
The cover letter said patches 5 and 6 are RFC, but patches 1-4 are ready and should go in I think.
Signature · 1 line
MST
Michael S. Tsirkin
maintainerPATCH v2 6/6 pci-testdev: add pv mmio test
In reply to Michael S. TsirkinAdd ability to test the speed of PV MMIO.
Signed-off-by Michael S. Tsirkin <[email protected]>
--- hw/pci-testdev.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
hw/pci-testdev.c +7 -4
@@ -19,6 +19,7 @@ typedef struct IOTest {EventNotifier notifier;bool hasnotifier;unsigned size; +bool pv;bool match_data;PCITestDevHdr *hdr;unsigned bufsize;
@@ -33,7 +34,8 @@ typedef struct IOTest {static const char *iotest_test[] = {"no-eventfd" ,"wildcard-eventfd" , -"datamatch-eventfd" +"datamatch-eventfd" , +"pv-eventfd" };static const char *iotest_type[] = {
@@ -80,7 +82,7 @@ static int pci_testdev_start(IOTest *test)memory_region_add_eventfd (test->mr,le32_to_cpu (test->hdr->offset), test->size, - false, + test->pv, test->match_data, test->hdr->data, &test->notifier);
@@ -95,7 +97,7 @@ static void pci_testdev_stop(IOTest *test)memory_region_del_eventfd (test->mr,le32_to_cpu (test->hdr->offset), test->size, - false, + test->pv, test->match_data, test->hdr->data, &test->notifier);
@@ -237,7 +239,8 @@ static int pci_testdev_init(PCIDevice *pci_dev)g_free (name); test->hdr->offset =cpu_to_le32 (IOTEST_SIZE (i) + i *IOTEST_ACCESS_WIDTH ); test->size =IOTEST_ACCESS_WIDTH ; - test->match_data =strcmp (IOTEST_TEST (i),"wildcard-eventfd" ); + test->pv = !strcmp (IOTEST_TEST (i),"pv-eventfd" ); + test->match_data = !strcmp (IOTEST_TEST (i),"datamatch-eventfd" ); test->hdr->test = i; test->hdr->data = test->match_data ?IOTEST_DATAMATCH :IOTEST_NOMATCH ; test->hdr->width =IOTEST_ACCESS_WIDTH ;
Signature · 1 line
MST