kvm: Make ioeventfd usable on s390.
Subsystems (5)
v5 cover kvm: Make ioeventfd usable on s390.
v5 of the ioeventfd patch set, this time with a proper return code from __diag_virtio_hypercall(), otherwise unchanged.
v4 -> v5:
- Proper return code in __diag_virtio_hypercall()
v3 -> v4:
- Pass cookies in virtio-ccw notify hypercall
- Coding style
v2 -> v3:
- Added a patch exporting the virtio-ccw api and use it for the diagnose implementation.
- Better naming: We're dealing with virtio-ccw notifications only. v1 -> v2:
- Move irqfd initialization from a module init function to kvm_init, eliminating the need for a second module for kvm/s390.
- Use kvm_io_device for s390 css devices.
Cornelia Huck (5):
KVM: s390: Export virtio-ccw api.
KVM: Initialize irqfd from kvm_init().
KVM: Introduce KVM_VIRTIO_CCW_NOTIFY_BUS.
KVM: ioeventfd for virtio-ccw devices.
KVM: s390: Wire up ioeventfd.
Michael S. Tsirkin (1):
virtio_ccw: pass a cookie value to kvm hypercall
Documentation/virtual/kvm/api.txt | 8 ++++++++ arch/s390/include/uapi/asm/Kbuild | 1 + arch/s390/include/uapi/asm/virtio-ccw.h | 21 +++++++++++++++++++++ arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 26 ++++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + drivers/s390/kvm/virtio_ccw.c | 16 +++++++++------- include/linux/kvm_host.h | 14 ++++++++++++++ include/uapi/linux/kvm.h | 3 +++ virt/kvm/eventfd.c | 24 +++++++++++++++--------- virt/kvm/kvm_main.c | 6 ++++++ 12 files changed, 106 insertions(+), 17 deletions(-) create mode 100644 arch/s390/include/uapi/asm/virtio-ccw.h
Signature · 1 line
1.7.12.4
PATCH v5 1/6 virtio_ccw: pass a cookie value to kvm hypercall
From: "Michael S. Tsirkin" <[email protected]>
Lookups by channel/vq pair on host during virtio notifications might be expensive. Interpret hypercall return value as a cookie which host can use to do device lookups for the next notification more efficiently.
--- drivers/s390/kvm/virtio_ccw.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
drivers/s390/kvm/virtio_ccw.c +8 -3
@@ -77,6 +77,7 @@ struct virtio_ccw_vq_info {void *queue;struct vq_info_block *info_block;struct list_head node; +long cookie; };#define KVM_VIRTIO_CCW_RING_ALIGN 4096
@@ -145,15 +146,18 @@ static int ccw_io_helper(struct virtio_ccw_device *vcdev, }static inline long do_kvm_notify (struct subchannel_id schid, -unsigned long queue_index) +unsigned long queue_index, +long cookie) { register unsigned long __nr asm("1") = KVM_S390_VIRTIO_CCW_NOTIFY; register struct subchannel_id __schid asm("2") = schid; register unsigned long __index asm("3") = queue_index; registerlong __rc asm("2" ); + register long __cookie asm("4") = cookie; asmvolatile ("diag 2,4,0x500\n" - :"=d" (__rc) :"d" (__nr),"d" (__schid),"d" (__index) + :"=d" (__rc) :"d" (__nr),"d" (__schid),"d" (__index), +"d" (__cookie) :"memory" ,"cc" );return __rc; }
@@ -166,7 +170,8 @@ static void virtio_ccw_kvm_notify(struct virtqueue *vq) vcdev =to_vc_device (info->vq->vdev);ccw_device_get_schid (vcdev->cdev, &schid); -do_kvm_notify (schid,virtqueue_get_queue_index (vq)); + info->cookie =do_kvm_notify (schid,virtqueue_get_queue_index (vq), + info->cookie); } static int virtio_ccw_read_vq_conf(struct virtio_ccw_device *vcdev,
Signature · 1 line
1.7.12.4
PATCH v5 2/6 KVM: s390: Export virtio-ccw api.
In reply to Cornelia HuckExport the virtio-ccw api in a header for usage by other code.
--- arch/s390/include/uapi/asm/Kbuild | 1 + arch/s390/include/uapi/asm/virtio-ccw.h | 21 +++++++++++++++++++++ drivers/s390/kvm/virtio_ccw.c | 5 +---- 3 files changed, 23 insertions(+), 4 deletions(-) create mode 100644 arch/s390/include/uapi/asm/virtio-ccw.h
arch/s390/include/uapi/asm/Kbuild +1 -0
@@ -44,5 +44,6 @@ header-y += termios.h header-y += types.h header-y += ucontext.h header-y += unistd.h +header-y += virtio-ccw.h header-y += vtoc.h header-y += zcrypt.h
arch/s390/include/uapi/asm/virtio-ccw.h new file +21 -0
@@ -0,0 +1,21 @@ +/* +* Definitions for virtio-ccw devices. +* +* Copyright IBM Corp. 2013 +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License (version 2 only) +* as published by the Free Software Foundation. +* +* Author(s): Cornelia Huck <[email protected]> +*/ +#ifndef __KVM_VIRTIO_CCW_H +#define __KVM_VIRTIO_CCW_H + +/* Alignment of vring buffers. */ +#define KVM_VIRTIO_CCW_RING_ALIGN 4096 + +/* Subcode for diagnose 500 (virtio hypercall). */ +#define KVM_S390_VIRTIO_CCW_NOTIFY 3 + +#endif
drivers/s390/kvm/virtio_ccw.c +1 -4
@@ -31,6 +31,7 @@#include <asm/irq.h> #include <asm/cio.h> #include <asm/ccwdev.h> +#include <asm/virtio-ccw.h> /* * virtio related functions
@@ -80,10 +81,6 @@ struct virtio_ccw_vq_info {long cookie; }; -#define KVM_VIRTIO_CCW_RING_ALIGN 4096 - -#define KVM_S390_VIRTIO_CCW_NOTIFY 3 -#define CCW_CMD_SET_VQ 0x13#define CCW_CMD_VDEV_RESET 0x33#define CCW_CMD_SET_IND 0x43
Signature · 1 line
1.7.12.4
PATCH v5 3/6 KVM: Initialize irqfd from kvm_init().
In reply to Cornelia HuckCurrently, eventfd introduces module_init/module_exit functions to initialize/cleanup the irqfd workqueue. This only works, however, if no other module_init/module_exit functions are built into the same module.
Let's just move the initialization and cleanup to kvm_init and kvm_exit. This way, it is also clearer where kvm startup may fail.
--- include/linux/kvm_host.h | 13 +++++++++++++ virt/kvm/eventfd.c | 7 ++----- virt/kvm/kvm_main.c | 6 ++++++ 3 files changed, 21 insertions(+), 5 deletions(-)
include/linux/kvm_host.h +13 -0
@@ -423,6 +423,19 @@ void kvm_vcpu_uninit(struct kvm_vcpu *vcpu);int __must_check vcpu_load (struct kvm_vcpu *vcpu);void vcpu_put (struct kvm_vcpu *vcpu); +#ifdef __KVM_HAVE_IOAPIC +int kvm_irqfd_init (void ); +void kvm_irqfd_exit (void ); +#else +static inline int kvm_irqfd_init (void ) +{ +return 0 ; +} + +static inline void kvm_irqfd_exit (void ) +{ +} +#endif int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align,struct module *module);void kvm_exit (void );
virt/kvm/eventfd.c +2 -5
@@ -544,7 +544,7 @@ void kvm_irq_routing_update(struct kvm *kvm,* aggregated from all vm* instances. We need our own isolated single-thread * queue to prevent deadlock against flushing the normal work-queue. */ -static int __init irqfd_module_init (void ) +int kvm_irqfd_init (void ) { irqfd_cleanup_wq =create_singlethread_workqueue ("kvm-irqfd-cleanup" );if (!irqfd_cleanup_wq)
@@ -553,13 +553,10 @@ static int __init irqfd_module_init(void) return 0; } -static void __exit irqfd_module_exit(void) +void kvm_irqfd_exit (void ) {destroy_workqueue (irqfd_cleanup_wq); } - -module_init (irqfd_module_init); -module_exit (irqfd_module_exit);#endif /*
virt/kvm/kvm_main.c +6 -0
@@ -2920,6 +2920,9 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,int r;int cpu; + r =kvm_irqfd_init (); +if (r) + goto out_irqfd; r =kvm_arch_init (opaque);if (r) goto out_fail;
@@ -3000,6 +3003,8 @@ out_free_0a: out_free_0:kvm_arch_exit (); out_fail: +kvm_irqfd_exit (); +out_irqfd:return r; }EXPORT_SYMBOL_GPL (kvm_init);
@@ -3016,6 +3021,7 @@ void kvm_exit(void)on_each_cpu (hardware_disable_nolock,NULL ,1 );kvm_arch_hardware_unsetup ();kvm_arch_exit (); +kvm_irqfd_exit ();free_cpumask_var (cpus_hardware_enabled); }EXPORT_SYMBOL_GPL (kvm_exit);
Signature · 1 line
1.7.12.4
PATCH v5 4/6 KVM: Introduce KVM_VIRTIO_CCW_NOTIFY_BUS.
In reply to Cornelia HuckAdd a new bus type for virtio-ccw devices on s390.
--- include/linux/kvm_host.h | 1 + 1 file changed, 1 insertion(+)
include/linux/kvm_host.h +1 -0
@@ -148,6 +148,7 @@ struct kvm_io_bus {enum kvm_bus {KVM_MMIO_BUS ,KVM_PIO_BUS , +KVM_VIRTIO_CCW_NOTIFY_BUS ,KVM_NR_BUSES };
Signature · 1 line
1.7.12.4
PATCH v5 5/6 KVM: ioeventfd for virtio-ccw devices.
In reply to Cornelia HuckEnhance KVM_IOEVENTFD with a new flag that allows to attach to virtio-ccw devices on s390 via the KVM_VIRTIO_CCW_NOTIFY_BUS.
--- Documentation/virtual/kvm/api.txt | 8 ++++++++ include/uapi/linux/kvm.h | 3 +++ virt/kvm/eventfd.c | 17 +++++++++++++---- 3 files changed, 24 insertions(+), 4 deletions(-)
Documentation/virtual/kvm/api.txt +8 -0
@@ -1468,15 +1468,23 @@ struct kvm_ioeventfd { __u8 pad[36]; }; +For the special case of virtio-ccw devices on s390, the ioevent is matched +to a subchannel/virtqueue tuple instead. + The following flags are defined: #define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch) #define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio) #define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign) +#define KVM_IOEVENTFD_FLAG_VIRTIO_CCW_NOTIFY \ + (1 << kvm_ioeventfd_flag_nr_virtio_ccw_notify) If datamatch flag is set, the event will be signaled only if the written value to the registered address is equal to datamatch in struct kvm_ioeventfd. +For virtio-ccw devices, addr contains the subchannel id and datamatch the +virtqueue index. + 4.60 KVM_DIRTY_TLB
include/uapi/linux/kvm.h +3 -0
@@ -448,12 +448,15 @@ enum { kvm_ioeventfd_flag_nr_datamatch, kvm_ioeventfd_flag_nr_pio, kvm_ioeventfd_flag_nr_deassign, + kvm_ioeventfd_flag_nr_virtio_ccw_notify, kvm_ioeventfd_flag_nr_max, };#define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch)#define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio)#define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign) +#define KVM_IOEVENTFD_FLAG_VIRTIO_CCW_NOTIFY \ + (1 << kvm_ioeventfd_flag_nr_virtio_ccw_notify)#define KVM_IOEVENTFD_VALID_FLAG_MASK ((1 << kvm_ioeventfd_flag_nr_max) - 1)
virt/kvm/eventfd.c +13 -4
@@ -675,15 +675,24 @@ ioeventfd_check_collision(struct kvm *kvm, struct _ioeventfd *p)return false; } +static enum kvm_bus ioeventfd_bus_from_flags (__u32 flags) +{ +if (flags &KVM_IOEVENTFD_FLAG_PIO ) +return KVM_PIO_BUS ; +if (flags &KVM_IOEVENTFD_FLAG_VIRTIO_CCW_NOTIFY ) +return KVM_VIRTIO_CCW_NOTIFY_BUS ; +return KVM_MMIO_BUS ; +} +static int kvm_assign_ioeventfd (struct kvm *kvm,struct kvm_ioeventfd *args) { -int pio = args->flags &KVM_IOEVENTFD_FLAG_PIO ; -enum kvm_bus bus_idx = pio ?KVM_PIO_BUS :KVM_MMIO_BUS ; +enum kvm_bus bus_idx;struct _ioeventfd *p;struct eventfd_ctx *eventfd;int ret; + bus_idx =ioeventfd_bus_from_flags (args->flags);/* must be natural-word sized */ switch (args->len) {case 1 :
@@ -758,12 +767,12 @@ fail:static int kvm_deassign_ioeventfd (struct kvm *kvm,struct kvm_ioeventfd *args) { -int pio = args->flags &KVM_IOEVENTFD_FLAG_PIO ; -enum kvm_bus bus_idx = pio ?KVM_PIO_BUS :KVM_MMIO_BUS ; +enum kvm_bus bus_idx;struct _ioeventfd *p, *tmp;struct eventfd_ctx *eventfd;int ret = -ENOENT ; + bus_idx =ioeventfd_bus_from_flags (args->flags); eventfd =eventfd_ctx_fdget (args->fd);if (IS_ERR (eventfd))return PTR_ERR (eventfd);
Signature · 1 line
1.7.12.4
PATCH v5 6/6 KVM: s390: Wire up ioeventfd.
In reply to Cornelia HuckEnable ioeventfd support on s390 and hook up diagnose 500 virtio-ccw notifications.
--- arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 26 ++++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + 4 files changed, 29 insertions(+), 1 deletion(-)
arch/s390/kvm/Kconfig +1 -0
@@ -22,6 +22,7 @@ config KVM select PREEMPT_NOTIFIERS select ANON_INODES select HAVE_KVM_CPU_RELAX_INTERCEPT + select HAVE_KVM_EVENTFD ---help--- Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should work
arch/s390/kvm/Makefile +1 -1
@@ -6,7 +6,7 @@ # it under the terms of the GNU General Public License (version 2 only) # as published by the Free Software Foundation. -common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o) +common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o eventfd.o) ccflags-y := -Ivirt/kvm -Iarch/s390/kvm
arch/s390/kvm/diag.c +26 -0
@@ -13,6 +13,7 @@#include <linux/kvm.h> #include <linux/kvm_host.h> +#include <asm/virtio-ccw.h> #include "kvm-s390.h" #include "trace.h" #include "trace-s390.h"
@@ -104,6 +105,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)return -EREMOTE ; } +static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu) +{ +int ret, idx; + +/* No virtio-ccw notification? Get out quickly. */ +if (!vcpu->kvm->arch.css_support || + (vcpu->run->s.regs.gprs[1 ] !=KVM_S390_VIRTIO_CCW_NOTIFY )) +return -EOPNOTSUPP ; + + idx =srcu_read_lock (&vcpu->kvm->srcu); +/* +* The layout is as follows: +* - gpr 2 contains the subchannel id (passed as addr) +* - gpr 3 contains the virtqueue index (passed as datamatch) +*/ + ret =kvm_io_bus_write (vcpu->kvm,KVM_VIRTIO_CCW_NOTIFY_BUS , + vcpu->run->s.regs.gprs[2 ], +8 , &vcpu->run->s.regs.gprs[3 ]); +srcu_read_unlock (&vcpu->kvm->srcu, idx); +/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ +return ret <0 ? ret :0 ; +} +int kvm_s390_handle_diag (struct kvm_vcpu *vcpu) {int code = (vcpu->arch.sie_block->ipb &0xfff0000 ) >>16 ;
@@ -118,6 +142,8 @@ int kvm_s390_handle_diag(struct kvm_vcpu *vcpu)return __diag_time_slice_end_directed (vcpu);case 0x308 :return __diag_ipl_functions (vcpu); +case 0x500 : +return __diag_virtio_hypercall (vcpu);default :return -EOPNOTSUPP ; }
arch/s390/kvm/kvm-s390.c +1 -0
@@ -142,6 +142,7 @@ int kvm_dev_ioctl_check_extension(long ext)case KVM_CAP_ONE_REG :case KVM_CAP_ENABLE_CAP :case KVM_CAP_S390_CSS_SUPPORT : +case KVM_CAP_IOEVENTFD : r =1 ;break ;case KVM_CAP_NR_VCPUS :
Signature · 1 line
1.7.12.4
Re: [PATCH v5 6/6] KVM: s390: Wire up ioeventfd.
Quoted 65 lines · Cornelia Huck
Enable ioeventfd support on s390 and hook up diagnose 500 virtio-ccw notifications.
Signed-off-by Cornelia Huck <[email protected]>--- arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 26 ++++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + 4 files changed, 29 insertions(+), 1 deletion(-)arch/s390/kvm/Kconfig +1
-0@@ -22,6 +22,7 @@ config KVM select PREEMPT_NOTIFIERS select ANON_INODES select HAVE_KVM_CPU_RELAX_INTERCEPT + select HAVE_KVM_EVENTFD ---help--- Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should workarch/s390/kvm/Makefile +1
-1@@ -6,7 +6,7 @@ # it under the terms of the GNU General Public License (version 2 only) # as published by the Free Software Foundation. -common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o) +common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o eventfd.o) ccflags-y := -Ivirt/kvm -Iarch/s390/kvmarch/s390/kvm/diag.c +14
-0@@ -13,6 +13,7 @@#include <linux/kvm.h> #include <linux/kvm_host.h> +#include <asm/virtio-ccw.h> #include "kvm-s390.h" #include "trace.h" #include "trace-s390.h" @@ -104,6 +105,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)return -EREMOTE ; } +static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu) +{ +int ret, idx; + +/* No virtio-ccw notification? Get out quickly. */ +if (!vcpu->kvm->arch.css_support || + (vcpu->run->s.regs.gprs[1 ] !=KVM_S390_VIRTIO_CCW_NOTIFY )) +return -EOPNOTSUPP ; + + idx =srcu_read_lock (&vcpu->kvm->srcu); +/* +* The layout is as follows: +* - gpr 2 contains the subchannel id (passed as addr)
arch/s390/kvm/diag.c static int __diag_ipl_functions(struct kvm_vcpu *vcpu) · line 130
Quoted 10 lines
arch/s390/kvm/diag.c +10
-0+* - gpr 3 contains the virtqueue index (passed as datamatch) +*/ + ret =kvm_io_bus_write (vcpu->kvm,KVM_VIRTIO_CCW_NOTIFY_BUS , + vcpu->run->s.regs.gprs[2 ], +8 , &vcpu->run->s.regs.gprs[3 ]); +srcu_read_unlock (&vcpu->kvm->srcu, idx); +/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ +return ret <0 ? ret :0 ; +} +
What about the cookie?
Re: [PATCH v5 6/6] KVM: s390: Wire up ioeventfd.
Quoted 70 lines · On Mon, 4 Mar 2013 21:07:41 -0300 Marcelo Tosatti <[email protected]> wrote:
Quoted 69 lines · Cornelia Huck
Enable ioeventfd support on s390 and hook up diagnose 500 virtio-ccw notifications.
Signed-off-by Cornelia Huck <[email protected]>--- arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 26 ++++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + 4 files changed, 29 insertions(+), 1 deletion(-)arch/s390/kvm/Kconfig +1
-0@@ -22,6 +22,7 @@ config KVM select PREEMPT_NOTIFIERS select ANON_INODES select HAVE_KVM_CPU_RELAX_INTERCEPT + select HAVE_KVM_EVENTFD ---help--- Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should workarch/s390/kvm/Makefile +1
-1@@ -6,7 +6,7 @@ # it under the terms of the GNU General Public License (version 2 only) # as published by the Free Software Foundation. -common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o) +common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o eventfd.o) ccflags-y := -Ivirt/kvm -Iarch/s390/kvmarch/s390/kvm/diag.c +18
-0@@ -13,6 +13,7 @@#include <linux/kvm.h> #include <linux/kvm_host.h> +#include <asm/virtio-ccw.h> #include "kvm-s390.h" #include "trace.h" #include "trace-s390.h" @@ -104,6 +105,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)return -EREMOTE ; } +static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu) +{ +int ret, idx; + +/* No virtio-ccw notification? Get out quickly. */ +if (!vcpu->kvm->arch.css_support || + (vcpu->run->s.regs.gprs[1 ] !=KVM_S390_VIRTIO_CCW_NOTIFY )) +return -EOPNOTSUPP ; + + idx =srcu_read_lock (&vcpu->kvm->srcu); +/* +* The layout is as follows: +* - gpr 2 contains the subchannel id (passed as addr) +* - gpr 3 contains the virtqueue index (passed as datamatch) +*/ + ret = kvm_io_bus_write(vcpu->kvm, KVM_VIRTIO_CCW_NOTIFY_BUS, + vcpu->run->s.regs.gprs[2],
Quoted 8 lines
arch/s390/kvm/diag.c static int __diag_ipl_functions(struct kvm_vcpu *vcpu) · line 130
Quoted 6 lines
arch/s390/kvm/diag.c +6
-0+ 8, &vcpu->run->s.regs.gprs[3]); +srcu_read_unlock (&vcpu->kvm->srcu, idx); +/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ +return ret <0 ? ret :0 ; +} +What about the cookie?
This is the not-performance-optimized version which went through our testing and which I'd like to see applied. I also have a not-yet-tested patch that adds cookie support (see below), but I'd like to postpone this to 3.10.
commit 8fce4a8a3478252f7ee2eb74d2732673f5911120
Author: Cornelia Huck <[email protected]> Date: Tue Feb 26 17:05:01 2013 +0100
virtio-ccw: ioeventfd: use cookies
arch/s390/kvm/diag.c +10 -3
@@ -108,22 +108,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu) {int ret, idx; +long cookie;/* No virtio-ccw notification? Get out quickly. */ if (!vcpu->kvm->arch.css_support || (vcpu->run->s.regs.gprs[1 ] !=KVM_S390_VIRTIO_CCW_NOTIFY ))return -EOPNOTSUPP ; + cookie = vcpu->run->s.regs.gprs[4 ]; idx =srcu_read_lock (&vcpu->kvm->srcu);/* * The layout is as follows: * - gpr 2 contains the subchannel id (passed as addr) * - gpr 3 contains the virtqueue index (passed as datamatch) +* - gpr 4 contains the index on the bus (optionally) */ - ret =kvm_io_bus_write (vcpu->kvm,KVM_VIRTIO_CCW_NOTIFY_BUS , - vcpu->run->s.regs.gprs[2 ], -8 , &vcpu->run->s.regs.gprs[3 ]); + ret =kvm_io_bus_write_cookie (vcpu->kvm,KVM_VIRTIO_CCW_NOTIFY_BUS , + vcpu->run->s.regs.gprs[2 ], +8 , &vcpu->run->s.regs.gprs[3 ], + &cookie);srcu_read_unlock (&vcpu->kvm->srcu, idx); +if (!ret) +/* Return cookie in gpr 2. */ + vcpu->run->s.regs.gprs[2 ] = cookie;/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ return ret <0 ? ret :0 ; }
include/linux/kvm_host.h +2 -0
@@ -154,6 +154,8 @@ enum kvm_bus {int kvm_io_bus_write (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr,int len,const void *val); +int kvm_io_bus_write_cookie (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr, +int len,const void *val,long *cookie);int kvm_io_bus_read (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr,int len,void *val); int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
virt/kvm/kvm_main.c +45 -0
51 lines not on this page · raw message
Re: [PATCH v5 6/6] KVM: s390: Wire up ioeventfd.
Quoted 82 lines · Cornelia Huck
Quoted 70 lines · On Mon, 4 Mar 2013 21:07:41 -0300 Marcelo Tosatti <[email protected]> wrote:
Quoted 69 lines · Cornelia Huck
Enable ioeventfd support on s390 and hook up diagnose 500 virtio-ccw notifications.
Signed-off-by Cornelia Huck <[email protected]>--- arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 26 ++++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + 4 files changed, 29 insertions(+), 1 deletion(-)arch/s390/kvm/Kconfig +1
-0@@ -22,6 +22,7 @@ config KVM select PREEMPT_NOTIFIERS select ANON_INODES select HAVE_KVM_CPU_RELAX_INTERCEPT + select HAVE_KVM_EVENTFD ---help--- Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should workarch/s390/kvm/Makefile +1
-1@@ -6,7 +6,7 @@ # it under the terms of the GNU General Public License (version 2 only) # as published by the Free Software Foundation. -common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o) +common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o eventfd.o) ccflags-y := -Ivirt/kvm -Iarch/s390/kvmarch/s390/kvm/diag.c +18
-0@@ -13,6 +13,7 @@#include <linux/kvm.h> #include <linux/kvm_host.h> +#include <asm/virtio-ccw.h> #include "kvm-s390.h" #include "trace.h" #include "trace-s390.h" @@ -104,6 +105,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)return -EREMOTE ; } +static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu) +{ +int ret, idx; + +/* No virtio-ccw notification? Get out quickly. */ +if (!vcpu->kvm->arch.css_support || + (vcpu->run->s.regs.gprs[1 ] !=KVM_S390_VIRTIO_CCW_NOTIFY )) +return -EOPNOTSUPP ; + + idx =srcu_read_lock (&vcpu->kvm->srcu); +/* +* The layout is as follows: +* - gpr 2 contains the subchannel id (passed as addr) +* - gpr 3 contains the virtqueue index (passed as datamatch) +*/ + ret = kvm_io_bus_write(vcpu->kvm, KVM_VIRTIO_CCW_NOTIFY_BUS, + vcpu->run->s.regs.gprs[2],Quoted 8 lines
arch/s390/kvm/diag.c static int __diag_ipl_functions(struct kvm_vcpu *vcpu) · line 130
Quoted 6 lines
arch/s390/kvm/diag.c +6
-0+ 8, &vcpu->run->s.regs.gprs[3]); +srcu_read_unlock (&vcpu->kvm->srcu, idx); +/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ +return ret <0 ? ret :0 ; +} +What about the cookie?
Quoted 4 lines
This is the not-performance-optimized version which went through our testing and which I'd like to see applied. I also have a not-yet-tested patch that adds cookie support (see below), but I'd like to postpone this to 3.10.
I agree it's best to delay this optimization, focus on basic functionality now. As guest with cookie support is compatible with host without said support (all we do really is reserve a gpr), there won't be issues adding it later. Some comments on the draft below.
Quoted 14 lines
commit
8fce4a8a3478252f7ee2eb74d2732673f5911120Author: Cornelia Huck <[email protected]> Date: Tue Feb 26 17:05:01 2013 +0100virtio-ccw: ioeventfd: use cookies
Signed-off-by Cornelia Huck <[email protected]>arch/s390/kvm/diag.c +0
-0@@ -108,22 +108,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu)
arch/s390/kvm/diag.c static int __diag_ipl_functions(struct kvm_vcpu *vcpu) · line 118
Quoted 10 lines
arch/s390/kvm/diag.c +2
-0{int ret, idx; +long cookie;/* No virtio-ccw notification? Get out quickly. */ if (!vcpu->kvm->arch.css_support || (vcpu->run->s.regs.gprs[1 ] !=KVM_S390_VIRTIO_CCW_NOTIFY ))return -EOPNOTSUPP ; + cookie = vcpu->run->s.regs.gprs[4 ];
Hmm signed so this can become negative.
Quoted 8 lines
arch/s390/kvm/diag.c +1
-1idx =srcu_read_lock (&vcpu->kvm->srcu);/* * The layout is as follows: * - gpr 2 contains the subchannel id (passed as addr) * - gpr 3 contains the virtqueue index (passed as datamatch) +* - gpr 4 contains the index on the bus (optionally) */ - ret = kvm_io_bus_write(vcpu->kvm, KVM_VIRTIO_CCW_NOTIFY_BUS,
arch/s390/kvm/diag.c static int __diag_ipl_functions(struct kvm_vcpu *vcpu) · line 133
Quoted 10 lines
arch/s390/kvm/diag.c +7
-2- vcpu->run->s.regs.gprs[2 ], - 8, &vcpu->run->s.regs.gprs[3]); + ret =kvm_io_bus_write_cookie (vcpu->kvm,KVM_VIRTIO_CCW_NOTIFY_BUS , + vcpu->run->s.regs.gprs[2 ], +8 , &vcpu->run->s.regs.gprs[3 ], + &cookie);srcu_read_unlock (&vcpu->kvm->srcu, idx); +if (!ret) +/* Return cookie in gpr 2. */ + vcpu->run->s.regs.gprs[2 ] = cookie;
We should also set gprs[2] to something sane if exiting to userspace so avoid adding overhead for that case.
Probably # of devices + 1 or a negative value.
Are there 32 bit guests? If yes can they pass a negative value?
Quoted 30 lines
arch/s390/kvm/diag.c +0
-0/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ return ret <0 ? ret :0 ; }include/linux/kvm_host.h +2
-0@@ -154,6 +154,8 @@ enum kvm_bus {int kvm_io_bus_write (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr,int len,const void *val); +int kvm_io_bus_write_cookie (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr, +int len,const void *val,long *cookie);int kvm_io_bus_read (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr,int len,void *val); int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,virt/kvm/kvm_main.c +6
-0@@ -2710,6 +2710,51 @@ int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,return -EOPNOTSUPP ; } +/* kvm_io_bus_write_cookie - called under kvm->slots_lock */ +int kvm_io_bus_write_cookie (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr, +int len,const void *val,long *cookie) +{ +int idx, ret = -EOPNOTSUPP ; +struct kvm_io_bus *bus;
virt/kvm/kvm_main.c int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr, · line 2728
Quoted 10 lines
virt/kvm/kvm_main.c +10
-0+struct kvm_io_range range; + + range = (struct kvm_io_range ) { + .addr = addr, + .len = len, + }; + +if (!cookie) +return -EINVAL ; +
Hmm does this ever trigger?
virt/kvm/kvm_main.c int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr, · line 2732
Quoted 4 lines
virt/kvm/kvm_main.c +4
-0+ bus =srcu_dereference (kvm->buses[bus_idx], &kvm->srcu); + +/* First try the device referenced by *cookie. */ +if (kvm_io_bus_sort_cmp (&range, &bus->range[*cookie]) ==0 )
We need to range-check the cookie.
Quoted 28 lines
virt/kvm/kvm_main.c +25
-0+if (!kvm_iodevice_write (bus->range[*cookie].dev, addr, len, + val)) +return 0 ; + +/* +* *cookie contained garbage; fall back to search and return the +* correct value in *cookie. +*/ + idx =kvm_io_bus_get_first_dev (bus, addr, len); +if (idx <0 ) + goto out; + +while (idx < bus->dev_count && +kvm_io_bus_sort_cmp (&range, &bus->range[idx]) ==0 ) { +if (!kvm_iodevice_write (bus->range[idx].dev, addr, len, val)) { + ret =0 ; + goto out; + } + idx++; + } +out: + *cookie = idx; +return ret; +} +/* kvm_io_bus_read - called under kvm->slots_lock */ int kvm_io_bus_read (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr,int len,void *val)
Re: [PATCH v5 6/6] KVM: s390: Wire up ioeventfd.
Quoted 136 lines · On Tue, 5 Mar 2013 15:19:43 +0200 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 82 lines · Cornelia Huck
Quoted 70 lines · On Mon, 4 Mar 2013 21:07:41 -0300 Marcelo Tosatti <[email protected]> wrote:
Quoted 69 lines · Cornelia Huck
Enable ioeventfd support on s390 and hook up diagnose 500 virtio-ccw notifications.
Signed-off-by Cornelia Huck <[email protected]>--- arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 26 ++++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + 4 files changed, 29 insertions(+), 1 deletion(-)arch/s390/kvm/Kconfig +1
-0@@ -22,6 +22,7 @@ config KVM select PREEMPT_NOTIFIERS select ANON_INODES select HAVE_KVM_CPU_RELAX_INTERCEPT + select HAVE_KVM_EVENTFD ---help--- Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should workarch/s390/kvm/Makefile +1
-1@@ -6,7 +6,7 @@ # it under the terms of the GNU General Public License (version 2 only) # as published by the Free Software Foundation. -common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o) +common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o eventfd.o) ccflags-y := -Ivirt/kvm -Iarch/s390/kvmarch/s390/kvm/diag.c +18
-0@@ -13,6 +13,7 @@#include <linux/kvm.h> #include <linux/kvm_host.h> +#include <asm/virtio-ccw.h> #include "kvm-s390.h" #include "trace.h" #include "trace-s390.h" @@ -104,6 +105,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)return -EREMOTE ; } +static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu) +{ +int ret, idx; + +/* No virtio-ccw notification? Get out quickly. */ +if (!vcpu->kvm->arch.css_support || + (vcpu->run->s.regs.gprs[1 ] !=KVM_S390_VIRTIO_CCW_NOTIFY )) +return -EOPNOTSUPP ; + + idx =srcu_read_lock (&vcpu->kvm->srcu); +/* +* The layout is as follows: +* - gpr 2 contains the subchannel id (passed as addr) +* - gpr 3 contains the virtqueue index (passed as datamatch) +*/ + ret = kvm_io_bus_write(vcpu->kvm, KVM_VIRTIO_CCW_NOTIFY_BUS, + vcpu->run->s.regs.gprs[2],Quoted 8 lines
arch/s390/kvm/diag.c static int __diag_ipl_functions(struct kvm_vcpu *vcpu) · line 130
Quoted 6 lines
arch/s390/kvm/diag.c +6
-0+ 8, &vcpu->run->s.regs.gprs[3]); +srcu_read_unlock (&vcpu->kvm->srcu, idx); +/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ +return ret <0 ? ret :0 ; +} +What about the cookie?
Quoted 4 lines
This is the not-performance-optimized version which went through our testing and which I'd like to see applied. I also have a not-yet-tested patch that adds cookie support (see below), but I'd like to postpone this to 3.10.
I agree it's best to delay this optimization, focus on basic functionality now. As guest with cookie support is compatible with host without said support (all we do really is reserve a gpr), there won't be issues adding it later. Some comments on the draft below.
Quoted 14 lines
commit
8fce4a8a3478252f7ee2eb74d2732673f5911120Author: Cornelia Huck <[email protected]> Date: Tue Feb 26 17:05:01 2013 +0100virtio-ccw: ioeventfd: use cookies
Signed-off-by Cornelia Huck <[email protected]>arch/s390/kvm/diag.c +0
-0@@ -108,22 +108,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu)arch/s390/kvm/diag.c static int __diag_ipl_functions(struct kvm_vcpu *vcpu) · line 118
Quoted 10 lines
arch/s390/kvm/diag.c +2
-0{int ret, idx; +long cookie;/* No virtio-ccw notification? Get out quickly. */ if (!vcpu->kvm->arch.css_support || (vcpu->run->s.regs.gprs[1 ] !=KVM_S390_VIRTIO_CCW_NOTIFY ))return -EOPNOTSUPP ; + cookie = vcpu->run->s.regs.gprs[4 ];Hmm signed so this can become negative.
Quoted 14 lines
arch/s390/kvm/diag.c +5
-3idx =srcu_read_lock (&vcpu->kvm->srcu);/* * The layout is as follows: * - gpr 2 contains the subchannel id (passed as addr) * - gpr 3 contains the virtqueue index (passed as datamatch) +* - gpr 4 contains the index on the bus (optionally) */ - ret =kvm_io_bus_write (vcpu->kvm,KVM_VIRTIO_CCW_NOTIFY_BUS , - vcpu->run->s.regs.gprs[2 ], -8 , &vcpu->run->s.regs.gprs[3 ]); + ret =kvm_io_bus_write_cookie (vcpu->kvm,KVM_VIRTIO_CCW_NOTIFY_BUS , + vcpu->run->s.regs.gprs[2 ], +8 , &vcpu->run->s.regs.gprs[3 ], + &cookie);
Quoted 8 lines
arch/s390/kvm/diag.c static int __diag_ipl_functions(struct kvm_vcpu *vcpu) · line 133
Quoted 4 lines
arch/s390/kvm/diag.c +3
-0srcu_read_unlock (&vcpu->kvm->srcu, idx); +if (!ret) +/* Return cookie in gpr 2. */ + vcpu->run->s.regs.gprs[2 ] = cookie;We should also set gprs[2] to something sane if exiting to userspace so avoid adding overhead for that case.
Probably # of devices + 1 or a negative value.
Negative value would probably be the most obvious.
Quoted 1 line
Are there 32 bit guests? If yes can they pass a negative value?
No, we only support 64 bit guests.
Quoted 35 lines
Quoted 35 lines
arch/s390/kvm/diag.c +0
-0/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ return ret <0 ? ret :0 ; }include/linux/kvm_host.h +2
-0@@ -154,6 +154,8 @@ enum kvm_bus {int kvm_io_bus_write (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr,int len,const void *val); +int kvm_io_bus_write_cookie (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr, +int len,const void *val,long *cookie);int kvm_io_bus_read (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr,int len,void *val); int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,virt/kvm/kvm_main.c +10
-0@@ -2710,6 +2710,51 @@ int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,return -EOPNOTSUPP ; } +/* kvm_io_bus_write_cookie - called under kvm->slots_lock */ +int kvm_io_bus_write_cookie (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr, +int len,const void *val,long *cookie) +{ +int idx, ret = -EOPNOTSUPP ; +struct kvm_io_bus *bus; +struct kvm_io_range range; + + range = (struct kvm_io_range ) { + .addr = addr,
Quoted 8 lines
virt/kvm/kvm_main.c int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr, · line 2728
Quoted 6 lines
virt/kvm/kvm_main.c +6
-0+ .len = len, + }; + +if (!cookie) +return -EINVAL ; +Hmm does this ever trigger?
Probably not.
Quoted 7 lines
virt/kvm/kvm_main.c int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr, · line 2732
Quoted 5 lines
virt/kvm/kvm_main.c +4
-0+ bus =srcu_dereference (kvm->buses[bus_idx], &kvm->srcu); + +/* First try the device referenced by *cookie. */ +if (kvm_io_bus_sort_cmp (&range, &bus->range[*cookie]) ==0 )We need to range-check the cookie.
Yup.
Quoted 30 lines
Quoted 29 lines
virt/kvm/kvm_main.c +25
-0+if (!kvm_iodevice_write (bus->range[*cookie].dev, addr, len, + val)) +return 0 ; + +/* +* *cookie contained garbage; fall back to search and return the +* correct value in *cookie. +*/ + idx =kvm_io_bus_get_first_dev (bus, addr, len); +if (idx <0 ) + goto out; + +while (idx < bus->dev_count && +kvm_io_bus_sort_cmp (&range, &bus->range[idx]) ==0 ) { +if (!kvm_iodevice_write (bus->range[idx].dev, addr, len, val)) { + ret =0 ; + goto out; + } + idx++; + } +out: + *cookie = idx; +return ret; +} +/* kvm_io_bus_read - called under kvm->slots_lock */ int kvm_io_bus_read (struct kvm *kvm,enum kvm_bus bus_idx,gpa_t addr,int len,void *val)
Quoted 16 lines · Cornelia Huck
v5 of the ioeventfd patch set, this time with a proper return code from
__diag_virtio_hypercall(), otherwise unchanged.v4 -> v5:
- Proper return code in__diag_virtio_hypercall()
v3 -> v4:
- Pass cookies in virtio-ccw notify hypercall
- Coding style
v2 -> v3:
- Added a patch exporting the virtio-ccw api and use it for the diagnose implementation.
- Better naming: We're dealing with virtio-ccw notifications only. v1 -> v2:
- Move irqfd initialization from a module init function tokvm_init, eliminating the need for a second module for kvm/s390.
- Usekvm_io_devicefor s390 css devices.
The patches look obviously correct to me.
I take no strong position on whether the userspace interface and the addition of virtio-specific bits in s390 kvm are acceptable, but looks ok to me too.
Quoted 27 lines
Cornelia Huck (5):
KVM: s390: Export virtio-ccw api.
KVM: Initialize irqfd fromkvm_init().
KVM: IntroduceKVM_VIRTIO_CCW_NOTIFY_BUS.
KVM: ioeventfd for virtio-ccw devices.
KVM: s390: Wire up ioeventfd.Michael S. Tsirkin (1):
virtio_ccw: pass a cookie value to kvm hypercallDocumentation/virtual/kvm/api.txt | 8 ++++++++ arch/s390/include/uapi/asm/Kbuild | 1 + arch/s390/include/uapi/asm/virtio-ccw.h | 21 +++++++++++++++++++++ arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 26 ++++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + drivers/s390/kvm/virtio_ccw.c | 16 +++++++++------- include/linux/kvm_host.h | 14 ++++++++++++++ include/uapi/linux/kvm.h | 3 +++ virt/kvm/eventfd.c | 24 +++++++++++++++--------- virt/kvm/kvm_main.c | 6 ++++++ 12 files changed, 106 insertions(+), 17 deletions(-) create mode 100644 arch/s390/include/uapi/asm/virtio-ccw.hSignature · 1 line
1.7.12.4
Marcelo Tosatti
AppliedQuoted 24 lines · Cornelia Huck
v5 of the ioeventfd patch set, this time with a proper return code from
__diag_virtio_hypercall(), otherwise unchanged.v4 -> v5:
- Proper return code in__diag_virtio_hypercall()
v3 -> v4:
- Pass cookies in virtio-ccw notify hypercall
- Coding style
v2 -> v3:
- Added a patch exporting the virtio-ccw api and use it for the diagnose implementation.
- Better naming: We're dealing with virtio-ccw notifications only. v1 -> v2:
- Move irqfd initialization from a module init function tokvm_init, eliminating the need for a second module for kvm/s390.
- Usekvm_io_devicefor s390 css devices.Cornelia Huck (5):
KVM: s390: Export virtio-ccw api.
KVM: Initialize irqfd fromkvm_init().
KVM: IntroduceKVM_VIRTIO_CCW_NOTIFY_BUS.
KVM: ioeventfd for virtio-ccw devices.
KVM: s390: Wire up ioeventfd.
Quoted 2 lines
Michael S. Tsirkin (1):
virtio_ccw: pass a cookie value to kvm hypercall
Applied, thanks.