kvm: Make ioeventfd usable on s390.
Subsystems (4)
v2 cover kvm: Make ioeventfd usable on s390.
Here's the second attempt at implementing ioeventfd for s390.
Rather than the architecture-specific functions used in v1, we now try to integrate with the kvm_io_device infrastructure. Calls to diagnose 500 subcode 3 are now mapped to _write. These devices are created on a new KVM_CSS_BUS when using a new flag KVM_IOEVENTFD_FLAG_CSS. addr and datamatch are (ab)used to contain the subchannel id and the virtqueue.
A drawback is that this interface is not easily extendable should we want to attach other hypercalls or carry more payload.
Another limitation is the limit of 1000 io devices per bus, which we would hit easily with a few hundred devices, but that should be fixable.
v1 -> v2:
- Move irqfd initialization from a module init function to kvm_init, eliminating the need for a second module for kvm/s390.
- Use kvm_io_device for s390 css devices.
Cornelia Huck (4):
KVM: Initialize irqfd from kvm_init().
KVM: Introduce KVM_CSS_BUS.
KVM: ioeventfd for s390 css devices.
KVM: s390: Wire up ioeventfd.
Documentation/virtual/kvm/api.txt | 7 +++++++ arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 25 +++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + include/linux/kvm_host.h | 14 ++++++++++++++ include/uapi/linux/kvm.h | 2 ++ virt/kvm/eventfd.c | 15 ++++++++------- virt/kvm/kvm_main.c | 6 ++++++ 9 files changed, 65 insertions(+), 8 deletions(-)
Signature · 1 line
1.7.12.4
RFC PATCH v2 1/4 KVM: Initialize irqfd from kvm_init().
Currently, eventfd introduces module_init/module_exit functions to initialize/cleanup the irqfd workqueue. This only works, however, if no other module_init/module_exit functions are built into the same module.
Let's just move the initialization and cleanup to kvm_init and kvm_exit. This way, it is also clearer where kvm startup may fail.
--- include/linux/kvm_host.h | 13 +++++++++++++ virt/kvm/eventfd.c | 7 ++----- virt/kvm/kvm_main.c | 6 ++++++ 3 files changed, 21 insertions(+), 5 deletions(-)
include/linux/kvm_host.h +13 -0
@@ -423,6 +423,19 @@ void kvm_vcpu_uninit(struct kvm_vcpu *vcpu);int __must_check vcpu_load (struct kvm_vcpu *vcpu);void vcpu_put (struct kvm_vcpu *vcpu); +#ifdef __KVM_HAVE_IOAPIC +int kvm_irqfd_init (void ); +void kvm_irqfd_exit (void ); +#else +static inline int kvm_irqfd_init (void ) +{ +return 0 ; +} + +static inline void kvm_irqfd_exit (void ) +{ +} +#endif int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align,struct module *module);void kvm_exit (void );
virt/kvm/eventfd.c +2 -5
@@ -544,7 +544,7 @@ void kvm_irq_routing_update(struct kvm *kvm,* aggregated from all vm* instances. We need our own isolated single-thread * queue to prevent deadlock against flushing the normal work-queue. */ -static int __init irqfd_module_init (void ) +int kvm_irqfd_init (void ) { irqfd_cleanup_wq =create_singlethread_workqueue ("kvm-irqfd-cleanup" );if (!irqfd_cleanup_wq)
@@ -553,13 +553,10 @@ static int __init irqfd_module_init(void) return 0; } -static void __exit irqfd_module_exit(void) +void kvm_irqfd_exit (void ) {destroy_workqueue (irqfd_cleanup_wq); } - -module_init (irqfd_module_init); -module_exit (irqfd_module_exit);#endif /*
virt/kvm/kvm_main.c +6 -0
@@ -2920,6 +2920,9 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,int r;int cpu; + r =kvm_irqfd_init (); +if (r) + goto out_irqfd; r =kvm_arch_init (opaque);if (r) goto out_fail;
@@ -3000,6 +3003,8 @@ out_free_0a: out_free_0:kvm_arch_exit (); out_fail: +kvm_irqfd_exit (); +out_irqfd:return r; }EXPORT_SYMBOL_GPL (kvm_init);
@@ -3016,6 +3021,7 @@ void kvm_exit(void)on_each_cpu (hardware_disable_nolock,NULL ,1 );kvm_arch_hardware_unsetup ();kvm_arch_exit (); +kvm_irqfd_exit ();free_cpumask_var (cpus_hardware_enabled); }EXPORT_SYMBOL_GPL (kvm_exit);
Signature · 1 line
1.7.12.4
RFC PATCH v2 2/4 KVM: Introduce KVM_CSS_BUS.
In reply to Cornelia HuckAdd a new bus type for s390 css kvm io devices.
--- include/linux/kvm_host.h | 1 + 1 file changed, 1 insertion(+)
include/linux/kvm_host.h +1 -0
@@ -148,6 +148,7 @@ struct kvm_io_bus {enum kvm_bus {KVM_MMIO_BUS ,KVM_PIO_BUS , +KVM_CSS_BUS ,KVM_NR_BUSES };
Signature · 1 line
1.7.12.4
Re: [RFC PATCH v2 2/4] KVM: Introduce KVM_CSS_BUS.
Quoted 12 lines · Cornelia Huck
Add a new bus type for s390 css kvm io devices.
Signed-off-by Cornelia Huck <[email protected]>--- include/linux/kvm_host.h | 1 + 1 file changed, 1 insertion(+)include/linux/kvm_host.h +0
-0
include/linux/kvm_host.h struct kvm_io_bus { · line 151
Quoted 4 lines
include/linux/kvm_host.h +1
-0@@ -148,6 +148,7 @@ struct kvm_io_bus {enum kvm_bus {KVM_MMIO_BUS ,KVM_PIO_BUS , +KVM_CSS_BUS ,
so maybe KVM_S390_VIRTIO_CCW_NOTIFY_BUS
to make the fact it's s390 and virtio specific, explicit?
Quoted 5 lines
include/linux/kvm_host.h +0
-0KVM_NR_BUSES };Signature · 1 line
1.7.12.4
Re: [RFC PATCH v2 2/4] KVM: Introduce KVM_CSS_BUS.
Quoted 20 lines · On Sun, 24 Feb 2013 11:57:25 +0200 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 12 lines · Cornelia Huck
Add a new bus type for s390 css kvm io devices.
Signed-off-by Cornelia Huck <[email protected]>--- include/linux/kvm_host.h | 1 + 1 file changed, 1 insertion(+)include/linux/kvm_host.h +0
-0include/linux/kvm_host.h struct kvm_io_bus { · line 151
Quoted 4 lines
include/linux/kvm_host.h +1
-0@@ -148,6 +148,7 @@ struct kvm_io_bus {enum kvm_bus {KVM_MMIO_BUS ,KVM_PIO_BUS , +KVM_CSS_BUS ,so maybe
KVM_S390_VIRTIO_CCW_NOTIFY_BUS
Quoted 1 line
to make the fact it's s390 and virtio specific, explicit?
Hm, I don't really see a need to point out that this is s390-specific, so I think I'd prefer KVM_VIRTIO_CCW_NOTIFY_BUS.
Quoted 7 lines
Quoted 6 lines
include/linux/kvm_host.h +0
-0KVM_NR_BUSES };Signature · 1 line
1.7.12.4
RFC PATCH v2 3/4 KVM: ioeventfd for s390 css devices.
In reply to Cornelia HuckEnhance KVM_IOEVENTFD with a new flag that allows to attach to s390 css devices.
--- Documentation/virtual/kvm/api.txt | 7 +++++++ include/uapi/linux/kvm.h | 2 ++ virt/kvm/eventfd.c | 8 ++++++-- 3 files changed, 15 insertions(+), 2 deletions(-)
Documentation/virtual/kvm/api.txt +7 -0
@@ -1468,15 +1468,22 @@ struct kvm_ioeventfd { __u8 pad[36]; }; +For the special case of s390 css devices, the ioevent is matched to a +subchannel/virtqueue tuple instead. + The following flags are defined: #define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch) #define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio) #define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign) +#define KVM_IOEVENTFD_FLAG_CSS (1 << kvm_ioeventfd_flag_nr_css) If datamatch flag is set, the event will be signaled only if the written value to the registered address is equal to datamatch in struct kvm_ioeventfd. +For css devices, addr contains the subchannel id and datamatch the virtqueue +index. + 4.60 KVM_DIRTY_TLB
include/uapi/linux/kvm.h +2 -0
@@ -448,12 +448,14 @@ enum { kvm_ioeventfd_flag_nr_datamatch, kvm_ioeventfd_flag_nr_pio, kvm_ioeventfd_flag_nr_deassign, + kvm_ioeventfd_flag_nr_css, kvm_ioeventfd_flag_nr_max, };#define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch)#define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio)#define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign) +#define KVM_IOEVENTFD_FLAG_CSS (1 << kvm_ioeventfd_flag_nr_css)#define KVM_IOEVENTFD_VALID_FLAG_MASK ((1 << kvm_ioeventfd_flag_nr_max) - 1)
virt/kvm/eventfd.c +6 -2
@@ -679,11 +679,13 @@ static int kvm_assign_ioeventfd(struct kvm *kvm, struct kvm_ioeventfd *args) { int pio = args->flags & KVM_IOEVENTFD_FLAG_PIO; -enum kvm_bus bus_idx = pio ?KVM_PIO_BUS :KVM_MMIO_BUS ; +int css = args->flags &KVM_IOEVENTFD_FLAG_CSS ; +enum kvm_bus bus_idx;struct _ioeventfd *p;struct eventfd_ctx *eventfd;int ret; + bus_idx = pio ?KVM_PIO_BUS : css ?KVM_CSS_BUS :KVM_MMIO_BUS ;/* must be natural-word sized */ switch (args->len) {case 1 :
@@ -759,11 +761,13 @@ static int kvm_deassign_ioeventfd(struct kvm *kvm, struct kvm_ioeventfd *args) { int pio = args->flags & KVM_IOEVENTFD_FLAG_PIO; -enum kvm_bus bus_idx = pio ?KVM_PIO_BUS :KVM_MMIO_BUS ; +int css = args->flags &KVM_IOEVENTFD_FLAG_CSS ; +enum kvm_bus bus_idx;struct _ioeventfd *p, *tmp;struct eventfd_ctx *eventfd;int ret = -ENOENT ; + bus_idx = pio ?KVM_PIO_BUS : css ?KVM_CSS_BUS :KVM_MMIO_BUS ; eventfd =eventfd_ctx_fdget (args->fd);if (IS_ERR (eventfd))return PTR_ERR (eventfd);
Signature · 1 line
1.7.12.4
Re: [RFC PATCH v2 3/4] KVM: ioeventfd for s390 css devices.
Quoted 43 lines · Cornelia Huck
Enhance
KVM_IOEVENTFDwith a new flag that allows to attach to s390 css devices.Signed-off-by Cornelia Huck <[email protected]>--- Documentation/virtual/kvm/api.txt | 7 +++++++ include/uapi/linux/kvm.h | 2 ++ virt/kvm/eventfd.c | 8 ++++++-- 3 files changed, 15 insertions(+), 2 deletions(-)Documentation/virtual/kvm/api.txt +7
-0@@ -1468,15 +1468,22 @@ struct kvm_ioeventfd { __u8 pad[36]; }; +For the special case of s390 css devices, the ioevent is matched to a +subchannel/virtqueue tuple instead. + The following flags are defined: #define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch) #define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio) #define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign) +#define KVM_IOEVENTFD_FLAG_CSS (1 << kvm_ioeventfd_flag_nr_css) If datamatch flag is set, the event will be signaled only if the written value to the registered address is equal to datamatch in struct kvm_ioeventfd. +For css devices, addr contains the subchannel id and datamatch the virtqueue +index. + 4.60 KVM_DIRTY_TLBinclude/uapi/linux/kvm.h +0
-0@@ -448,12 +448,14 @@ enum { kvm_ioeventfd_flag_nr_datamatch,
include/uapi/linux/kvm.h enum { · line 458
Quoted 10 lines
include/uapi/linux/kvm.h +2
-0kvm_ioeventfd_flag_nr_pio, kvm_ioeventfd_flag_nr_deassign, + kvm_ioeventfd_flag_nr_css, kvm_ioeventfd_flag_nr_max, };#define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch)#define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio)#define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign) +#define KVM_IOEVENTFD_FLAG_CSS (1 << kvm_ioeventfd_flag_nr_css)
So let's explicitly name is KVM_IOEVENTFD_S390_VIRTIO_CCW or even KVM_IOEVENTFD_S390_VIRTIO_CCW_NOTIFY?
Also, maybe we want to add KVM_IOEVENTFD_S390_VIRTIO_NOTIFY as well?
Quoted 39 lines
include/uapi/linux/kvm.h +0
-0#define KVM_IOEVENTFD_VALID_FLAG_MASK ((1 << kvm_ioeventfd_flag_nr_max) - 1)virt/kvm/eventfd.c +6
-2@@ -679,11 +679,13 @@ static int kvm_assign_ioeventfd(struct kvm *kvm, struct kvm_ioeventfd *args) { int pio = args->flags & KVM_IOEVENTFD_FLAG_PIO; -enum kvm_bus bus_idx = pio ?KVM_PIO_BUS :KVM_MMIO_BUS ; +int css = args->flags &KVM_IOEVENTFD_FLAG_CSS ; +enum kvm_bus bus_idx;struct _ioeventfd *p;struct eventfd_ctx *eventfd;int ret; + bus_idx = pio ?KVM_PIO_BUS : css ?KVM_CSS_BUS :KVM_MMIO_BUS ;/* must be natural-word sized */ switch (args->len) {case 1 :@@ -759,11 +761,13 @@ static int kvm_deassign_ioeventfd(struct kvm *kvm, struct kvm_ioeventfd *args) { int pio = args->flags & KVM_IOEVENTFD_FLAG_PIO; -enum kvm_bus bus_idx = pio ?KVM_PIO_BUS :KVM_MMIO_BUS ; +int css = args->flags &KVM_IOEVENTFD_FLAG_CSS ; +enum kvm_bus bus_idx;struct _ioeventfd *p, *tmp;struct eventfd_ctx *eventfd;int ret = -ENOENT ; + bus_idx = pio ?KVM_PIO_BUS : css ?KVM_CSS_BUS :KVM_MMIO_BUS ; eventfd =eventfd_ctx_fdget (args->fd);if (IS_ERR (eventfd))return PTR_ERR (eventfd);Signature · 1 line
1.7.12.4
Re: [RFC PATCH v2 3/4] KVM: ioeventfd for s390 css devices.
Quoted 50 lines · On Sun, 24 Feb 2013 11:47:50 +0200 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 49 lines · Cornelia Huck
Enhance
KVM_IOEVENTFDwith a new flag that allows to attach to s390 css devices.Signed-off-by Cornelia Huck <[email protected]>--- Documentation/virtual/kvm/api.txt | 7 +++++++ include/uapi/linux/kvm.h | 2 ++ virt/kvm/eventfd.c | 8 ++++++-- 3 files changed, 15 insertions(+), 2 deletions(-)Documentation/virtual/kvm/api.txt +7
-0@@ -1468,15 +1468,22 @@ struct kvm_ioeventfd { __u8 pad[36]; }; +For the special case of s390 css devices, the ioevent is matched to a +subchannel/virtqueue tuple instead. + The following flags are defined: #define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch) #define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio) #define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign) +#define KVM_IOEVENTFD_FLAG_CSS (1 << kvm_ioeventfd_flag_nr_css) If datamatch flag is set, the event will be signaled only if the written value to the registered address is equal to datamatch in struct kvm_ioeventfd. +For css devices, addr contains the subchannel id and datamatch the virtqueue +index. + 4.60 KVM_DIRTY_TLBinclude/uapi/linux/kvm.h +1
-0@@ -448,12 +448,14 @@ enum { kvm_ioeventfd_flag_nr_datamatch, kvm_ioeventfd_flag_nr_pio, kvm_ioeventfd_flag_nr_deassign, + kvm_ioeventfd_flag_nr_css, kvm_ioeventfd_flag_nr_max, };
Quoted 7 lines
include/uapi/linux/kvm.h enum { · line 458
Quoted 4 lines
include/uapi/linux/kvm.h +1
-0#define KVM_IOEVENTFD_FLAG_DATAMATCH (1 << kvm_ioeventfd_flag_nr_datamatch)#define KVM_IOEVENTFD_FLAG_PIO (1 << kvm_ioeventfd_flag_nr_pio)#define KVM_IOEVENTFD_FLAG_DEASSIGN (1 << kvm_ioeventfd_flag_nr_deassign) +#define KVM_IOEVENTFD_FLAG_CSS (1 << kvm_ioeventfd_flag_nr_css)So let's explicitly name is
KVM_IOEVENTFD_S390_VIRTIO_CCWor evenKVM_IOEVENTFD_S390_VIRTIO_CCW_NOTIFY?
How about KVM_IOEVENTFD_FLAG_VIRTIO_CCW_NOTIFY?
Quoted 1 line
Also, maybe we want to add
KVM_IOEVENTFD_S390_VIRTIO_NOTIFYas well?
No, since s390-virtio is a dead end.
Quoted 41 lines
Quoted 40 lines
include/uapi/linux/kvm.h +0
-0#define KVM_IOEVENTFD_VALID_FLAG_MASK ((1 << kvm_ioeventfd_flag_nr_max) - 1)virt/kvm/eventfd.c +6
-2@@ -679,11 +679,13 @@ static int kvm_assign_ioeventfd(struct kvm *kvm, struct kvm_ioeventfd *args) { int pio = args->flags & KVM_IOEVENTFD_FLAG_PIO; -enum kvm_bus bus_idx = pio ?KVM_PIO_BUS :KVM_MMIO_BUS ; +int css = args->flags &KVM_IOEVENTFD_FLAG_CSS ; +enum kvm_bus bus_idx;struct _ioeventfd *p;struct eventfd_ctx *eventfd;int ret; + bus_idx = pio ?KVM_PIO_BUS : css ?KVM_CSS_BUS :KVM_MMIO_BUS ;/* must be natural-word sized */ switch (args->len) {case 1 :@@ -759,11 +761,13 @@ static int kvm_deassign_ioeventfd(struct kvm *kvm, struct kvm_ioeventfd *args) { int pio = args->flags & KVM_IOEVENTFD_FLAG_PIO; -enum kvm_bus bus_idx = pio ?KVM_PIO_BUS :KVM_MMIO_BUS ; +int css = args->flags &KVM_IOEVENTFD_FLAG_CSS ; +enum kvm_bus bus_idx;struct _ioeventfd *p, *tmp;struct eventfd_ctx *eventfd;int ret = -ENOENT ; + bus_idx = pio ?KVM_PIO_BUS : css ?KVM_CSS_BUS :KVM_MMIO_BUS ; eventfd =eventfd_ctx_fdget (args->fd);if (IS_ERR (eventfd))return PTR_ERR (eventfd);Signature · 1 line
1.7.12.4
RFC PATCH v2 4/4 KVM: s390: Wire up ioeventfd.
In reply to Cornelia HuckEnable ioeventfd support on s390 and hook up diagnose 500 virtio-ccw notifications.
--- arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 25 +++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + 4 files changed, 28 insertions(+), 1 deletion(-)
arch/s390/kvm/Kconfig +1 -0
@@ -22,6 +22,7 @@ config KVM select PREEMPT_NOTIFIERS select ANON_INODES select HAVE_KVM_CPU_RELAX_INTERCEPT + select HAVE_KVM_EVENTFD ---help--- Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should work
arch/s390/kvm/Makefile +1 -1
@@ -6,7 +6,7 @@ # it under the terms of the GNU General Public License (version 2 only) # as published by the Free Software Foundation. -common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o) +common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o eventfd.o) ccflags-y := -Ivirt/kvm -Iarch/s390/kvm
arch/s390/kvm/diag.c +25 -0
@@ -104,6 +104,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)return -EREMOTE ; } +static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu) +{ +int ret, idx; + +/* No virtio-ccw notification? Get out quickly. */ +if (!vcpu->kvm->arch.css_support || + (vcpu->run->s.regs.gprs[1 ] !=3 )) +return -EOPNOTSUPP ; + + idx =srcu_read_lock (&vcpu->kvm->srcu); +/* +* The layout is as follows: +* - gpr 2 contains the subchannel id (passed as addr) +* - gpr 3 contains the virtqueue index (passed as datamatch) +*/ + ret =kvm_io_bus_write (vcpu->kvm,KVM_CSS_BUS , + vcpu->run->s.regs.gprs[2 ], +8 , &vcpu->run->s.regs.gprs[3 ]); +srcu_read_unlock (&vcpu->kvm->srcu, idx); +/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ +return ret; +} +int kvm_s390_handle_diag (struct kvm_vcpu *vcpu) {int code = (vcpu->arch.sie_block->ipb &0xfff0000 ) >>16 ;
@@ -118,6 +141,8 @@ int kvm_s390_handle_diag(struct kvm_vcpu *vcpu)return __diag_time_slice_end_directed (vcpu);case 0x308 :return __diag_ipl_functions (vcpu); +case 0x500 : +return __diag_virtio_hypercall (vcpu);default :return -EOPNOTSUPP ; }
arch/s390/kvm/kvm-s390.c +1 -0
@@ -142,6 +142,7 @@ int kvm_dev_ioctl_check_extension(long ext)case KVM_CAP_ONE_REG :case KVM_CAP_ENABLE_CAP :case KVM_CAP_S390_CSS_SUPPORT : +case KVM_CAP_IOEVENTFD : r =1 ;break ;case KVM_CAP_NR_VCPUS :
Signature · 1 line
1.7.12.4
Re: [RFC PATCH v2 4/4] KVM: s390: Wire up ioeventfd.
Quoted 42 lines · Cornelia Huck
Enable ioeventfd support on s390 and hook up diagnose 500 virtio-ccw notifications.
Signed-off-by Cornelia Huck <[email protected]>--- arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 25 +++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + 4 files changed, 28 insertions(+), 1 deletion(-)arch/s390/kvm/Kconfig +1
-0@@ -22,6 +22,7 @@ config KVM select PREEMPT_NOTIFIERS select ANON_INODES select HAVE_KVM_CPU_RELAX_INTERCEPT + select HAVE_KVM_EVENTFD ---help--- Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should workarch/s390/kvm/Makefile +1
-1@@ -6,7 +6,7 @@ # it under the terms of the GNU General Public License (version 2 only) # as published by the Free Software Foundation. -common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o) +common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o eventfd.o) ccflags-y := -Ivirt/kvm -Iarch/s390/kvmarch/s390/kvm/diag.c +0
-0@@ -104,6 +104,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)return -EREMOTE ;
arch/s390/kvm/diag.c static int __diag_ipl_functions(struct kvm_vcpu *vcpu) · line 114
Quoted 10 lines
arch/s390/kvm/diag.c +8
-0} +static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu) +{ +int ret, idx; + +/* No virtio-ccw notification? Get out quickly. */ +if (!vcpu->kvm->arch.css_support || + (vcpu->run->s.regs.gprs[1 ] !=3 )) +return -EOPNOTSUPP ;
Looks like we have:
arch/s390/include/uapi/asm/kvm_virtio.h:#define KVM_S390_VIRTIO_NOTIFY 0 arch/s390/include/uapi/asm/kvm_virtio.h:#define KVM_S390_VIRTIO_RESET 1
arch/s390/include/uapi/asm/kvm_virtio.h:#define KVM_S390_VIRTIO_SET_STATUS 2
KVM_S390_VIRTIO_CSS_NOTIFY is missing? Strange. Maybe it should be added in that header and included here? Do some guests use it?
Also, do you want to support KVM_S390_VIRTIO_NOTIFY as well?
Quoted 41 lines
arch/s390/kvm/diag.c +17
-0+ + idx =srcu_read_lock (&vcpu->kvm->srcu); +/* +* The layout is as follows: +* - gpr 2 contains the subchannel id (passed as addr) +* - gpr 3 contains the virtqueue index (passed as datamatch) +*/ + ret =kvm_io_bus_write (vcpu->kvm,KVM_CSS_BUS , + vcpu->run->s.regs.gprs[2 ], +8 , &vcpu->run->s.regs.gprs[3 ]); +srcu_read_unlock (&vcpu->kvm->srcu, idx); +/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ +return ret; +} +int kvm_s390_handle_diag (struct kvm_vcpu *vcpu) {int code = (vcpu->arch.sie_block->ipb &0xfff0000 ) >>16 ;@@ -118,6 +141,8 @@ int kvm_s390_handle_diag(struct kvm_vcpu *vcpu)return __diag_time_slice_end_directed (vcpu);case 0x308 :return __diag_ipl_functions (vcpu); +case 0x500 : +return __diag_virtio_hypercall (vcpu);default :return -EOPNOTSUPP ; }arch/s390/kvm/kvm-s390.c +1
-0@@ -142,6 +142,7 @@ int kvm_dev_ioctl_check_extension(long ext)case KVM_CAP_ONE_REG :case KVM_CAP_ENABLE_CAP :case KVM_CAP_S390_CSS_SUPPORT : +case KVM_CAP_IOEVENTFD : r =1 ;break ;case KVM_CAP_NR_VCPUS :Signature · 1 line
1.7.12.4
Re: [RFC PATCH v2 4/4] KVM: s390: Wire up ioeventfd.
Quoted 59 lines · On Sun, 24 Feb 2013 11:45:22 +0200 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 42 lines · Cornelia Huck
Enable ioeventfd support on s390 and hook up diagnose 500 virtio-ccw notifications.
Signed-off-by Cornelia Huck <[email protected]>--- arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 25 +++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + 4 files changed, 28 insertions(+), 1 deletion(-)arch/s390/kvm/Kconfig +1
-0@@ -22,6 +22,7 @@ config KVM select PREEMPT_NOTIFIERS select ANON_INODES select HAVE_KVM_CPU_RELAX_INTERCEPT + select HAVE_KVM_EVENTFD ---help--- Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should workarch/s390/kvm/Makefile +1
-1@@ -6,7 +6,7 @@ # it under the terms of the GNU General Public License (version 2 only) # as published by the Free Software Foundation. -common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o) +common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o eventfd.o) ccflags-y := -Ivirt/kvm -Iarch/s390/kvmarch/s390/kvm/diag.c +0
-0@@ -104,6 +104,29 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)return -EREMOTE ;arch/s390/kvm/diag.c static int __diag_ipl_functions(struct kvm_vcpu *vcpu) · line 114
Quoted 10 lines
arch/s390/kvm/diag.c +8
-0} +static int __diag_virtio_hypercall (struct kvm_vcpu *vcpu) +{ +int ret, idx; + +/* No virtio-ccw notification? Get out quickly. */ +if (!vcpu->kvm->arch.css_support || + (vcpu->run->s.regs.gprs[1 ] !=3 )) +return -EOPNOTSUPP ;Looks like we have:
arch/s390/include/uapi/asm/kvm_virtio.h:#defineKVM_S390_VIRTIO_NOTIFY0 arch/s390/include/uapi/asm/kvm_virtio.h:#defineKVM_S390_VIRTIO_RESET1
arch/s390/include/uapi/asm/kvm_virtio.h:#defineKVM_S390_VIRTIO_SET_STATUS2
Quoted 2 lines
KVM_S390_VIRTIO_CSS_NOTIFYis missing? Strange. Maybe it should be added in that header and included here?
kvm_virtio.h is for the old-style s390-virtio mechanism. Subcode 3 will be included in 3.9.
Quoted 2 lines
Do some guests use it?
Also, do you want to supportKVM_S390_VIRTIO_NOTIFYas well?
We don't want to invest any more effort in the old style s390-virtio transport (other than keeping it working). Any future work will be for virtio-ccw.
Quoted 44 lines
Quoted 43 lines
arch/s390/kvm/diag.c +17
-0+ + idx =srcu_read_lock (&vcpu->kvm->srcu); +/* +* The layout is as follows: +* - gpr 2 contains the subchannel id (passed as addr) +* - gpr 3 contains the virtqueue index (passed as datamatch) +*/ + ret =kvm_io_bus_write (vcpu->kvm,KVM_CSS_BUS , + vcpu->run->s.regs.gprs[2 ], +8 , &vcpu->run->s.regs.gprs[3 ]); +srcu_read_unlock (&vcpu->kvm->srcu, idx); +/* kvm_io_bus_write returns -EOPNOTSUPP if it found no match. */ +return ret; +} +int kvm_s390_handle_diag (struct kvm_vcpu *vcpu) {int code = (vcpu->arch.sie_block->ipb &0xfff0000 ) >>16 ;@@ -118,6 +141,8 @@ int kvm_s390_handle_diag(struct kvm_vcpu *vcpu)return __diag_time_slice_end_directed (vcpu);case 0x308 :return __diag_ipl_functions (vcpu); +case 0x500 : +return __diag_virtio_hypercall (vcpu);default :return -EOPNOTSUPP ; }arch/s390/kvm/kvm-s390.c +1
-0@@ -142,6 +142,7 @@ int kvm_dev_ioctl_check_extension(long ext)case KVM_CAP_ONE_REG :case KVM_CAP_ENABLE_CAP :case KVM_CAP_S390_CSS_SUPPORT : +case KVM_CAP_IOEVENTFD : r =1 ;break ;case KVM_CAP_NR_VCPUS :Signature · 1 line
1.7.12.4
Quoted 8 lines · Cornelia Huck
Here's the second attempt at implementing ioeventfd for s390.
Rather than the architecture-specific functions used in v1, we now try to integrate with the
kvm_io_deviceinfrastructure. Calls to diagnose 500 subcode 3 are now mapped to_write. These devices are created on a newKVM_CSS_BUSwhen using a new flagKVM_IOEVENTFD_FLAG_CSS. addr and datamatch are (ab)used to contain the subchannel id and the virtqueue.
OK so a question: qemu has:
#define KVM_S390_VIRTIO_NOTIFY 0 #define KVM_S390_VIRTIO_RESET 1 #define KVM_S390_VIRTIO_SET_STATUS 2 #define KVM_S390_VIRTIO_CCW_NOTIFY 3
So this is really s390_virtio_ccw not generally css? Maybe should say so in all headers.
Quoted 31 lines
we want to attach other hypercalls or carry more payload.
Another limitation is the limit of 1000 io devices per bus, which we would hit easily with a few hundred devices, but that should be fixable.
v1 -> v2:
- Move irqfd initialization from a module init function tokvm_init, eliminating the need for a second module for kvm/s390.
- Usekvm_io_devicefor s390 css devices.Cornelia Huck (4):
KVM: Initialize irqfd fromkvm_init().
KVM: IntroduceKVM_CSS_BUS.
KVM: ioeventfd for s390 css devices.
KVM: s390: Wire up ioeventfd.Documentation/virtual/kvm/api.txt | 7 +++++++ arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 25 +++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + include/linux/kvm_host.h | 14 ++++++++++++++ include/uapi/linux/kvm.h | 2 ++ virt/kvm/eventfd.c | 15 ++++++++------- virt/kvm/kvm_main.c | 6 ++++++ 9 files changed, 65 insertions(+), 8 deletions(-)Signature · 1 line
1.7.12.4
Quoted 1 line · Cornelia Huck
Here's the second attempt at implementing ioeventfd for s390.
The patchset looks fine overall.
Minor comments and questions below.
Quoted 10 lines
Rather than the architecture-specific functions used in v1, we now try to integrate with the
kvm_io_deviceinfrastructure. Calls to diagnose 500 subcode 3 are now mapped to_write. These devices are created on a newKVM_CSS_BUSwhen using a new flagKVM_IOEVENTFD_FLAG_CSS. addr and datamatch are (ab)used to contain the subchannel id and the virtqueue.A drawback is that this interface is not easily extendable should we want to attach other hypercalls or carry more payload.
Under s390 kvm_hypercallX already uses diagnose 500 so that seems fine. If you want to make it more generic and support more subcodes, I think you'll have to pass an extra u64 field:
to bus to both avoid overflowing int value and avoid ugly bus-specific hacks in generic code.
Will we ever need that? Code using subcode 3 does not yet seem to be upstream in 3.8 so maybe yes, but you decide. An alternative is to add new bus types when kvm needs to handle new subcodes. So e.g. KVM_BUS_S390_VIRTIO_CCW_NOTIFY and KVM_BUS_S390_VIRTIO_NOTIFY ?
You decide, I'm fine with either approach.
More minor comments and questions in response to individual patches.
Quoted 29 lines
Another limitation is the limit of 1000 io devices per bus, which we would hit easily with a few hundred devices, but that should be fixable.
v1 -> v2:
- Move irqfd initialization from a module init function tokvm_init, eliminating the need for a second module for kvm/s390.
- Usekvm_io_devicefor s390 css devices.Cornelia Huck (4):
KVM: Initialize irqfd fromkvm_init().
KVM: IntroduceKVM_CSS_BUS.
KVM: ioeventfd for s390 css devices.
KVM: s390: Wire up ioeventfd.Documentation/virtual/kvm/api.txt | 7 +++++++ arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 25 +++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + include/linux/kvm_host.h | 14 ++++++++++++++ include/uapi/linux/kvm.h | 2 ++ virt/kvm/eventfd.c | 15 ++++++++------- virt/kvm/kvm_main.c | 6 ++++++ 9 files changed, 65 insertions(+), 8 deletions(-)Signature · 1 line
1.7.12.4
Quoted 5 lines · On Sun, 24 Feb 2013 11:56:39 +0200 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 1 line · Cornelia Huck
Here's the second attempt at implementing ioeventfd for s390.
The patchset looks fine overall.
Minor comments and questions below.
Cool, thanks for reviewing.
Quoted 12 lines
Quoted 11 lines
Rather than the architecture-specific functions used in v1, we now try to integrate with the
kvm_io_deviceinfrastructure. Calls to diagnose 500 subcode 3 are now mapped to_write. These devices are created on a newKVM_CSS_BUSwhen using a new flagKVM_IOEVENTFD_FLAG_CSS. addr and datamatch are (ab)used to contain the subchannel id and the virtqueue.A drawback is that this interface is not easily extendable should we want to attach other hypercalls or carry more payload.
Quoted 8 lines
Under s390
kvm_hypercallXalready uses diagnose 500 so that seems fine. If you want to make it more generic and support more subcodes, I think you'll have to pass an extra u64 field:
to bus to both avoid overflowing int value and avoid ugly bus-specific hacks in generic code.Will we ever need that? Code using subcode 3 does not yet seem to be upstream in 3.8 so maybe yes, but you decide.
Subcode 3 will be in 3.9.
Quoted 3 lines
An alternative is to add new bus types when kvm needs to handle new subcodes. So e.g.
KVM_BUS_S390_VIRTIO_CCW_NOTIFYandKVM_BUS_S390_VIRTIO_NOTIFY?
I think I'll fall back to a new bus type should we ever need a new notification type - the less strange hacks, the better.
Quoted 35 lines
You decide, I'm fine with either approach.
More minor comments and questions in response to individual patches.
Quoted 29 lines
Another limitation is the limit of 1000 io devices per bus, which we would hit easily with a few hundred devices, but that should be fixable.
v1 -> v2:
- Move irqfd initialization from a module init function tokvm_init, eliminating the need for a second module for kvm/s390.
- Usekvm_io_devicefor s390 css devices.Cornelia Huck (4):
KVM: Initialize irqfd fromkvm_init().
KVM: IntroduceKVM_CSS_BUS.
KVM: ioeventfd for s390 css devices.
KVM: s390: Wire up ioeventfd.Documentation/virtual/kvm/api.txt | 7 +++++++ arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 25 +++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + include/linux/kvm_host.h | 14 ++++++++++++++ include/uapi/linux/kvm.h | 2 ++ virt/kvm/eventfd.c | 15 ++++++++------- virt/kvm/kvm_main.c | 6 ++++++ 9 files changed, 65 insertions(+), 8 deletions(-)Signature · 1 line
1.7.12.4